Measurement Precision and Efficiency of Computerized Adaptive Testing for the Activities-specific Balance Confidence Scale in People With Stroke

Abstract

Objective: Administrative burden often prevents clinical assessment of balance confidence in people with stroke. A computerized adaptive test (CAT) version of the Activities-specific Balance Confidence Scale (ABC CAT) can dramatically reduce this burden. The objective of this study was to test balance confidence measurement precision and efficiency in people with stroke with an ABC CAT.

Methods: We conducted a retrospective, cross-sectional, simulation study with data from 406 adults approximately 2 months post-stroke in the Locomotor-Experience Applied Post-Stroke trial. Item parameters for CAT calibration were estimated with the Rasch model using a random sample of participants (n = 203). Computer simulation was used with response data from the remaining 203 participants to evaluate the ABC CAT algorithm under varying stopping criteria. We compared estimated levels of balance confidence from each simulation to actual levels predicted from the Rasch model (Pearson correlations and mean standard error [SE]).

Results: Results from simulations with number of items as a stopping criterion strongly correlated with actual ABC scores (full item, r = 1, 12-item, r = 0.994; 8-item, r = 0.98; 4-item, r = 0.929). Mean SE increased with decreasing number of items administered (full item, SE = 0.31; 12 item, SE = 0.33; 8 item, SE = 0.38; 4 item, SE = 0.49). A precision-based stopping rule (mean SE = 0.5) also strongly correlated with actual ABC scores (r = 0.941) and optimized the relationship between number of items administrated with precision (mean number of items 4.37, range [4-9]).

Conclusion: An ABC CAT can determine accurate and precise measures of balance confidence in people with stroke with as few as 4 items. Individuals with lower balance confidence may require a greater number of items (up to 9) which could be attributed to the Locomotor-Experience Applied Post-Stroke trial excluding more functionally impaired persons.

Impact: Computerized adaptive testing can drastically reduce the ABC test's administration time while maintaining accuracy and precision. This should greatly enhance clinical utility, facilitating adoption of clinical practice guidelines in stroke rehabilitation.

Lay summary: If you have had a stroke, your physical therapist will likely test your balance confidence. A CAT version of the ABC scale can accurately identify balance with as few as 4 questions, which takes much less time.

Trial registration: ClinicalTrials.gov NCT00243919.

Keywords: Balance; Measurement—Applied; Stroke.

© The Author(s) 2021. Published by Oxford University Press on behalf of the American Physical Therapy Association. All rights reserved. For permissions, please email: journals.permissions@oup.com.

Figures

Figure 1

Example audit trails from simulated responses to the full item ABC CAT. Audit trails in this figure show the estimated balance confidence (theta) measurement scores from the full item ABC CAT and associated 95% CI (theta ±1.96 * SE) for an individual after they responded to each presented item. Audit trails are presented for a representative individual (A) near the sample mean, (B) above the sample mean, and (C) below the sample mean with respect to their actual balance confidence score, represented by a red dash line. The order of the items presented by the ABC CAT is listed above each audit trail. In all 3 cases, the individual’s balance confidence was correctly estimated between 3 and 5 items. The audit trails demonstrate the diminishing return in measurement precision (mean SE) associated with increasing the number of administrated items. In these cases, the magnitude of improved precision appears negligible after and individual responds to 5 items.

Figure 2

CAT estimated balance confidence vs actual balance confidence. Plotted are the estimated balance confidence scores (theta) from the ABC CAT for each individual against their actual balance confidence score. Estimated thetas from the ABC CAT were highly correlated (r = 0.941) with actual balance confidence.

Figure 3

Number of items administered for each individual based on estimated balance confidence. This figure plots the number of items administered by the ABC CAT using the precision-based stopping rule for each individual based on their estimated balance confidence (theta). The mean balance confidence of the sample is anchored at 0. The number of items needed to reach a mean SE of 0.5 logits when estimating balance confidence increases as an individual’s balance confidence is further from the population mean.