Computerized Cognitive Testing for Use in Clinical Trials: A Comparison of the NIH Toolbox and Cogstate C3 Batteries

Abstract

Background: As prevention trials for Alzheimer's disease move into asymptomatic populations, identifying older individuals who manifest the earliest cognitive signs of Alzheimer's disease is critical. Computerized cognitive testing has the potential to replace current gold standard paper and pencil measures and may be a more efficient means of assessing cognition. However, more empirical evidence about the comparability of novel computerized batteries to paper and pencil measures is required.

Objectives: To determine whether two computerized IPad batteries, the NIH Toolbox Cognition Battery and Cogstate-C3, similarly predict subtle cognitive impairment identified using the Preclinical Alzheimer Cognitive Composite (PACC).

Design, setting, participants: A pilot sample of 50 clinically normal older adults (Mage=68.5 years±7.6, 45% non-Caucasian) completed the PACC assessment, and the NIH Toolbox and Cogstate-C3 at research centers of Massachusetts General and Brigham and Women's Hospitals. Participants made 3-4 in-clinic visits, receiving the PACC first, then the NIH Toolbox, and finally the Cogstate-C3.>= 0.5SD), versus subtle cognitive impairment (<0.5SD). Composites for each computerized battery were created using principle components analysis, and compared with the PACC using non-parametric Spearman correlations. Logistic regression analyses were used to determine which composite was best able to classify subtle cognitive impairment from typical performance.

Results: The NIH Toolbox formed one composite and exhibited the strongest within-battery alignment, while the Cogstate-C3 formed two distinct composites (Learning-Memory and Processing Speed-Attention). The NIH Toolbox and C3 Learning-Memory composites exhibited positive correlations with the PACC (ρ=0.49, p<0.001; ρ=0.58, p<0.001, respectively), but not the C3 Processing Speed-Attention composite, ρ=-0.18, p=0.22. The C3 Learning-Memory was the only composite that classified subtle cognitive impairment, and demonstrated the greatest sensitivity (62%) and specificity (81%) for that subtle cognitive impairment.

Conclusions: Preliminary findings suggest that the NIH Toolbox has the advantage of showing the strongest overall clustering and alignment with standardized paper-and-pencil tasks. By contrast, Learning-Memory tasks within the Cogstate-C3 battery have the greatest potential to identify cross-sectional, subtle cognitive impairment as defined by the PACC.

Keywords: Cognition; aging; computerized testing; neuropsychology.

Figures

Figure 1

Visualization of clusters using PCA of NIHTB-CB with C3 tasks. Arrows indicate the loading coefficients of each variable of interest

Figure 2

ROCs for the NIHTB-CB and Cogstate C3 composites, and the C3 FNLT task alone to distinguish between high and low PACC performance (Blue = C3 Learning-Memory, Red = NIHTB-CB, Green = C3 Processing Speed-Attention, Grey = C3 FNLT, Black-dash = NIH PSMT)

Figure 3

Scatterplot of association between NIHTB-CB and Cogstate C3 battery, with slopes estimating group effect of high and low PACC performance

Figure 4

Diagrammatic representation of each composite arising from the Cogstate C3 and NIHTH-CB computerized batteries, and their corresponding tests. Each composite is also attached to an odds ratio (OR) which represents the ability of each composite to distinguish between typical and subtly impaired PACC performance. The pink boxes denote the tasks that were most contributory to the variance explained in the logistic regression model