Validity, reliability, and psychometric properties of a computerized, cognitive assessment test (Cognivue®)

Diego Cahn-Hidalgo, Paul W Estes, Reina Benabou, Diego Cahn-Hidalgo, Paul W Estes, Reina Benabou

Abstract

Background: Cognitive issues such as Alzheimer's disease and other dementias confer a substantial negative impact. Problems relating to sensitivity, subjectivity, and inherent bias can limit the usefulness of many traditional methods of assessing cognitive impairment.

Aim: To determine cut-off scores for classification of cognitive impairment, and assess Cognivue® safety and efficacy in a large validation study.

Methods: Adults (age 55-95 years) at risk for age-related cognitive decline or dementia were invited via posters and email to participate in two cohort studies conducted at various outpatient clinics and assisted- and independent-living facilities. In the cut-off score determination study (n = 92), optimization analyses by positive percent agreement (PPA) and negative percent agreement (NPA), and by accuracy and error bias were conducted. In the clinical validation study (n = 401), regression, rank linear regression, and factor analyses were conducted. Participants in the clinical validation study also completed other neuropsychological tests.

Results: For the cut-off score determination study, 92 participants completed St. Louis University Mental Status (SLUMS, reference standard) and Cognivue® tests. Analyses showed that SLUMS cut-off scores of < 21 (impairment) and > 26 (no impairment) corresponded to Cognivue® scores of 54.5 (NPA = 0.92; PPA = 0.64) and 78.5 (NPA = 0.5; PPA = 0.79), respectively. Therefore, conservatively, Cognivue® scores of 55-64 corresponded to impairment, and 74-79 to no impairment. For the clinical validation study, 401 participants completed ≥ 1 testing session, and 358 completed 2 sessions 1-2 wk apart. Cognivue® classification scores were validated, demonstrating good agreement with SLUMS scores (weighted κ 0.57; 95%CI: 0.50-0.63). Reliability analyses showed similar scores across repeated testing for Cognivue® (R 2 = 0.81; r = 0.90) and SLUMS (R 2 = 0.67; r = 0.82). Psychometric validity of Cognivue® was demonstrated vs. traditional neuropsychological tests. Scores were most closely correlated with measures of verbal processing, manual dexterity/speed, visual contrast sensitivity, visuospatial/executive function, and speed/sequencing.

Conclusion: Cognivue® scores ≤ 50 avoid misclassification of impairment, and scores ≥ 75 avoid misclassification of unimpairment. The validation study demonstrates good agreement between Cognivue® and SLUMS; superior reliability; and good psychometric validity.

Keywords: Cognitive screening test; Dementia; Memory; Motor control; Perceptual processing; Visual salience.

Conflict of interest statement

Conflict-of-interest statement: Dr. Cahn-Hidalgo has acted as a consultant and speaker for Cognivue Inc. Estes PW and Dr. Benabou are employees of Cognivue Inc.

©The Author(s) 2019. Published by Baishideng Publishing Group Inc. All rights reserved.

Figures

Figure 1
Figure 1
Cognivue® quantitative sub-batteries. A: Visuomotor testing (motor control, visual salience); B: Perceptual processing (letter, word, shape, motion); C: Memory testing (letter, word, shape, motion).
Figure 2
Figure 2
Scatterplot showing the St Louis University Mental Status and Cognivue® scores for the 92 study participants. The table to the left of the scatter plot provides a key for relating the plot to participant classifications. Above the upper horizontal red line shows Cognivue® scores of < 50 and to the left of the left vertical blue line shows St. Louis University Mental Status (SLUMS) scores < 21 denoting impaired. Below the bottom horizontal red line shows Cognivue® scores of > 75 and to the right of the right vertical blue line shows SLUMS > 27 denoting unimpaired. Results of the analyses of classification are included in the table enclosed in the scatter plot. ACC: Accuracy; FN: False negative; FP: False positive; N%A: Negative percent agreement; P%A: Positive percent agreement; SLUMS: St. Louis University Mental Status; TN: True negative; TP: True positive.
Figure 3
Figure 3
Scatterplot showing first test scores (abscissa) and second test scores (ordinate) co-plotted with a Deming regression line (dashed) and 45° line (solid) for Cognivue® and the St Louis University Mental Status. A: Cognivue®; B: St. Louis University Mental Status. Cognivue®: Intercept of line: 95% confidence interval (CI): 4.27-13.84 (SE = 2.433; P = 0.0002); slope of line: 95%CI: 0.880-0.993 (SE = 0.0285; P = 0.0264); regression fit: R2 = 0.81 (r = 0.90). St. Louis University Mental Status: Intercept of line: 95%CI: 2.24-6.06 (SE = 0.970; P < 0.0001); slope of line: 95%CI: 0.82-0.97 (SE = 0.039; P = 0.039); regression fit: R2 = 0.67 (r = 0.82). SLUMS: St. Louis University Mental Status.
Figure 4
Figure 4
Scatterplot showing regression standardized predicted values including all five factors and Cognivue® scores or the St Louis University Mental Status scores co-plotted with linear regression lines. A: Cognivue®; B: St. Louis University Mental Status. Avg: Average; R2: Coefficient of determination; SLUMS: St. Louis University Mental Status.

References

    1. Zadikoff C, Fox SH, Tang-Wai DF, Thomsen T, de Bie RM, Wadia P, Miyasaki J, Duff-Canning S, Lang AE, Marras C. A comparison of the mini mental state exam to the Montreal cognitive assessment in identifying cognitive deficits in Parkinson's disease. Mov Disord. 2008;23:297–299.
    1. Athilingam P, Visovsky C, Elliott AF, Rogal PJ. Cognitive screening in persons with chronic diseases in primary care: challenges and recommendations for practice. Am J Alzheimers Dis Other Demen. 2015;30:547–558.
    1. Connor DJ, Jenkins CW, Carpenter D, Crean R, Perera P. Detection of Rater Errors on Cognitive Instruments in a Clinical Trial Setting. J Prev Alzheimers Dis. 2018;5:188–196.
    1. Segal-Gidan F. Cognitive screening tools. Clin Rev. 2013;23:12–18.
    1. Ranson JM, Kuźma E, Hamilton W, Muniz-Terrera G, Langa KM, Llewellyn DJ. Predictors of dementia misclassification when using brief cognitive assessments. Neurol Clin Pract. 2019;9:109–117.
    1. Cordell CB, Borson S, Boustani M, Chodosh J, Reuben D, Verghese J, Thies W, Fried LB Medicare Detection of Cognitive Impairment Workgroup. Alzheimer's Association recommendations for operationalizing the detection of cognitive impairment during the Medicare Annual Wellness Visit in a primary care setting. Alzheimers Dement. 2013;9:141–150.
    1. Collie A, Darby D, Maruff P. Computerised cognitive assessment of athletes with sports related head injury. Br J Sports Med. 2001;35:297–302.
    1. Nieuwenhuis-Mark RE. The death knoll for the MMSE: has it outlived its purpose? J Geriatr Psychiatry Neurol. 2010;23:151–157.
    1. Sheehan B. Assessment scales in dementia. Ther Adv Neurol Disord. 2012;5:349–358.
    1. Bradford A, Kunik ME, Schulz P, Williams SP, Singh H. Missed and delayed diagnosis of dementia in primary care: prevalence and contributing factors. Alzheimer Dis Assoc Disord. 2009;23:306–314.
    1. Fernandez R, Duffy CJ. Early Alzheimer's disease blocks responses to accelerating self-movement. Neurobiol Aging. 2012;33:2551–2560.
    1. Velarde C, Perelstein E, Ressmann W, Duffy CJ. Independent deficits of visual word and motion processing in aging and early Alzheimer's disease. J Alzheimers Dis. 2012;31:613–621.
    1. Kavcic V, Vaughn W, Duffy CJ. Distinct visual motion processing impairments in aging and Alzheimer's disease. Vision Res. 2011;51:386–395.
    1. Mapstone M, Dickerson K, Duffy CJ. Distinct mechanisms of impairment in cognitive ageing and Alzheimer's disease. Brain. 2008;131:1618–1629.
    1. Tariq SH, Tumosa N, Chibnall JT, Perry MH, 3rd, Morley JE. Comparison of the Saint Louis University mental status examination and the mini-mental state examination for detecting dementia and mild neurocognitive disorder--a pilot study. Am J Geriatr Psychiatry. 2006;14:900–910.
    1. Alzheimer’s Association. 2018 Alzheimer’s disease facts and figures. Alzheimers Dement. 2018;14:367–429.
    1. Katz MJ, Lipton RB, Hall CB, Zimmerman ME, Sanders AE, Verghese J, Dickson DW, Derby CA. Age-specific and sex-specific prevalence and incidence of mild cognitive impairment, dementia, and Alzheimer dementia in blacks and whites: a report from the Einstein Aging Study. Alzheimer Dis Assoc Disord. 2012;26:335–343.
    1. Petersen RC, Roberts RO, Knopman DS, Geda YE, Cha RH, Pankratz VS, Boeve BF, Tangalos EG, Ivnik RJ, Rocca WA. Prevalence of mild cognitive impairment is higher in men. The Mayo Clinic Study of Aging. Neurology. 2010;75:889–897.
    1. Ganguli M, Chang CC, Snitz BE, Saxton JA, Vanderbilt J, Lee CW. Prevalence of mild cognitive impairment by multiple classifications: The Monongahela-Youghiogheny Healthy Aging Team (MYHAT) project. Am J Geriatr Psychiatry. 2010;18:674–683.
    1. Wild K, Howieson D, Webbe F, Seelye A, Kaye J. Status of computerized cognitive testing in aging: a systematic review. Alzheimers Dement. 2008;4:428–437.
    1. Boise L, Camicioli R, Morgan DL, Rose JH, Congleton L. Diagnosing dementia: perspectives of primary care physicians. Gerontologist. 1999;39:457–464.
    1. Canadian Task Force on Preventive Health Care. Pottie K, Rahal R, Jaramillo A, Birtwhistle R, Thombs BD, Singh H, Gorber SC, Dunfield L, Shane A, Bacchus M, Bell N, Tonelli M. Recommendations on screening for cognitive impairment in older adults. CMAJ. 2016;188:37–46.
    1. Lin JS, O'Connor E, Rossom RC, Perdue LA, Burda BU, Thompson M, Eckstrom E. Screening for cognitive impairment in older adults: An evidence update for the U.S. Preventive Services Task Force. Rockville (MD): Agency for Healthcare Research and Quality; 2013.
    1. World Health Organization. Geneva: World Health Organization; 2019. Risk reduction of cognitive decline and dementia: WHO guidelines. Available from: .
    1. Borson S, Scanlan JM, Watanabe J, Tu SP, Lessig M. Improving identification of cognitive impairment in primary care. Int J Geriatr Psychiatry. 2006;21:349–355.
    1. Perry W, Lacritz L, Roebuck-Spencer T, Silver C, Denney RL, Meyers J, McConnel CE, Pliskin N, Adler D, Alban C, Bondi M, Braun M, Cagigas X, Daven M, Drozdick L, Foster NL, Hwang U, Ivey L, Iverson G, Kramer J, Lantz M, Latts L, Ling SM, Maria Lopez A, Malone M, Martin-Plank L, Maslow K, Melady D, Messer M, Most R, Norris MP, Shafer D, Silverberg N, Thomas CM, Thornhill L, Tsai J, Vakharia N, Waters M, Golden T. Population Health Solutions for Assessing Cognitive Impairment in Geriatric Patients. Innov Aging. 2018;2:igy025.
    1. Cao L, Hai S, Lin X, Shu D, Wang S, Yue J, Liu G, Dong B. Comparison of the Saint Louis University Mental Status Examination, the Mini-Mental State Examination, and the Montreal Cognitive Assessment in detection of cognitive impairment in Chinese elderly from the geriatric department. J Am Med Dir Assoc. 2012;13:626–629.
    1. Buckingham D, Mackor K, Miller R, Pullam N, Molloy K. Comparing the cognitive screening tools: MMSE and SLUMS. PURE Insights. 2013;2:3.
    1. Feliciano L, Horning SM, Klebe KJ, Anderson SL, Cornwell RE, Davis HP. Utility of the SLUMS as a cognitive screening tool among a nonveteran sample of older adults. Am J Geriatr Psychiatry. 2013;21:623–630.
    1. Smith AD, 3rd, Duffy C, Goodman AD. Novel computer-based testing shows multi-domain cognitive dysfunction in patients with multiple sclerosis. Mult Scler J Exp Transl Clin. 2018;4:2055217318767458.
    1. US Food and Drug Administration. De Novo Classification Request For Cognivue. De Novo Summary (DEN130033). [accessed December 22, 2018] Available from: .

Source: PubMed

Sponsorer og samarbeidspartnere

Medisinsk tilstand

Legemiddelintervensjoner

3
Abonnere