Optimization of cognitive assessment in Parkinsonisms by applying artificial intelligence to a comprehensive screening test

Paola Ortelli, Davide Ferrazzoli, Viviana Versace, Veronica Cian, Marianna Zarucchi, Anna Gusmeroli, Margherita Canesi, Giuseppe Frazzitta, Daniele Volpe, Lucia Ricciardi, Raffaele Nardone, Ingrid Ruffini, Leopold Saltuari, Luca Sebastianelli, Daniele Baranzini, Roberto Maestri, Paola Ortelli, Davide Ferrazzoli, Viviana Versace, Veronica Cian, Marianna Zarucchi, Anna Gusmeroli, Margherita Canesi, Giuseppe Frazzitta, Daniele Volpe, Lucia Ricciardi, Raffaele Nardone, Ingrid Ruffini, Leopold Saltuari, Luca Sebastianelli, Daniele Baranzini, Roberto Maestri

Abstract

The assessment of cognitive deficits is pivotal for diagnosis and management in patients with parkinsonisms. Low levels of correspondence are observed between evaluations assessed with screening cognitive tests in comparison with those assessed with in-depth neuropsychological batteries. A new tool, we named CoMDA (Cognition in Movement Disorders Assessment), was composed by merging Mini-Mental State Examination (MMSE), Montreal Cognitive Assessment (MoCA), and Frontal Assessment Battery (FAB). In total, 500 patients (400 with Parkinson's disease, 41 with vascular parkinsonism, 31 with progressive supranuclear palsy, and 28 with multiple system atrophy) underwent CoMDA (level 1-L1) and in-depth neuropsychological battery (level 2-L2). Machine learning was developed to classify the CoMDA score and obtain an accurate prediction of the cognitive profile along three different classes: normal cognition (NC), mild cognitive impairment (MCI), and impaired cognition (IC). The classification accuracy of CoMDA, assessed by ROC analysis, was compared with MMSE, MoCA, and FAB. The area under the curve (AUC) of CoMDA was significantly higher than that of MMSE, MoCA and FAB (p < 0.0001, p = 0.028 and p = 0.0007, respectively). Among 15 different algorithmic methods, the Quadratic Discriminant Analysis algorithm (CoMDA-ML) showed higher overall-metrics performance levels in predictive performance. Considering L2 as a 3-level continuous feature, CoMDA-ML produces accurate and generalizable classifications: micro-average ROC curve, AUC = 0.81; and AUC = 0.85 for NC, 0.67 for MCI, and 0.83 for IC. CoMDA and COMDA-ML are reliable and time-sparing tools, accurate in classifying cognitive profile in parkinsonisms.This study has been registered on ClinicalTrials.gov (NCT04858893).

Conflict of interest statement

The suthors declare no competing interests.

© 2022. The Author(s).

Figures

Fig. 1. ROC curves.
Fig. 1. ROC curves.
ROC curves obtained for the scores of the four cognitive-screening tools considered (see the text).
Fig. 2. ROC curves for QDA.
Fig. 2. ROC curves for QDA.
CoMDA-ML: ROC curves of multilevel classification.
Fig. 3. Confusion matrix.
Fig. 3. Confusion matrix.
Accuracy-prediction levels for L2 classes as 0 = IC, 1 = MCI, and 2 = NC. Observed values (True Class) are reported in row-wise, while the predicted values (Predicted Class) are reported in column-wise.

References

    1. Chaudhuri KR, Schapira AH. Non-motor symptoms of Parkinson’s disease: dopaminergic pathophysiology and treatment. Lancet Neurol. 2009;8:464–474. doi: 10.1016/S1474-4422(09)70068-7.
    1. Ferrazzoli D, et al. Basal ganglia and beyond: the interplay between motor and cognitive aspects in Parkinson’s disease rehabilitation. Neurosci. Biobehav. Rev. 2018;90:294–308. doi: 10.1016/j.neubiorev.2018.05.007.
    1. Ferrazzoli D, et al. Efficacy of intensive multidisciplinary rehabilitation in Parkinson’s disease: a randomised controlled study. J. Neurol. Neurosurg. Psychiatry. 2018;89:828–835. doi: 10.1136/jnnp-2017-316437.
    1. Bhatia KP, Stamelou M. Nonmotor features in atypical Parkinsonism. Int. Rev. Neurobiol. 2017;134:1285–1301. doi: 10.1016/bs.irn.2017.06.001.
    1. Martinez-Horta S, Horta-Barba A, Kulisevsky J. Cognitive and behavioral assessment in Parkinson’s disease. Expert Rev. Neurother. 2019;19:613–622. doi: 10.1080/14737175.2019.1629290.
    1. Santangelo G, et al. Comparative cognitive and neuropsychiatric profiles between Parkinson’s disease, multiple system atrophy and progressive supranuclear palsy. J. Neurol. 2018;265:2602–2613. doi: 10.1007/s00415-018-9038-x.
    1. Emre M, et al. Clinical diagnostic criteria for dementia associated with Parkinson’s disease. Mov. Disord. 2007;22:1689–1707. doi: 10.1002/mds.21507.
    1. Litvan I, et al. Diagnostic criteria for mild cognitive impairment in Parkinson’s disease: Movement Disorder Society Task Force guidelines. Mov. Disord. 2012;27:349–356. doi: 10.1002/mds.24893.
    1. Gill DJ, Freshman A, Blender JA, Ravina B. The Montreal cognitive assessment as a screening tool for cognitive impairment in Parkinson’s disease. Mov. Disord. 2008;23:1043–1046. doi: 10.1002/mds.22017.
    1. Hoops S, et al. Validity of the MoCA and MMSE in the detection of MCI and dementia in Parkinson disease. Neurology. 2009;73:1738–1745. doi: 10.1212/WNL.0b013e3181c34b47.
    1. Nasreddine ZS, et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J. Am. Geriatr. Soc. 2005;53:695–699. doi: 10.1111/j.1532-5415.2005.53221.x.
    1. Zadikoff C, et al. A comparison of the mini mental state exam to the Montreal cognitive assessment in identifying cognitive deficits in Parkinson’s disease. Mov. Disord. 2008;23:297–299. doi: 10.1002/mds.21837.
    1. Fiorenzato E, et al. Montreal Cognitive Assessment (MoCA) and mini-mental state examination (MMSE) performance in progressive supranuclear palsy and multiple system atrophy. J. Neural Transm. 2016;123:1435–1442. doi: 10.1007/s00702-016-1589-3.
    1. Eschlböck S, et al. Cognition in multiple system atrophy: a single-center cohort study. Ann. Clin. Transl. Neurol. 2020;7:219–228. doi: 10.1002/acn3.50987.
    1. Goldman JG, et al. Diagnosing PD-MCI by MDS Task Force criteria: how many and which neuropsychological tests? Mov. Disord. 2015;30:402–406. doi: 10.1002/mds.26084.
    1. Rosenblum S, et al. The Montreal Cognitive Assessment: is it suitable for identifying mild cognitive impairment in Parkinson’s disease? Mov. Disord. Clin. Pract. 2020;7:648–655. doi: 10.1002/mdc3.12969.
    1. Marras C, et al. Measuring mild cognitive impairment in patients with Parkinson’s disease. Mov. Disord. 2013;28:626–633. doi: 10.1002/mds.25426.
    1. Bezdicek O, et al. Frontal assessment battery in Parkinson’s disease: validity and morphological correlates. J. Int. Neuropsychol. Soc. 2017;23:675–684. doi: 10.1017/S1355617717000522.
    1. Brunetti A, Carnimeo L, Trotta GF, Bevilacqua V. Computer-assisted frameworks for classification of liver, breast and blood neoplasias via neural networks: a survey based on medical images. Neurocomputing. 2019;335:274–298. doi: 10.1016/j.neucom.2018.06.080.
    1. Li X, et al. Using machine learning to predict stroke-associated pneumonia in Chinese acute ischaemic stroke patients. Eur. J. Neurol. 2020;27:1656–1663. doi: 10.1111/ene.14295.
    1. Jayatilake S, Ganegoda GU. Involvement of machine learning tools in healthcare decision making. J. Health. Eng. 2021;2021:6679512. doi: 10.1155/2021/6679512.
    1. Sidey-Gibbons JAM, Sidey-Gibbons CJ. Machine learning in medicine: a practical introduction. BMC Med. Res. Methodol. 2019;19:64. doi: 10.1186/s12874-019-0681-4.
    1. McKinney SM, et al. International evaluation of an AI system for breast cancer screening. Nature. 2020;577:89–94. doi: 10.1038/s41586-019-1799-6.
    1. Pedregosa F, et al. Scikit-learn: machine learning in python. J. Mach. Learn. Res. 2011;12:2825–2830.
    1. de Lau LM, et al. Incidence of parkinsonism and Parkinson disease in a general population: the rotterdam study. Neurology. 2004;63:1240–1244. doi: 10.1212/.
    1. Goldman JG, et al. Cognitive impairment in Parkinson’s disease: a report from a multidisciplinary symposium on unmet needs and future directions to maintain cognitive health. NPJ Parkinsons Dis. 2018;4:19. doi: 10.1038/s41531-018-0055-3.
    1. Bak TH, et al. Cognitive bedside assessment in atypical parkinsonian syndromes. J. Neurol. Neurosurg. Psychiatry. 2005;76:420–422. doi: 10.1136/jnnp.2003.029595.
    1. Boxer AL, et al. Advances in progressive supranuclear palsy: new diagnostic criteria, biomarkers, and therapeutic approaches. Lancet Neurol. 2017;16:552–563. doi: 10.1016/S1474-4422(17)30157-6.
    1. Gerstenecker A. The neuropsychology (broadly conceived) of multiple system atrophy, progressive supranuclear palsy, and corticobasal degeneration. Arch. Clin. Neuropsychol. 2017;32:861–875. doi: 10.1093/arclin/acx093.
    1. Ortelli P, et al. How Cognition and motivation “Freeze” the motor behavior in Parkinson’s disease. Front. Neurosci. 2019;13:1302. doi: 10.3389/fnins.2019.01302.
    1. Lange F, Brückner C, Knebel A, Seer C, Kopp B. Executive dysfunction in Parkinson’s disease: a meta-analysis on the Wisconsin Card Sorting Test literature. Neurosci. Biobehav Rev. 2018;93:38–56. doi: 10.1016/j.neubiorev.2018.06.014.
    1. Foltynie T, Brayne CE, Robbins TW, Barker RA. The cognitive ability of an incident cohort of Parkinson’s patients in the UK. The CamPaIGN study. Brain. 2004;127:550–560. doi: 10.1093/brain/awh067.
    1. Marinus J, et al. Assessment of cognition in Parkinson’s disease. Neurology. 2003;61:1222–1228. doi: 10.1212/01.WNL.0000091864.39702.1C.
    1. Pagonabarraga J, et al. Parkinson’s disease-cognitive rating scale: a new cognitive scale specific for Parkinson’s disease. Mov. Disord. 2008;23:998–1005. doi: 10.1002/mds.22007.
    1. Bosco A, et al. Italians do it worse. Montreal Cognitive Assessment (MoCA) optimal cut-off scores for people with probable Alzheimer’s disease and with probable cognitive impairment. Aging Clin. Exp. Res. 2017;29:1113–1120. doi: 10.1007/s40520-017-0727-6.
    1. Cools R, Altamirano L, D’Esposito M. Reversal learning in Parkinson’s disease depends on medication status and outcome valence. Neuropsychologia. 2006;44:1663–1673. doi: 10.1016/j.neuropsychologia.2006.03.030.
    1. van Balkom TD, et al. COGTIPS: a double-blind randomized active controlled trial protocol to study the effect of home-based, online cognitive training on cognition and brain networks in Parkinson’s disease. BMC Neurol. 2019;19:179. doi: 10.1186/s12883-019-1403-6.
    1. Postuma RB, et al. MDS clinical diagnostic criteria for Parkinson’s disease. Mov. Disord. 2015;30:1591–1601. doi: 10.1002/mds.26424.
    1. Höglinger GU, et al. Clinical diagnosis of progressive supranuclear palsy: the movement disorder society criteria. Mov. Disord. 2017;32:853–864. doi: 10.1002/mds.26987.
    1. Gilman S, et al. Second consensus statement on the diagnosis of multiple system atrophy. Neurology. 2008;71:670–676. doi: 10.1212/01.wnl.0000324625.00404.15.
    1. Zijlmans JC, Daniel SE, Hughes AJ, Révész T, Lees AJ. Clinicopathological investigation of vascular parkinsonism, including clinical criteria for diagnosis. Mov. Disord. 2004;19:630–640. doi: 10.1002/mds.20083.
    1. Measso G, et al. The mini‐mental state examination: normative study of an Italian random sample. Dev. Neuropsychol. 1993;9:77–85. doi: 10.1080/87565649109540545.
    1. Santangelo G, et al. Normative data for the Montreal Cognitive Assessment in an Italian population sample. Neurol. Sci. 2015;36:585–591. doi: 10.1007/s10072-014-1995-y.
    1. Appollonio I, et al. The frontal assessment battery (FAB): normative values in an Italian population sample. Neurol. Sci. 2005;26:108–116. doi: 10.1007/s10072-005-0443-4.
    1. Novelli, G., Papagno, C., Capitani, E. & Laiacona, M. Tre test clinici di memoria verbale a lungo termine: Taratura su soggetti normali. / Three clinical tests for the assessment of verbal long-term memory function: Norms from 320 normal subjects. Archivio di Psicologia, Neurologia e Psichiatria, 278–296 (1970).
    1. Giovagnoli AR, et al. Trail making test: normative values from 287 normal adult controls. Ital. J. Neurol. Sci. 1996;17:305–309. doi: 10.1007/BF01997792.
    1. Caffarra P, Vezzadini G, Dieci F, Zonato F, Venneri A. Rey-Osterrieth complex figure: normative values in an Italian population sample. Neurol. Sci. 2002;22:443–447. doi: 10.1007/s100720200003.
    1. Caffarra P, Vezzadini G, Francesca D, Zonato F, Venneri A. A short version of the Stroop test: Normative data in an Italian population sample. Nuova Riv. di Neurologia. 2002;12:111–115.
    1. Laiacona M, Inzaghi MG, De Tanti A, Capitani E. Wisconsin card sorting test: a new global score, with Italian norms, and its relationship with the Weigl sorting test. Neurol. Sci. 2000;21:279–291. doi: 10.1007/s100720070065.
    1. Spinnler, H. T. G. Standardizzazione e taratura italiana di test neuropsicologici: gruppo italiano per lo studio neuropsicologico dell’invecchiamento. (Masson Italia Periodici, 1987).
    1. Battiti R. Using mutual information for selecting features in supervised neural net learning. IEEE Trans. Neural Netw. 1994;5:537–550. doi: 10.1109/72.298224.
    1. Brown G, Pocock A, Zhao M-J, Luján M. Conditional likelihood maximisation: a unifying framework for information theoretic feature selection. J. Mach. Learn. Res. 2012;13:27–66.
    1. Akiba, T., Sano, S., Yanase, T., Ohta, T. & Koyama, M. in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining 2623–2631 (Association for Computing Machinery, 2019).

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe