Validation of an ecological momentary assessment to measure processing speed and executive function in schizophrenia

Abstract

Cognitive impairments are a core feature of schizophrenia that have negative impacts on functional outcomes. However, it remains challenging to assess these impairments in clinical settings. Smartphone apps provide the opportunity to measure cognitive impairments in an accessible way; however, more research is needed to validate these cognitive assessments in schizophrenia. We assessed the initial accessibility, validity, and reliability of a smartphone-based cognitive test to measure cognition in schizophrenia. A total of 29 individuals with schizophrenia and 34 controls were included in the analyses. Participants completed the standard pen-and-paper Trail Making Tests (TMT) A and B, and smartphone-based versions, Jewels Trail Tests (JTT) A and B, at the single in-lab visit. Participants were asked to complete the JTT remotely once per week for three months. We also investigated how subjective sleep quality and mood may affect cognitive performance longitudinally. In-lab and remote JTT scores moderately and positively correlated with in-lab TMT scores. Moderate test-retest reliability was observed across the in-lab, first remote, and last remote completion times of the JTT. Additionally, individuals with schizophrenia had significantly lower performance compared to controls on both the in-lab JTT and TMT. Self-reported mood had a significant effect on JTT A performance over time but no other significant relationships were found remotely. Our results support the initial accessibility, validity and reliability of using the JTT to measure cognition in schizophrenia. Future research to develop additional smartphone-based cognitive tests as well as with larger samples and in other psychiatric populations are warranted.

Conflict of interest statement

The authors declare no competing interests.

© 2021. The Author(s).

Figures

Fig. 1. JTT smartphone-based cognitive assessments on the mindLAMP app.

a On the JTT A, participants tap on the numbers in chronological order, b on the JTT B; participants alternate between numbers and letters in chronological order.

Fig. 2. Relationship between performance on smartphone app and pen-and-paper in-lab cognitive assessments at baseline.

a JTT A is positively correlated with TMT A; b JTT B is positively correlated with TMT B. HC Healthy controls, SZ schizophrenia, JTT Jewels Trail Tests, TMT Trail Making Tests. Cognitive performance was measured in seconds where a greater completion time represents a worse score.

Fig. 3. Differences in cognitive performance between individuals with schizophrenia and healthy controls on in-lab cognitive assessments.

JTT Jewels Trail Tests, TMT Trail Making Tests, HC healthy controls, SZ individuals with schizophrenia/schizoaffective disorder. All comparisons survive Bonferroni corrections p < 0.05. Error bars represent 95% confidence intervals. Cognitive performance was measured in seconds where a greater completion time represents a poorer score.

Fig. 4. Receiver operating curves results comparing the sensitivity and specificity for both the smartphone app JTT and pen-and-paper TMT.

a JTT A and TMT A show similar sensitivity and specificity; b JTT B and TMT B show similar sensitivity and specificity. ROC receiver operating characteristic curves, AUC area under the curve, TMT Trail Making Test, JTT Jewels Trail Tests.

Fig. 5. Timeline of the study procedures.

M.I.N.I MINI International Neuropsychiatric Interview, PANSS Positive and Negative Syndrome Scale, TMT Trail Making Tests, JTT Jewels Trail Tests. The time of the last remote assessment differs per participant based on the number of times they completed the JTT A and B over the 12-week study period.