Comparison Between Analog Neurocognitive Tests Conducted by a Trained Test Leader Versus Self-administered Digital Tests (PICASU-Eq)

Comparison Between Analog Neurocognitive Tests Conducted by a Trained Test Leader Versus Self-administered Digital Tests: a Protocol for an Equivalence Study

The study has a randomized crossover design including qualitative interviews of the participant's test experiences. Healthy participants, ≥60 years are eligible to participate in the study. Cognitive function will be measured by using the International Study Group of Postoperative Cognitive Dysfunction (ISPOCD) test battery and the Mindmore digital test battery. The participants will self-report their likelihood of depression with the geriatric depression scale-15, their user experience of the digital test by a modified version of System Usability scale, and answer questionnaires targeting their experiences of the test sessions. Furthermore, concentration difficulties, according to the Swedish Quality of Recovery-scale will also be measured.

Study Overview

• Status: Completed
• Conditions: Feasibility; Costs; Neurocognitive Decline; Interviews
• Intervention / Treatment: Diagnostic test: Analog testing conducted by a trained test leader using, ISPOCD test battery; Diagnostic test: self-administered Mindmore digital cognitive screening battery

Detailed Description

Study design: Randomized crossover design including qualitative interviews of the test procedures.

Participants:The participants are planned to be healthy controls (n=50) comprising men and women of >60 years of age. The number of participants is guided by an earlier cross over studies with older adults assessing cognition digitally and analogy The participants will self-report if they meet any of the exclusion criteria: inability to read and speak Swedish, suffering from a nervous system disease, severe psychiatric disorder, alcoholism, drug dependance and severe visual or auditory disorder.

Cognitive tests Verbal episodic memory ISPOCD: The Visual Verbal Learning Test with 15 words that are individually presented over a series of 3 consecutive presentations. Each word is presented visually for 2 seconds. The participant is asked to recall as many words as possible after each trial as well as after 20 min without further presentation of word, i.e. delayed recall.

Mindmore: CERAD is a 10-word verbal learning test with 3 learning trials, a recall trial after ~7 min and a recognition trial. Compared to RAVLT this test is shorter, but also a little less sensitive. Words are read and visually presented in writing to the patient and the patient replies in speech.

Executive, visuospatial (assess processing speed) ISPOCD: The Concept Shifting Test is a trail-making test where the participant is required to alternate between letters and digits. Each part consists of 16 circles on a paper. In part A, the circles are numbered from 1 to 16. The test person is instructed to draw a short over each number in ascending order from 1 to 16, as rapidly as possible. Part B consists of letters from A-P. Part C consists of 8 digits and 8 letters to be paired and the test person is instructed to draw a short over each and letter order (1-A, 2-B, 3-C and so on).

Mindmore: Trail Making Test A & B Each part consists of 25 circles on a paper. In part A, the circles are numbered from 1 to 25. The test person is instructed to draw a line between them in ascending order of numbers, from 1 to 25, as rapidly as possible. Part B consists of 13 digits and 12 letters to be paired and the test person is instructed to draw a dash between them in ascending number and letter order (1-A, 2-B, 3-C and so on). The digitized versions of the TMT have shown strong correlation with the results on the traditional test vs. the digital version in healthy individuals

Executive, selective attention (assess the ability to concentrate and ignore distracting stimuli) ISPOCD Stroop color-word test 40 words spelling out a color are printed in contrasting ink colors (e.g., green printed with red ink) and the participant is asked to tell the printed color of the word rather than the actual meaning of the word. The Stroop test is repeated three times with different words.

Mindmore: Stroop color-word test 24 words spelling out a color are printed in contrasting ink colors (e.g., green printed with red ink) and the participant is asked to tell the printed color of the word rather than the actual meaning of the word The digitized versions of the Stroop Word and Color Test have shown moderate to strong correlation with the results on the traditional test vs. the digital version in healthy individuals

Executive, visuospatial (assess mental processing speed and concentration) ISPOCD: In the Letter Digit Coding Test, the participant is asked to match letters with a specific digit for 60 seconds.

Mindmore: Symbol Digits Processing Test (SDPT) is based on the original SDMT. The symbols in the Mindmore SDPT test are randomised to minimize practice effects. Presented are an array of nine symbol-digit pairings at the top of the screen, one symbol in the middle of the screen, and a 3 by 3 number grid at the bottom of the screen. The participant is required to associate the symbol in the middle to one of the digits on the grid below. As soon as a choice is made a new symbol appears. The final score is the number of correct matches in 90 seconds. The digitalized SDMT shows comparable sensitivity and specificity to the traditional paper-SDMT in detecting overall cognitive impairment on a neuropsychological test battery in adult MS patients

Outcome measures Acceptability, Feasibility and Usability The time taken by participants to complete the tasks will be recorded. After each test, i.e ISPOCD or Mindmore, the participants will be asked to rate on a 5-point scale, from not at all to extremely, how hard, stressful, and acceptable they considered the cognitive assessments were. The participants will answer by a yes or no question if they considered the tests difficult, and if so, describe which parts of the tests. After both tests are conducted, the participants will be asked which tests they thought were preferable and why. Furthermore, they will be asked if they were undergoing surgery if it would be possible to do the digital cognitive to detect a postoperative cognitive decline, and if yes, how often?

The usability of the digital version will be assessed with a questionnaire consisting of six items from the System Usability Scale (SUS)31 to be answered on a 5-point scale from "strongly agree" to "strongly disagree".

1. I think it is possible to use the system for repeated measurements.
2. I think the system was easy to use.
3. I think I would need the support of a technical competent person to be able to use this system.
4. I think that most people would learn to use this system quickly.
5. I think the system is cumbersome to use.
6. I felt confident using this system.

The original SUS questionnaire consists of ten items31 but only those items of the SUS will be used whose answers were considered in terms of usability after this short test period.

To further explore the feasibility and acceptability for future implementation of neurocognitive assessment, the participants' experience of being assessed for neurocognitive performance will be studied and if they would prefer the test sessions to be different. Semi-structured one-on-one interviews with 20 of the participants will be conducted within 1 month after undergoing the two test procedures. The interviews will take place online via Zoom, Teams or via a phone call. The person who will conduct the interviews will not be involved in the cognitive test procedures.

Concentration difficulties The symptoms concentration difficulties the past 24 hours, (one of the symptoms included in the Swedish web version of Quality of Recover) rated on 11-point numerical scales from 0, "none of the time", to 10, "all of the time", will also be measured.

Depression screening The geriatric depression scale (GDS) is a widely used instrument to screen depression in the older adult population. The original scale is based on 30 items, a shorter 15-item scale was developed to be used in settings lacking time and where patients experience fatigue. The 15-item geriatric depression scale is proven to have sufficient reliability, validity, and efficacy. Evidence has proven that the Geriatric depression scale in the 15-item version has high diagnostic performance, pool sensitivity and pool specificity.The cutoff score is standardized with a score of 5. The scale is composed of items with questions such as "do you feel helpless?" or "are you in good spirits most of the time?. Every participant will answer the Swedish version of the GDS-15 after each test session.

Demographic data such as age, gender and education level will also be collected.

Procedure Participants will be recruited from Karolinska Institutet's web page Research subjects wanted ( https://ki.se/en/research/research-subjects-wanted), non-governmental organisations serving older adults, and activity centres for seniors The participants will be randomized to one of two conditions: 1) ISPOCD test battery followed by Mindmore digitalized cognitive test battery, or 2) Mindmore digitalized cognitive test battery followed by ISPOCD test battery, with 2 weeks passing between the cognitive testing for both groups. The chosen interval are guided by earlier studies comparing digitalized vs analog tests of cognitive function in healthy participants people with a wash out period ranging from 30 minutes - 4 weeks between the studies .

The ISPOCD testing will be conducted by a test leader for this purpose. The digitized version will be self-administered in a touch screen tablet (10.1" Windows). Both tests will be carried out in the same location, with a duration of approximately 20-30 min each in a quiet environment adapted for this purpose. The duration of time the test leader spend on instructions will be counted as well as the time for conducting the ISPOCD test.

Analysis Descriptive statistics of demographic data will be presented with number, percentage, and mean (SD) or min-max, as appropriate. Non-parametric tests will be used to analyse differences in feasibility and usability between the analog vs the digital test. If the data is normal distributed Pearson correlation coefficient will be used to measure the association between ISPOCD and Mindmore four cognitive domains. If the data is non normally distributed Spearman rank correlation will be used. The values for the correlation coefficient irrespectively if using Person or Spearman range between -1.0 and 1.0. A correlation of -1.0 shows a perfect negative correlation, while a correlation of 1.0 shows a perfect positive correlation. However, a correlation equal to 0 does not imply the absence of association38. Correlation analysis will also be conducted between the symptom Concentration difficulties and the four different domains in ISPOCD and Mindmore.

Analysis of variance (ANOVA) will be used to determine the GDS scores with differences in the cognitive tests. Ordinal logistic regression to study associations between depression scores, non-depression scores and cognitive performance.

Exploratory multiple linear regression models will be used to analyse cognitive performance for each cognitive domain of each test (ISPOCD and Mindmore) as dependent variables and age and level of education as independent variables, due to their potential impact as risk factors for postoperative cognitive decline.

The analysis considered costs for the digital screening tests includes the application software, licence, web administrator interface, data storage, analysis, security, and IT support (obtained from Mindmore) and time the test leader instructing the participant. The cost fort the analog test will include time the test leader instructing the participant, conducting the tests, documentation and analysis. All cost estimates included social fees and overheads and will be converted from Swedish krona (SEK) to Euro using an approximate exchange rate.

All data will be analysed using IBM SPSS Statistics version 27 (IBM Corp), and a two-tailed p-value < 0.05 was considered significant.

• Study Type: Interventional
• Enrollment (Actual): 50
• Phase: Not Applicable

Contacts and Locations

Study Locations

• Sweden
  • Stockholm, Sweden, 141 83
    • Ulrica Nilsson

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 60 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

• > 59 years of age
• ability to read and speak Swedish,

Exclusion Criteria:

• suffering from a nervous system disease,
• suffering from a severe psychiatric disorder
• alcoholism
• drug dependence
• severe visual or auditory disorder.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Screening
• Allocation: Randomized
• Interventional Model: Crossover Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Other: ISPOCD test battery; The participants will be randomized to start with ISPOCD test battery followed by Mindmore digitalized cognitive test battery with 2 weeks passing between the cognitive testing for both groups	Diagnostic test: Analog testing conducted by a trained test leader using, ISPOCD test battery; Cognitive function will be measured by using the International Study Group of Postoperative Cognitive Dysfunction (ISPOCD) analog test battery v; Diagnostic test: self-administered Mindmore digital cognitive screening battery; Cognitive function will be measured by using the Mindmores digital test battery
Other: Mindmore digitalized cognitive test battery; Strating with Mindmore digitalized cognitive test battery followed by ISPOCD test battery, with 2 weeks passing between the cognitive testing for both groups	Diagnostic test: Analog testing conducted by a trained test leader using, ISPOCD test battery; Cognitive function will be measured by using the International Study Group of Postoperative Cognitive Dysfunction (ISPOCD) analog test battery v; Diagnostic test: self-administered Mindmore digital cognitive screening battery; Cognitive function will be measured by using the Mindmores digital test battery

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
equivalence between the two tests	If the data is normal distributed Pearson correlation coefficient will be used to measure the association between ISPOCD and Mindmore four cognitive domains. If the data is non normally distributed Spearman rank correlation will be used. The values for the correlation coefficient irrespectively if using Person or Spearman range between -1.0 and 1.0. A correlation of -1.0 shows a perfect negative correlation, while a correlation of 1.0 shows a perfect positive correlation. However, a correlation equal to 0 does not imply the absence of association	through study completion, an average of 6 months

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Usability of the digital test version	Assessed by the System Usability Scale (SUS)	through study completion, an average of 6 months
Acceptability and Feasibility (I)	After each test, i.e ISPOCD or Mindmore, the participants will be asked to rate on a 5-point scale, from not at all to extremely, how hard, stressful, and acceptable they considered the cognitive assessments were. The participants will answer by a yes or no question if they considered the tests difficult, and if so, describe which parts of the tests. After both tests are conducted, the participants will be asked which tests they thought were preferable and why. Furthermore, they will be asked if they were undergoing surgery if it would be possible to do the digital cognitive to detect a postoperative cognitive decline, and if yes, how often?	through study completion, an average of 6 months
Acceptability and Feasibility (II)	To further explore the feasibility and acceptability for future implementation of neurocognitive assessment, the participants' experience of being assessed for neurocognitive performance will be studied and if they would prefer the test sessions to be different. Semi-structured one-on-one interviews with 20-25 of the participants will be conducted within 1 month after undergoing the two test procedures. The interviews will take place online via Zoom, Teams or via a phone call. The person who will conduct the interviews will not be involved in the cognitive test procedures.	through study completion, an average of 6 months
Concentration difficulties the past 24 hours	rated on 11-point numerical scales from 0, "none of the time", to 10, "all of the time", .	through study completion, an average of 6 months
Depression screening	The 15-item geriatric depression scale	through study completion, an average of 6 months
Costs for the different screening tests	The analysis considers the costs for the digital screening tests, including the application software, licence, web administrator interface, data storage, analysis, security, and IT support (obtained from Mindmore) as well as time the test leader spends instructing the participant. The cost for the analog test will include time the test leader spends instructing the participant and the time for conducting the tests, documentation, and analysis. All cost estimates related to the tests leader will be mean salary/min including social fees and overheads and will be converted from Swedish krona (SEK) to euro using an approximate exchange rate.	through study completion, an average of 6 months

Collaborators and Investigators

• Sponsor: Karolinska Institutet
• Investigators: Principal Investigator: Ulrica Nilsson, Professor, Karolinska Institutet

Publications and helpful links

General Publications

• Amirpour A, Eckerblad J, Bergman L, Nilsson U. Comparing analog and digital neurocognitive tests with older adults: a study of the ISPOCD battery vs. a digital test battery from Mindmore. BMC Geriatr. 2024 Jan 8;24(1):34. doi: 10.1186/s12877-023-04648-w.

Study record dates

Study Major Dates

• Study Start (Actual): February 17, 2022
• Primary Completion (Actual): April 5, 2022
• Study Completion (Actual): April 5, 2022

Study Registration Dates

• First Submitted: December 22, 2021
• First Submitted That Met QC Criteria: February 13, 2022
• First Posted (Actual): February 24, 2022

Study Record Updates

• Last Update Posted (Actual): May 1, 2024
• Last Update Submitted That Met QC Criteria: April 30, 2024
• Last Verified: April 1, 2024

More Information

Terms related to this study

• Other Study ID Numbers: 2021-01095

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO
• IPD Plan Description: deidentified/anonymous can be shared

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No