Validating Studies to Assess the Diagnostic Accuracy of a Software Application for Detection and Monitoring of Attentional Deficits in Delirium

MICA: Development of a Software Application for Detection and Monitoring of Attentional Deficits in Delirium

The study has two phases: Phase B and Phase C. The purpose of the Phase B study is to determine the performance (sensitivity and specificity) of a smartphone application (DelApp) to identify delirium in the whole inpatient sample. The study also aims to determine the performance (sensitivity and specificity) of the DelApp to discriminate between delirium (with or without dementia) and dementia (without delirium).

The purpose of the Phase C study is to determine the efficacy of the DelApp software application in detecting delirium in unselected patients in an inpatient sample and to determine the performance of the DelApp to discriminate between delirium and dementia. The study also aims to explore the performance of the DelApp in tracking change in cognitive function.

Study Overview

• Status: Completed
• Conditions: Delirium; Dementia
• Intervention / Treatment: Other: No intervention given to participants

Detailed Description

The investigators previously developed a new neuropsychological test for the objective measurement of inattention in delirium, which was implemented on a purpose-built computerised device (Delbox). A prototype software application (DelApp) for smartphones has now been developed based on the same test.

The DelApp has many advantages over existing tests, including objectivity, automated scoring and ease-of-use. Studies in 30 hospital patients showed highly comparable performance between the Delbox and DelApp. Further proof-of-principle single-rater studies using the DelApp in more than 200 hospital patients have provided additional evidence of good performance of the DelApp.

Though there are strong positive findings from investigator pilot studies, larger-scale formal studies with blinded raters and more representative patient samples are now required.

The proposed study is part of a larger programme of work funded by the Medical Research Council (grant value £1.01M) on the optimisation and clinical validation of the DelApp, which consists of three stages: Phase A - optimisation of the DelApp and feasibility testing (REC no 15/SC00/19); Phase B - case-control studies in selected patients; and Phase C - validation studies in unselected patients. Ultimately investigators aim to generate a software product which will be fully licensed and commercially available to healthcare organisations.

The proposed Phase B studies will be carried out in hospitalised patients in general and acute medical wards and ICU. The objectives of these studies are (a) to assess diagnostic performance of the DelApp in discrete patient groups and to inform optimal cut-points (Phase B); and (b) to evaluate the diagnostic performance of the DelApp in unselected patients (Phase C). The studies will also contribute to a better understanding of the nature of neuropsychological deficits underlying delirium. At the end of these studies, the DelApp may be modified depending on the study results. Investigators will select the stimuli and subtests that provide the best balance of sensitivity and specificity.

Two case-control studies will be conducted involving patients recruited from general and acute wards (Study B.1) and ICU (Study B.2). Each study will run in two sites, i.e. the Royal Infirmary of Edinburgh (RIE) and the Glasgow Royal Infirmary (GRI). The performance of the DelApp (index test) against reference standard diagnosis will be independently assessed by two trained raters, with one pair of raters on each site. As in our prior case-control studies, convenience samples will be used, where clinicians are asked to identify patients in the target groups.

Hospitalised patients will be asked to undertake a number of tests at their bedside. These tests will be administered to patients once only. Patients will be visited by two different researchers on the same day, with a gap of approximately 15-60 min between assessment visits assuming that consent is provided in the first visit. The first visit will involve assessment of capacity and (in those thought likely to be capable of giving consent) obtaining informed consent, as well as assessment of delirium (reference tests) and cognition. The second visit will involve the DelApp assessment (index test). Thus, the index test and reference tests are administered by independent raters. Where the patient is unable to provide consent for themselves, proxy consent will be sought, and if granted, the same visits as above will occur.

The Phase C formal validation studies have a similar design to the case-control studies. The main difference is that unselected samples of patients will be recruited from general and acute medical wards (Study C.1) and ICU (Study C.2) in the GRI and RIE.

A subset of patients in studies C.1 and C.2 (N=60 combined) will undergo up to 4 repeat assessments (at least one day apart), along with the reference standard, to explore the performance of the DelApp in tracking change in cognitive function.

In the longitudinal assessments, researchers will ask patients able to provide consent at the time of recruitment if they consent to subsequent assessments even if at the time of these subsequent assessments the patient lacks capacity. However, during the subsequent assessments, it will be carefully checked if the patient assents to continue their participation in the study. In patients who initially lack capacity and have been included through consent from a legal proxy, and who are undergoing serial assessments, researchers will seek informed consent from patients at the beginning of each new testing session. If the patient regains capacity after the final assessment, the investigators will seek retrospective consent from the patient where possible.

• Study Type: Observational
• Enrollment (Actual): 907

Contacts and Locations

Study Locations

• United Kingdom
  • Edinburgh, United Kingdom, EH16 4SA
    • University of Edinburgh
  • Glasgow, United Kingdom, G4 0SF
    • University of Glasgow

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (ADULT, OLDER_ADULT)
• Accepts Healthy Volunteers: Yes
• Genders Eligible for Study: All

• Sampling Method: Probability Sample

Study Population

Royal Infirmary of Edinburgh (RIE) and Glasgow Royal Infirmary (GRI) general wards and intensive care units.

Scottish Dementia Research Network Register and Psychology, Philosophy and Language Science (PPLS) volunteer database.

Description

Inclusion Criteria:

• General Wards

  • Aged 65 or over
  • Capacity to provide written, informed consent or the availability of a suitable relative or welfare guardian/attorney who is able to provide informed consent on behalf of the participant.

ICU

• Aged 18 or over
• Capacity to provide written, informed consent or the availability of a suitable relative or welfare guardian/attorney who is able to provide informed consent on behalf of the patient.

Exclusion Criteria:

• General Wards

  • Unable to understand spoken task instructions
  • Vision or hearing impairment severe enough to preclude testing or interview
  • Photosensitive epilepsy

ICU

• Unable to understand spoken task instructions
• Pre-existing, known cognitive impairment including dementia
• Vision or hearing impairment severe enough to preclude testing or interview
• Photosensitive epilepsy
• A score on the Richmond Agitation-Sedation Scale of below -3

Study Plan

How is the study designed?

Number of groups / cohorts

7

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
Hospitalised patients with delirium; hospitalised patients with delirium in general wards.	Other: No intervention given to participants; This is an observational study. Participants will receive no intervention.
Hospitalised patients with dementia/ no delirium; Hospitalised patients with a diagnosis of dementia who do not have delirium in general wards.	Other: No intervention given to participants; This is an observational study. Participants will receive no intervention.
Hospitalised patients with no delirium or dementia; Hospitalised patients with no delirium or dementia in general wards.	Other: No intervention given to participants; This is an observational study. Participants will receive no intervention.
Outpatients with dementia; People with a diagnosis of dementia who are living in the community.	Other: No intervention given to participants; This is an observational study. Participants will receive no intervention.
Healthy volunteers; Healthy volunteers who are living in the community who do not have a cognitive impairment.	Other: No intervention given to participants; This is an observational study. Participants will receive no intervention.
ICU delirium; Patients with delirium who are hospitalised in the intensive care units.	Other: No intervention given to participants; This is an observational study. Participants will receive no intervention.
ICU no delirium; Patients who do not have delirium who are hospitalised in intensive care units.	Other: No intervention given to participants; This is an observational study. Participants will receive no intervention.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Diagnostic accuracy of the DelApp for diagnosing delirium	DelApp score against delirium diagnosed using DSM-5 criteria in the whole inpatient sample.	Baseline
Diagnostic accuracy of the DelApp to discriminate between delirium (with or without dementia) and dementia (without delirium).	DelApp score against delirium diagnosed using DSM-5 criteria in groups with dementia and delirium.	Baseline

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Accuracy of DelApp in measuring delirium severity	DelApp score against score on Delirium Rating Scale revised 98 (DRS-R98).	Baseline
Length of stay	length of stay in hospital	up to 3 months
Mortality	Mortality at three months post assessment	3 months
Discharge location	Where patient is discharged to after hospital. e.g home, care home.	up to 3 months
Accuracy of DelApp to measure within-subject change in attentional functioning and delirium status over time	Comparision of DelApp scores to DSM-5 criteria for delirium.	2 weeks
Inter-rater agreement of the DelApp	Assessed by comparing researcher assessment scores to the scores of clinical staff who are delirium specialists.	Baseline

Collaborators and Investigators

• Sponsor: University of Edinburgh
• Collaborators: Medical Research Council; NHS Lothian; NHS Greater Glasgow and Clyde; University of Glasgow; Cambridge Cognition Ltd
• Investigators: Principal Investigator: Elizabeth Wilson, Royal Infirmary of Edinburgh; Principal Investigator: Jennifer Barnett, Cambridge Cognition Ltd

Publications and helpful links

General Publications

• Rutter LM, Nouzova E, Stott DJ, Weir CJ, Assi V, Barnett JH, Clarke C, Duncan N, Evans J, Green S, Hendry K, McGinlay M, McKeever J, Middleton DG, Parks S, Shaw R, Tang E, Walsh T, Weir AJ, Wilson E, Quasim T, MacLullich AMJ, Tieges Z. Diagnostic test accuracy of a novel smartphone application for the assessment of attention deficits in delirium in older hospitalised patients: a prospective cohort study protocol. BMC Geriatr. 2018 Sep 17;18(1):217. doi: 10.1186/s12877-018-0901-5.

Study record dates

Study Major Dates

• Study Start (ACTUAL): October 12, 2015
• Primary Completion (ACTUAL): March 22, 2018
• Study Completion (ACTUAL): March 22, 2018

Study Registration Dates

• First Submitted: October 14, 2015
• First Submitted That Met QC Criteria: October 28, 2015
• First Posted (ESTIMATE): October 29, 2015

Study Record Updates

• Last Update Posted (ACTUAL): May 24, 2018
• Last Update Submitted That Met QC Criteria: May 23, 2018
• Last Verified: May 1, 2018

More Information

Terms related to this study

• Keywords: Dementia; Diagnosis; Delirium; Medical device; Validation study; Inattention; Software application
• Additional Relevant MeSH Terms: Mental Disorders; Brain Diseases; Central Nervous System Diseases; Nervous System Diseases; Neurologic Manifestations; Confusion; Neurobehavioral Manifestations; Neurocognitive Disorders; Delirium; Dementia
• Other Study ID Numbers: DelApp