Large Language Model for Understanding and Monitoring Elderly Neurocognition (LUMEN)

LUMEN (Large Language Model for Understanding and Monitoring Elderly Neurocognition): A Clinical Feasibility Study With Nested Qualitative Evaluation for AI- Assisted Dementia Assessment

Dementia affects millions of people worldwide, and early diagnosis is essential for getting the right care and support. Doctors rely on collateral histories (accounts from family members or caregivers) to understand changes in a person's memory and thinking. However, these histories can be incomplete, unstructured, or difficult to obtain, making diagnosis more challenging.

This study will test LUMEN (Large Language Model for Understanding and Monitoring Elderly Neurocognition), an AI-powered conversation tool designed to help caregivers describe their loved one's symptoms more effectively. By asking structured questions and guiding the conversation, LUMEN can create clear, well-organised reports for memory clinic doctors. This could make assessments quicker, more accurate, and less stressful for families.

We will test LUMEN in real-world clinics by asking caregivers and doctors to use it and provide feedback. We want to understand how easy it is to use, whether it could improve the quality of information shared, and how it fits into existing NHS memory clinic processes. We will also run co-production workshops with community groups to ensure the tool is accessible to people from diverse cultural and language backgrounds.

This research is exciting because it explores how artificial intelligence can improve dementia care. If successful, LUMEN could enhance the diagnostic process, reduce carer burden, and help more people access dementia support sooner. In the future, this tool could be used nationwide in memory clinics, improving care for thousands of families.

Study Overview

• Status: Recruiting
• Conditions: Dementia
• Intervention / Treatment: Other: LUMEN prototype software interaction

Detailed Description

1. Background and Rationale Dementia diagnosis relies on clinical history, cognitive assessments, and collateral information from caregivers. However, obtaining structured and reliable collateral histories is challenging due to caregiver burden, recall bias, and time constraints in memory clinics. Large language models (LLMs) have the potential to improve this process by guiding structured history-taking, extracting clinically relevant details, and standardising information gathering.

   LUMEN (Large Language Model for Understanding and Monitoring Elderly Neurocognition) is an AI-powered conversational tool designed to assist caregivers in providing structured collateral histories before clinical dementia assessments. Seed-funded by the Royal College of Psychiatrists, the Alzheimer's Research UK Northern Network and the Newcastle Biomedical Research Network, LUMEN was developed to enhance diagnostic accuracy, reduce caregiver burden, and improve workflow efficiency in memory clinics.

   This study will evaluate the feasibility, usability, and acceptability of LUMEN in real-world clinical settings through a nested qualitative feasibility study. We will recruit caregivers and clinicians to interact with LUMEN and assess its effectiveness in gathering collateral histories. The study will also incorporate co-production workshops with community groups to explore issues around language, cultural inclusivity, and accessibility. Findings will inform future refinements and a larger validation study.

2. Research Objectives

This study aims to:

1. Evaluate LUMEN's usability -Measure caregiver and clinician experiences using validated scales (System Usability Scale [SUS] and NASA Task Load Index [NASA-TLX]).
2. Explore acceptability and implementation barriers and facilitators-Conduct qualitative interviews and coproduction workshops to refine LUMEN's design, ensuring accessibility across diverse user groups.
3. Compare LUMEN's outputs to clinician assessments-Investigate the alignment between LUMEN-generated reports and clinician-elicited histories using Cohen's Kappa for inter-rater reliability.

3. Background Work Undertaken LUMEN was co-developed with clinicians and stakeholders using a Modified Delphi process involving 150 clinicians, primarily old age psychiatrists. This produced 2,600 questions mapped to validated dementia tools, ensuring clinical relevance. Early technical testing evaluated open-source LLMs (e.g., Mistral 7B, LLaMA2 7B) using simulated clinical vignettes. ROC curve analysis demonstrated an AUC of 0.89 for distinguishing dementia-related narratives, and Cohen's Kappa 0.82 indicated high inter-rater reliability between LUMEN's outputs and clinician ratings. Early PPIE (Patient and Public Involvement and Engagement) feedback highlighted the need for simpler language, cultural sensitivity, and improved user experience. These insights will be further refined through the proposed co-production workshops.

4. Study Design and Methodology This study will employ a mixed-methods approach, combining quantitative usability testing with qualitative interviews to evaluate LUMEN's feasibility and acceptability.

4.1. Participants and Recruitment

• Carers (n=20-30 dyads): Recruited via Northumbria NHS Memory Clinics. Eligible carers will be adults (≥18 years) who know the patient well, have basic English literacy, and can provide informed consent.
• Patients (n=20-30): Individuals (≥65 years) attending a memory clinic for cognitive assessment.
• Clinicians (n=8-10): Specialists in dementia care (neurologists, psychiatrists, geriatricians, advanced nurse practitioners) with ≥2 years of experience.

4.2. Study Procedures

1. Baseline Data Collection:

   • Demographics (age, gender, ethnicity, socioeconomic status).
   • Clinical diagnosis (where available).
   • Cognitive test scores (MoCA/ACE).

2. LUMEN Interaction:

   • Carers will use LUMEN on a laptop or tablet to provide structured collateral histories (~20-30 min).
   • Clinicians will review LUMEN-generated histories to assess completeness and clinical utility.

3. Usability and Cognitive Load Assessment:

   • SUS (System Usability Scale): 10-item Likert-scale questionnaire assessing ease of use (score ≥70 = good usability).
   • NASA-TLX (Task Load Index): Evaluates perceived cognitive workload (scores 0-29 = low workload).

4. Qualitative Evaluation:

   • Semi-structured interviews (n=10 carers, n=4-5 clinicians).
   • Co-production workshops with community groups to explore language, cultural representation, and accessibility.

5. Exploratory Analysis:

   • Inter-rater reliability (Cohen's Kappa): Agreement between LUMEN outputs and clinician assessments.
   • Diagnostic accuracy (AUROC curve analysis): Comparison of LUMEN's assessments with clinical diagnoses.

5. Milestones and Timescales

The study will run for 12 months (April 2025 - April 2026), with key phases as follows:

Phase Month Activities Preparation & Ethics 1-3 Final IRAS approval, NHS R&D approvals, team setup Recruitment & Data Collection 4-9 Patient-carer dyad recruitment, clinician recruitment, LUMEN testing Usability Workshops 6-8 Co-production workshops, qualitative interviews Data Analysis & Refinement 9-10 Thematic analysis, SUS/NASA-TLX scoring, LLM refinements Final Reporting & Next Steps 11-12 Dissemination, grant preparation for large-scale validation study 6. Expected Impact This study will generate critical feasibility data to inform a larger clinical validation study and subsequent NHS integration. Expected outcomes include:

• A refined, co-designed LUMEN prototype with improved usability and accessibility.
• Quantitative evidence on usability and cognitive workload, supporting future funding applications.
• Preliminary insights into AI-assisted collateral history-taking, laying the foundation for regulatory approval as a Software as a Medical Device (SaMD).
• Potential for early NHS adoption under the MHRA framework, streamlining dementia diagnostics.

• Study Type: Observational
• Enrollment (Estimated): 60

Contacts and Locations

• Study Contact: Name: Judith R Harrison, MBChB; Phone Number: +441912081314; Email: Judith.Harrison@newcastle.ac.uk
• Study Contact Backup: Name: Lisa Gallagher; Phone Number: +44191 293 4087; Email: Lisa.gallagher@northumbria-healthcaretrust.nhs.uk

Study Locations

• United Kingdom
  • North Shields, United Kingdom, NE29 8NH
    • Recruiting
    • North Tyneside General Hospital is Rake Lane
    • Contact: Lorelle Dismore; Phone Number: +44191 293 4087; Email: lorelle.dismore@northumbria-healthcare.nhs.uk

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Older Adult
• Accepts Healthy Volunteers: No

• Sampling Method: Non-Probability Sample

Study Population

Patient and carer dyads: patients attending the Memory Clinic for assessment of cognitive problems.

Clinicians: specialist doctors or advanced nurse practitioners responsible for dementia diagnosis

Description

Carers of patients attending Memory Clinic appointments; Aged 18 or over; Both patients and cares can provide informed consent; Basic English literacy

Clinicians must have at least 2 years of specialist dementia care experience and be personally responsible for dementia diagnosis; Must be provide informed consent

Study Plan

How is the study designed?

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
Patient-carer dyads; Carers (n=20-30 dyads): Recruited via Northumbria NHS Memory Clinics. Eligible carers will be adults (≥18 years) who know the patient well, have basic English literacy, and can provide informed consent. • Patients (n=20-30): Individuals (≥65 years) attending a memory clinic for cognitive assessment.; Baseline Data Collection:Demographics (age, gender, ethnicity, socioeconomic status).Clinical diagnosis (where available).Cognitive test scores (MoCA/ACE).; LUMEN Interaction:o Carers will use LUMEN on a laptop or tablet to provide structured collateral histories (~20-30 min).; Usability and Cognitive Load Assessment:SUS (System Usability Scale): 10-item Likert-scale questionnaire assessing ease of use (score ≥70 = good usability).NASA-TLX (Task Load Index): Evaluates perceived cognitive workload (scores 0-29 = low workload).; Qualitative Evaluation:Semi-structured interviews (n=10 carers).	Other: LUMEN prototype software interaction; This is a prototype software which seeks to gather collateral information relevant to a dementia clinical assessment.
Clinicians; 4.1. Participants and Recruitment • Clinicians (n=8-10): Specialists in dementia care (neurologists, psychiatrists, geriatricians, advanced nurse practitioners) with ≥2 years of experience. 4.2. Study Procedures; Baseline Data Collection:o Demographics (age, gender, ethnicity, socioeconomic status).; LUMEN Interaction:o Clinicians will review LUMEN-generated histories to assess completeness and clinical utility.; Usability and Cognitive Load Assessment:SUS (System Usability Scale): 10-item Likert-scale questionnaire assessing ease of use (score ≥70 = good usability).NASA-TLX (Task Load Index): Evaluates perceived cognitive workload (scores 0-29 = low workload).; Qualitative Evaluation:Semi-structured interviews (n=4-5 clinicians).	Other: LUMEN prototype software interaction; This is a prototype software which seeks to gather collateral information relevant to a dementia clinical assessment.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
SUS (System Usability Scale)	10-item Likert-scale questionnaire assessing ease of use (score ≥70 = good usability).	Immediately after use of software
NASA-TLX (Task Load Index):	Evaluates perceived cognitive workload (scores 0-29 = low workload).	Immediately after prototype interaction

Other Outcome Measures

Outcome Measure	Measure Description	Time Frame
Thematic analysis	Qualitative analysis of dialogues from semi-structured interviews	1-10 days after use of prototype software

Collaborators and Investigators

• Sponsor: Northumbria Healthcare NHS Foundation Trust
• Collaborators: Newcastle University
• Investigators: Principal Investigator: Judith R Harrison, MBChB PhD, Northumbria Healthcare NHS Foundation Trust

Publications and helpful links

Helpful Links

• Related Info

Study record dates

Study Major Dates

• Study Start (Actual): March 19, 2026
• Primary Completion (Estimated): December 31, 2026
• Study Completion (Estimated): December 31, 2026

Study Registration Dates

• First Submitted: January 8, 2026
• First Submitted That Met QC Criteria: January 8, 2026
• First Posted (Actual): January 16, 2026

Study Record Updates

• Last Update Posted (Actual): April 21, 2026
• Last Update Submitted That Met QC Criteria: April 20, 2026
• Last Verified: March 1, 2026

More Information

Terms related to this study

• Keywords: Alzheimer's Disease; Dementia; Artificial Intelligence; Large Language Models
• Additional Relevant MeSH Terms: Brain Diseases; Central Nervous System Diseases; Nervous System Diseases; Mental Disorders; Neurocognitive Disorders; Tauopathies; Neurodegenerative Diseases; Alzheimer Disease; Dementia
• Other Study ID Numbers: NHCT0571

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: UNDECIDED
• IPD Plan Description: This will include personal narratives of participants' dementia journeys which may contain sensitive information

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No