A Mechanism Randomised Controlled Trial of a Three-Agent LLM-Augmented mHealth Intervention for Late-Life Loneliness in Older Adults

Loneliness in later life is associated with adverse physical and mental health outcomes. Conversational applications based on large language models have been proposed as a scalable form of support, but most published evaluations remain at the pilot feasibility stage, and few have used a comparator that holds interface richness, attention, and novelty constant. Without such a comparator, any observed benefit cannot be attributed to the language model rather than to the surrounding experience.

This trial addresses that gap using an architecturally matched comparator. Both arms deliver the identical mobile application: the same visual interface, navigation, three named conversational companions, tool layer, and in-app assessment prompts. The sole difference is the backend that generates companion response content. In the Hybrid arm, responses are generated by a large language model using each companion's system prompt, conversation history, and user input, and the arm exhibits language-model-distinctive conversational behaviours including anchoring on specific content, cross-session memory, explicit admission of unfamiliarity, mixed-content routing, and generative summarisation. In the Rule-based arm, responses are produced by a template system driven by keyword matching and conversational state, without cross-session memory and without those behaviours; conversational content in this arm is not transmitted to any external service.

Seventy-two community-dwelling Cantonese-speaking adults aged 60 or above who score at least 2 on the De Jong Gierveld Emotional and Social Loneliness Scale are allocated 1:1, stratified by baseline emotional loneliness using permuted blocks within stratum, with allocation concealment. A designated staff member performs randomisation and onboarding and does not collect outcome data. The Principal Investigator conducts all in-person assessments while unaware of allocation. Participants are not informed which version they use and are debriefed at study completion.

The intervention period is four weeks, followed by four weeks of follow-up with continued application access and continuous usage logging. The primary analysis is a longitudinal 1-1-1 multilevel mediation model with arm as the between-person predictor, session-level perceived responsiveness as the within-person mediator, and emotional loneliness as the outcome; the indirect effect is tested using Monte Carlo confidence intervals. The study is powered for the indirect effect rather than for confirmatory testing of the between-arm main effect, which is reported as an effect estimate with a 95% confidence interval. A falsifiable specification of the proposed mechanism is that an arm advantage appears on emotional loneliness but not on social loneliness.

A three-layer safety architecture operates in both arms: a Cantonese-calibrated distress detector that surfaces Hong Kong crisis resources in real time and alerts the research team; a daily 100% audit of Thought Exercise events against pre-specified scope criteria; and weekly Principal Investigator review of all flags and audits. Pre-specified pause criteria are an intervention-related serious adverse event, boundary-crossed events exceeding 10%, or distress detector recall below 0.90 on a real Cantonese corpus.