Designing a Large Language Model Architecture for Shared Decision Making and Informed Intentions (DEONCAI3-1)

DEONCAi 3-1: Can Large Language Models Support Shared Decision-Making and Informed Intentions in the Field: An Online Experiment Comparing Them to a Standard Model and to an Evidence-Based Decision Aid

This study investigates a novel, agentic Large Language Model (LLM) architecture designed to facilitate Shared Decision-Making (SDM) in healthcare. While patients increasingly use standard LLMs for health information, these models often struggle with multi-turn conversations and fail to adapt to varying reading levels, disadvantaging vulnerable groups. By utilizing an agentic state-machine, this project aims to overcome common LLM deficits-such as context loss and uncritical agreement-to ensure clinically accurate, participatory patient conversations. The study evaluates whether this architecture improves informed decision-making compared to standard LLMs, particularly for patients with low health literacy.

Study Overview

• Status: Not yet recruiting
• Conditions: Artificial Intelligence; Patient Participation; Health Literacy; Shared Decision Making
• Intervention / Treatment: Behavioral: Agentic LLM Chatbot; Behavioral: Usual Care LLM Chatbot; Behavioral: IQWiG Standard Brochure

Detailed Description

Background & Current State of Research Shared Decision-Making is widely recognized as the gold standard of patient-centered care. However, its successful implementation in clinical practice is frequently hindered by systemic time and budget constraints. Consequently, patients are increasingly turning to Large Language Models to independently access health information and navigate their medical options.

The Problem: Limitations of Standard LLMs Recent studies indicate that using standard LLMs can digitally reproduce existing health inequalities and participation gaps. These models struggle to adapt to different reading levels without a significant loss in quality. While highly developed prompting techniques can enhance clinical accuracy, the results remain highly variable depending on the specific model and technique used.

A central deficit becomes apparent in multi-turn interactions, which are essential for a natural SDM workflow. In these prolonged conversations, standard LLMs tend to exhibit "context rot," leading to a sharp decline in accuracy as the dialogue progresses. This issue is further exacerbated by model sycophancy (an uncritical tendency to agree with the user) and completion eagerness (a tendency to prematurely conclude the conversation). Because current LLMs often fail to meet evidence-based reporting standards and lack systematic validation of user comprehension, the burden of fact-checking remains entirely on the user. This dynamic significantly disadvantages vulnerable groups, particularly those with lower health literacy.

Project Objectives and Research Questions

This project proposes a technical and clinical solution to these challenges through the following core research questions:

Technical-Conceptual Framework: How can we develop a transparent and replicable workflow that formalizes the SDM process through an agentic state-machine, thereby eliminating structural deficits of LLMs (e.g., context rot and sycophancy) in prolonged interactions?

Clinical Evaluation (Primary Objective): Does this newly developed agentic LLM architecture achieve higher scores in participatory conversation management and demonstrate a significantly greater objective increase in informed decision-making among users during multi-turn interactions, compared to a standard baseline LLM?

Health Equity (Secondary Objective): To what extent can this agentic LLM architecture guarantee effective participatory conversation management for vulnerable patient groups with low health literacy?

• Study Type: Interventional
• Enrollment (Estimated): 330
• Phase: Not Applicable

Contacts and Locations

• Study Contact: Name: Felix G Rebitschek, PhD; Phone Number: +493319772162; Email: felix.rebitschek@fgw-brandenburg.de
• Study Contact Backup: Name: Martin Lipsdorf; Email: martin.lipsdorf@mhb-fontane.de

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

• Biological sex: Female.
• Age: 40 to 70 years old.
• Country of residence: United States (US) or United Kingdom (UK).
• Language: Native English speaker (English as first language).
• Registered and verified user on the academic research platform Prolific.
• Able to read, understand, and provide informed consent digitally.

Exclusion Criteria:

• Individuals who do not meet the automated pre-screening criteria on the Prolific platform.
• Inability to use or access a computer, smartphone, or internet browser required to complete the digital study.

Note: Quotas will be enforced during recruitment to ensure a 50/50 stratified split between participants with and without university entrance qualifications to guarantee variance in educational backgrounds. Once a quota is filled, further participants matching that educational profile will be excluded.

-

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

3

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: Shared Decision Making Chatbot; Participants will engage in a multi-turn, AI-assisted consultation about mammography screening. They will interact with a newly developed agentic Large Language Model architecture that uses an integrated state-machine designed to actively guide the Shared Decision-Making process, adapt to the user's reading level, and prevent context loss.	Behavioral: Agentic LLM Chatbot; An interactive AI chatbot built with an agentic state-machine architecture designed to systematically guide users through the Shared Decision-Making process regarding mammography screening, with built-in ethical guardrails to prevent hallucination and sycophancy.
Active Comparator: Standard Chatbot; Participants will engage in a conversation about mammography screening using a standard, usual care Large Language Model (Mistral Large). This model represents the current consumer standard for AI health queries and lacks the specialized agentic state-machine and SDM workflow guidance.	Behavioral: Usual Care LLM Chatbot; An unmodified, standard generative Large Language Model chatbot acting as a baseline control, used by participants to discuss mammography screening.
Active Comparator: Information brochure; Participants will receive and read the standard informational patient brochure on mammography screening published by the German Institute for Quality and Efficiency in Health Care (IQWiG). This represents the current standard of care for patient information.	Behavioral: IQWiG Standard Brochure; The standard digital informational brochure on mammography screening provided by the german Institute for Quality and Efficiency in Health Care (IQWiG), used as a usual care baseline for patient education.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Patient-Reported Quality of Shared Decision-Making	Assessed using the 9-item Shared Decision Making Questionnaire (SDM-Q-9). This validated instrument measures the patient's perceived involvement in the medical decision-making process. The questionnaire consists of 9 items, each rated on a 6-point scale ranging from 0 ("completely disagree") to 5 ("completely agree"). The raw scores are summed and multiplied by 20/9 to yield a total score ranging from 0 to 100. Higher scores indicate a higher perceived quality and greater patient involvement in shared decision-making.	Day 1

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Multidimensional Informed Decision-Making	Based on Marteau's conceptualization of informed decision-making. An informed decision is achieved when a participant has sufficient objective knowledge about mammography screening and their behavioral intention (to screen or not to screen) is congruent with their personal attitudes toward the screening. This is calculated as a composite measure integrating an objective knowledge test, an attitude assessment, and a screening intention question. Higher objective knowledge scores and higher attitude-intention congruence indicate a better-informed decision.	Day 1
Subjective Decisional Conflict	Assessed using a standardized Decisional Conflict Scale. This measures the participant's perceived uncertainty in choosing a screening option, their clarity of personal values, and their feeling of being supported in decision-making. Scores are transformed to a 0 to 100 scale. Lower scores indicate less decisional conflict and greater decision certainty.	Day 1
Medical Accuracy and Safety of Generated Information	A quantitative and qualitative assessment of the medical accuracy of the AI-generated texts. This is operationalized by counting the frequency (number of occurrences) and categorizing the severity of deviations from established medical evidence (i.e., misinformation or "hallucinations"). These deviations are identified via an integrated ethics module and/or expert review. Lower frequencies and lower severity ratings indicate higher medical accuracy.	Day 1

Collaborators and Investigators

• Sponsor: Harding Center for Risk Literacy
• Investigators: Principal Investigator: Felix G Rebitschek, PhD, Harding Center for Risk Literacy

Publications and helpful links

General Publications

• Rebitschek FG, Carella A, Kohlrausch-Pazin S, Zitzmann M, Steckelberg A, Wilhelm C. Evaluating evidence-based health information from generative AI using a cross-sectional study with laypeople seeking screening information. NPJ Digit Med. 2025 Jun 9;8(1):343. doi: 10.1038/s41746-025-01752-6.
• Mendel T, Singh N, Mann DM, Wiesenfeld B, Nov O. Laypeople's Use of and Attitudes Toward Large Language Models and Search Engines for Health Queries: Survey Study. J Med Internet Res. 2025 Feb 13;27:e64290. doi: 10.2196/64290.
• Laban, P., Hayashi, H., Zhou, Y., & Neville, J. (2025). Llms get lost in multi-turn conversation. arXiv preprint arXiv:2505.06120.
• Krenn C, Loder C, Berger N, Jeitler K, Semlitsch T, Siebenhofer A, Wilfling D. Automated Approaches of Text Simplification of Patient Education Materials: Scoping Review. J Med Internet Res. 2026 May 7;28:e88365. doi: 10.2196/88365.
• Keij SM, Branda ME, Montori VM, Brito JP, Kunneman M, Pieterse AH. Patient Characteristics and the Extent to Which Clinicians Involve Patients in Decision Making: Secondary Analyses of Pooled Data. Med Decis Making. 2024 Apr;44(3):346-356. doi: 10.1177/0272989X241231721. Epub 2024 Mar 4.

Study record dates

Study Major Dates

• Study Start (Estimated): September 15, 2026
• Primary Completion (Estimated): September 30, 2026
• Study Completion (Estimated): September 30, 2026

Study Registration Dates

• First Submitted: September 9, 2026
• First Submitted That Met QC Criteria: September 15, 2026
• First Posted (Actual): September 18, 2026

Study Record Updates

• Last Update Posted (Actual): September 18, 2026
• Last Update Submitted That Met QC Criteria: September 15, 2026
• Last Verified: September 1, 2026

More Information

Terms related to this study

• Keywords: Artificial Intelligence; Patient Participation; Informed Decision Making; Shared Decision Making (SDM); Generative AI; Agentic AI; Large Language Models (LLM); Conversational Agents
• Additional Relevant MeSH Terms: Behavior; Treatment Adherence and Compliance; Health Behavior; Patient Acceptance of Health Care; Patient Participation
• Other Study ID Numbers: DEONCAI3-1

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: YES
• IPD Plan Description: De-identified individual participant data (IPD) underlying the results reported in the published article, including survey data (SDM-Q-9, decisional conflict, knowledge scores) and anonymized dialog transcripts, will be shared.
• IPD Sharing Time Frame: Data will become available immediately following publication of the primary results and will be accessible indefinitely.
• IPD Sharing Access Criteria: Data will be made publicly available as supplement material on the publishers website to any researcher or individual for non-commercial research purposes.

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No