Feasibility Study of a Conversational Chatbot for Outpatient Postoperative Follow-up in Otorhinolaryngology (ENT): Acceptability, Patterns of Use, and Patient Experience. (ORL-CareBOT)
Artificial intelligence (AI), and particularly large language models (LLMs), is playing an increasingly important role in medical practice by facilitating documentation, data analysis, and communication with patients. These tools may improve access to medical information and reduce certain repetitive tasks; however, their use remains limited by risks of errors, hallucinations, variability in responses, and the lack of evaluation in real-world healthcare settings.
In the field of patient-provider communication, several studies have shown that AI-generated responses may be perceived as being as relevant as, or even more empathetic and clearer than, those provided by physicians. However, these findings are based predominantly on simulated scenarios and do not allow conclusions regarding the safety of autonomous use in clinical practice.
Managing patient inquiries outside scheduled consultations represents a substantial workload for healthcare teams. Generative AI could help address this demand, but its implementation requires medical oversight and rigorous safety evaluation.
The postoperative ENT setting provides a relevant area for investigation, as patients frequently experience symptoms that can be difficult to interpret, such as pain, bleeding, or wound-healing complications. A conversational agent could provide consistent responses and continuous accessibility; however, it should never replace on-call medical services. Its primary requirement is the ability to correctly identify situations requiring prompt or urgent medical assessment.
With this objective, a postoperative chatbot was developed using ChatGPT-4 through a secure Retrieval-Augmented Generation (RAG) architecture. Its knowledge base was restricted to recommendations from the French National Authority for Health (HAS), the French Society of Otorhinolaryngology (SFORL), and the PubMed literature in order to improve response reliability. An internal feasibility study yielded encouraging results: 68% of responses were rated as good to excellent by ENT experts, with writing quality and completeness sometimes exceeding those of practicing physicians, although clinicians remained superior in the nuanced assessment of clinical risk.
The objective of the present study is to evaluate the feasibility and acceptability of this ENT postoperative follow-up chatbot among 100 ambulatory surgery patients treated at the Arthur Vernes Institute. The primary aim is to assess the patient experience before evaluating any potential impact on healthcare utilization or medico-economic outcomes.
Study Overview
Status
Status
Conditions
Conditions
Intervention / Treatment
Intervention / Treatment
Study Type
Study Type
Enrollment (Estimated)
Enrollment
Phase
Phase
- Not Applicable
Contacts and Locations
Study Contact
Study Contact
- Name: juliette courtiade mahler, pHD
- Phone Number: +33144127963
- Email: jcourtiade@ghpsj.fr
Study Locations
-
-
-
Paris, France
- Institut Arthur Vernes
-
Contact:
- Jean-Baptiste Lecanu, MD
- Phone Number: +33682137107
- Email: jblecanu@institut-vernes.fr
-
-
Participation Criteria
Eligibility Criteria
Eligibility Criteria
Ages Eligible for Study
Accepts Healthy Volunteers
Genders Eligible for Study
Description
Inclusion Criteria:
- Patient aged 18 years or older
- Patient undergoing outpatient ENT surgery at the Arthur Vernes Institute
- Patient covered by a health insurance scheme
- French-speaking patient
- Patient who has provided free, informed, and explicit oral consent
Exclusion Criteria:
- Patient already enrolled in a Category 1 interventional research protocol (RIPH 1)
- Patient without access to equipment enabling use of the chatbot (smartphone or computer)
- Patient under guardianship or curatorship
- Patient deprived of liberty
- Patient under judicial protection (safeguard of justice)
- Pregnant or breastfeeding patient
Study Plan
How is the study designed?
Design Details
- Primary Purpose: Other
- Allocation: N/A
- Interventional Model: Single Group Assignment
- Masking: None (Open Label)
Number of Arms
Arms and Interventions
Participant Group / ArmParticipant Group / Arm |
Intervention / TreatmentIntervention / Treatment |
|---|---|
|
Experimental: Post op Care
at D30 post surgery, patients will complete 2 scores : Lickert scale and SUS |
Patient will be given access to a post surgery Chat Bot. All the questions asked to the chat bot will be analyzed. At D30 patients will complete a likert scale (0 to 5) and the System usabilty score (0 to 100) |
What is the study measuring?
Primary Outcome Measures
Primary Outcome Measures
Outcome Measure |
Measure Description |
Time Frame |
|---|---|---|
|
Determine the effective use of the chatbot
Time Frame: up to 30days post surgery
|
Proportion of patients having use the chat bot at least once within the 30Days post surgery |
up to 30days post surgery
|
Secondary Outcome Measures
Secondary Outcome Measures
Outcome Measure |
Measure Description |
Time Frame |
|---|---|---|
|
Describe how users can interact with the chatbot.
Time Frame: up to 30days post surgery
|
number of interaction with the chatbot, topic of the questions |
up to 30days post surgery
|
|
Assess patient acceptance of the chatbot
Time Frame: 30days post surgery
|
System acceptability scale (from 0 to 100) |
30days post surgery
|
|
Assess patient satisfaction
Time Frame: 30days post surgery
|
likert scale (1 to 5) |
30days post surgery
|
|
Assess the organizational impact
Time Frame: 30days post surgery
|
number of phone call made during the 30days follow up period after surgery |
30days post surgery
|
|
Describe clinical events occurring during the follow-up period
Time Frame: 30days post surgery
|
number of out patient visit and hospital stay during the 30days post surgery follow up period |
30days post surgery
|
Collaborators and Investigators
Sponsor
Sponsor
Study record dates
Study Major Dates
Study Start (Estimated)
Study Start
Primary Completion (Estimated)
Primary Completion
Study Completion (Estimated)
Study Completion
Study Registration Dates
First Submitted
First Submitted
First Submitted That Met QC Criteria
First Submitted That Met QC Criteria
First Posted (Actual)
First Posted
Study Record Updates
Last Update Posted (Actual)
Last Update Posted
Last Update Submitted That Met QC Criteria
Last Update Submitted That Met QC Criteria
Last Verified
Last Verified
More Information
Terms related to this study
Other Study ID Numbers
Other Study ID Numbers
- 853_ORL-CareBOT
Plan for Individual participant data (IPD)
Plan to Share Individual Participant Data (IPD)?
Drug and device information, study documents
Studies a U.S. FDA-regulated drug product
Studies a U.S. FDA-regulated device product
This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.