Feasibility Study of a Conversational Chatbot for Outpatient Postoperative Follow-up in Otorhinolaryngology (ENT): Acceptability, Patterns of Use, and Patient Experience. (ORL-CareBOT)

Artificial intelligence (AI), and particularly large language models (LLMs), is playing an increasingly important role in medical practice by facilitating documentation, data analysis, and communication with patients. These tools may improve access to medical information and reduce certain repetitive tasks; however, their use remains limited by risks of errors, hallucinations, variability in responses, and the lack of evaluation in real-world healthcare settings.

In the field of patient-provider communication, several studies have shown that AI-generated responses may be perceived as being as relevant as, or even more empathetic and clearer than, those provided by physicians. However, these findings are based predominantly on simulated scenarios and do not allow conclusions regarding the safety of autonomous use in clinical practice.

Managing patient inquiries outside scheduled consultations represents a substantial workload for healthcare teams. Generative AI could help address this demand, but its implementation requires medical oversight and rigorous safety evaluation.

The postoperative ENT setting provides a relevant area for investigation, as patients frequently experience symptoms that can be difficult to interpret, such as pain, bleeding, or wound-healing complications. A conversational agent could provide consistent responses and continuous accessibility; however, it should never replace on-call medical services. Its primary requirement is the ability to correctly identify situations requiring prompt or urgent medical assessment.

With this objective, a postoperative chatbot was developed using ChatGPT-4 through a secure Retrieval-Augmented Generation (RAG) architecture. Its knowledge base was restricted to recommendations from the French National Authority for Health (HAS), the French Society of Otorhinolaryngology (SFORL), and the PubMed literature in order to improve response reliability. An internal feasibility study yielded encouraging results: 68% of responses were rated as good to excellent by ENT experts, with writing quality and completeness sometimes exceeding those of practicing physicians, although clinicians remained superior in the nuanced assessment of clinical risk.

The objective of the present study is to evaluate the feasibility and acceptability of this ENT postoperative follow-up chatbot among 100 ambulatory surgery patients treated at the Arthur Vernes Institute. The primary aim is to assess the patient experience before evaluating any potential impact on healthcare utilization or medico-economic outcomes.