Integrating ChatGPT in Anesthesia

Implementation of Large Language Models in Anesthesia to Answer Patients' Questions During Pre-Anesthesia Visits: A Prospective, Observational Study

Patients are waiting for surgery longer than ever before. While hospitals are trying to get patients access to surgery as quickly as possible, a shortage of anesthesia doctors (called anesthesiologists) prevents this. Every surgery needs an anesthesiologist, but anesthesiologists also assess patients beforehand to ensure they can safely undergo surgery and help patients understand what to expect and the risks involved. Ensuring that patients move through surgery as safely and efficiently as possible requires that the limited number of anesthesiologists in Canada work as efficiently as possible. Technology like Artificial Intelligence (AI), in programs such as ChatGPT, might assist by answering common questions patients have, allowing anesthesiologists to focus their limited time on addressing complex and personalized issues for each patient. However, studies conducted so far on how well AI can answer questions about anesthesia and surgery have been poorly designed and have not included patients as members of the research team. This study will determine whether AI can answer patients' questions before surgery as effectively as both patients and expert anesthesiologists would expect, which could be beneficial to patients and the healthcare system.

Study Overview

• Status: Not yet recruiting
• Conditions: Artificial Intelligence; Anesthesia; Preoperative Care; Patient Education

• Study Type: Observational
• Enrollment (Estimated): 190

Contacts and Locations

• Study Contact: Name: Arnaud Romeo, Mbadjeu Hondjeu, MD; Phone Number: 613-737-8187; Email: ambadjeu@toh.ca
• Study Contact Backup: Name: Daniel McIsaac, MD; Phone Number: 613-737-8187; Email: dmcisaac@toh.ca

Study Locations

• Canada
  • Ontario
    • Ottawa, Ontario, Canada, ON K1Y 4E9
      • Ottawa Hospital Research Institute
      • Contact: Daniel McIsaac, MD; Phone Number: 613-737-8187; Email: dmcisaac@toh.ca
      • Contact: Arnaud Romeo Mbadjeu Hondjeu, MD; Phone Number: 613-737-8187; Email: ambadjeu@toh.ca

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: Yes

• Sampling Method: Probability Sample

Study Population

Adults having an in-person preoperative anesthesiology consultation before elective surgery at The Ottawa Hospital

Description

Inclusion Criteria:

Patients

• 1. Age > 18 years 2. Elective non-cardiac surgery 3. In-person preoperative anesthesiology consultation 4. Ability to participate and provide informed consent independently.

Exclusion Criteria:

• 1. Unable to communicate in English or French 2. Nurse consultation only

Physicians

Inclusion Criteria:

Anesthesiology residents or staff providing service at the PAU at TOH

Exclusion Criteria:

Anesthesiology residents or staff unwilling to participate in the research study.

Study Plan

How is the study designed?

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort
Patient scheduled for a pre-surgery consultation with a physician anesthesiologist; After obtaining consent, patients will be asked to respond to the following query: "What is the most important question you had hoped to ask your anesthesiologist today about your anesthesia care, pain management or the time immediately around surgery?'.
Specialty-trained anesthesiologists in independent practice within the pre-admission; anesthesiologists will be designated as either "response generators" or "response raters." Response generators will craft answers to patient queries, while response raters-anesthesiologists with over five years of clinical experience-will evaluate the quality of these responses.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
knowledge	Patients will be blinded to the source of the two responses (GPT-4 and Anesthesiologist). After reading both responses, they will complete a questionnaire that includes an 11-point Likert-scale question for the primary outcome of knowledge: "To what extent does the response address the knowledge or information you hoped to gain?" (0 = not at all addressed; 10 = fully addressed)	From the date of the pre-anesthesia consultation until 2 weeks after the consultation, assessed over a period of up to 2 weeks post-consultation

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Patient satisfaction	Secondary Outcome: Patient satisfaction, Patient satisfaction will be assessed using a likelihood to recommend measurement based on a 11-point Likert scale (Question: Thinking about the response you received to your question, how likely are you to recommend the response's provider to a family member or friend going for a similar surgery? 0-not at all likely; 10-extremely likely).	From the date of the pre-anesthesia consultation until 2 weeks after the consultation, assessed over a period of up to 2 weeks post-consultation

Other Outcome Measures

Outcome Measure	Measure Description	Time Frame
Tertiary outcomes	Three tertiary outcomes will be assessed using an 11-point Likert scale: 1. Health Literacy:Assessed through the prompt: "I have sufficient information to make decisions about my surgery." 2. Perceived Empathy:Evaluated with the question: "To what extent did you find the response empathetic?" 3. Completeness: Measured by the question: "To what extent did you find the response complete?	From the date of the pre-anesthesia consultation until 2 weeks after the consultation, assessed over a period of up to 2 weeks post-consultation

Collaborators and Investigators

• Sponsor: Ottawa Hospital Research Institute

Study record dates

Study Major Dates

• Study Start (Estimated): December 9, 2024
• Primary Completion (Estimated): May 30, 2025
• Study Completion (Estimated): June 30, 2025

Study Registration Dates

• First Submitted: October 17, 2024
• First Submitted That Met QC Criteria: October 18, 2024
• First Posted (Actual): October 22, 2024

Study Record Updates

• Last Update Posted (Actual): October 22, 2024
• Last Update Submitted That Met QC Criteria: October 18, 2024
• Last Verified: October 1, 2024

More Information

Terms related to this study

• Other Study ID Numbers: 20240526-01H

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO
• IPD Plan Description: The data collected from this study will not be shared with other researchers, however the data may be used to inform and guide future research on the integration of AI technologies in clinical practice, particularly in improving patient communication and decision-making during pre-anesthesia consultations.

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No