Use of Artificial Intelligence by Urogynecologic Patients

Use of Artificial Intelligence by Patients for Understanding of Diagnosis of Urogynecologic Conditions and Its Role in Treatment Decisions

The goal of this clinical trial is to learn about how Urogynecology patients use Artificial Intelligence (AI) Chatbots like ChatGPT, and how it affects healthcare decision making. The main question[s] it aims to answer are:

• How does the AI Chatbot affect participants' understanding of diagnoses and participant satisfaction with a urogynecology consultation?
• How accurate is the chatbot-provided diagnosis and counseling information? Participants will be asked to use the ChatGPT chatbot and ask it questions about the main problem the participant is seeing the doctor for, and will also be asked to fill out some questionnaires.

Researchers will compare using the Chatbot before the visit, after the visit, or not at all to see if the way participants understand the information changes based on timing of use.

Study Overview

• Status: Active, not recruiting
• Conditions: Lower Urinary Tract Symptoms; Urinary Incontinence; Uterovaginal Prolapse
• Intervention / Treatment: Behavioral: Use of ChatGPT

Detailed Description

Artificial Intelligence (AI) in medicine and the use of machine learning to improve patient care and outcomes is a quickly developing field. Interest is building in the use and accuracy of AI chatbot programs such as ChatGPT for patient diagnosis and counseling. A recent study of Chat GPT accuracy compared with patient pamphlets about pelvic organ prolapse found comparable accuracy and completeness.Given the novelty of this field, no current literature exists regarding the use of AI chatbot technology for patient care and patient counseling in Urogynecology.

This will be a single-center, prospective, randomized, non-blinded study examining patient use of AI Chatbot technology (Chat GPT4) at initial visits to supplement understanding of urogynecologic problems. The primary aim of this study is to investigate the effect of use of an AI Chatbot platform on patient understanding of disease processes and treatment options prior to or following a consult with a urogynecologist at the initial visit. The secondary aims are to evaluate the accuracy of the chatbot-provided diagnosis (for participants applicable through randomization) and counseling information, and to evaluate patient satisfaction with the visit.

This study will recruit patients with presenting problems of prolapse, lower urinary tract symptoms, or incontinence into one of three arms: use of an AI chatbot prior to seeing the urogynecologist, use of an AI chatbot following a consult with the urogynecologist, no use of an AI chatbot at the time of the visit. During time of their initial urogynecology visit, data will be collected including demographics, Pelvic Floor Disorders Inventory (PFDI) intake questionnaire data, health literacy, Chat GPT conversation, office consultation diagnoses/treatment, physician questionnaire, and post-consultation questionnaire (Diagnosis and Treatment, Decisional Conflict Scale, Patient Satisfaction, Chatbot Satisfaction). Patients will be asked three months after their visit to complete the post-consultation questionnaire again.

• Study Type: Interventional
• Enrollment (Actual): 125
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • Connecticut
    • Hartford, Connecticut, United States, 06106
      • Hartford Hospital

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: No

Description

Inclusion Criteria:

• female

• presenting for their initial evaluation by a urogynecology physician for one of the following:

  • urinary incontinence (UI)
  • lower urinary tract symptoms (LUTS)
  • pelvic organ prolapse (POP)
• greater than or equal to18 and less than or equal to89 years old
• any race/ethnicity
• able to read or speak English or Spanish
• able/willing to consent to participate

Exclusion Criteria:

• male
• primary presenting problem other than UI, LUTS, or POP
• non-English or non-Spanish speaking
• pregnant or lactating, as this may affect patient treatment counseling
• unable/unwilling to consent to participate

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Other
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

3

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: Pre-Visit ChatGPT Use; After being consented, the participant will be provided with ChatGPT-4 application and a brief orientation to the program. The participant will then be instructed to ask ChatGPT about the participant's primary presenting problem with will be discussed at their urogynecology consultation. The participant will be allowed up to five follow-up/clarification entries into the Chat GPT program, but may finish asking questions at any time. This should take no more than five minutes of the participant's time. After completing this, the participant will be returned to the waiting room, and will proceed through the urogynecology visit as normal.	Behavioral: Use of ChatGPT; Patients will be provided with the opportunity to ask ChatGPT questions about their primary presenting problem at a time point during their initial Urogynecology consultation visit.
Experimental: Post-Visit ChatGPT Use; After being consented, the participant will be returned to the waiting room and will proceed through the urogynecology visit as normal. After the visit, the participant will be provided with ChatGPT-4 application and a brief orientation to the program. The participant will then be instructed to ask ChatGPT about the participant's primary presenting problem with will be discussed at their urogynecology consultation. The participant will be allowed up to five follow-up/clarification entries into the Chat GPT program, but may finish asking questions at any time. This should take no more than five minutes of the participant's time. After completing this, the participant will be allowed to leave the visit.	Behavioral: Use of ChatGPT; Patients will be provided with the opportunity to ask ChatGPT questions about their primary presenting problem at a time point during their initial Urogynecology consultation visit.
No Intervention: No ChatGPT Use; After undergoing the consent process, participants will be returned to the waiting room to await the beginning of the appointment. The participant will complete the urogynecology visit as normal.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Patient understanding of diagnosis and treatment - physician understanding of diagnosis	The effect of use of an AI Chatbot platform on participant understanding of diagnosis at the initial urogynecology visit. This will be measured by the incidence of agreement of participant-reported diagnosis with their physician-reported diagnosis. The physician will be asked the patient's primary diagnosis in one multiple choice question (part of Physician Questionnaire). There are no minimum or maximum values to this questionnaire, and there are no answers that would mean a better or worse outcome.	Baseline
Patient understanding of diagnosis and treatment - physician perception of participant understanding	The effect of use of an AI Chatbot platform on participant understanding of diagnosis at the initial urogynecology visit. This will be measured by the incidence of agreement of participant-reported diagnosis with their physician-reported diagnosis. The physician will be asked about the participants' understanding of their diagnosis and treatment plans in three multiple choice questions (part of Physician Questionnaire). The questionnaire will ask for responses on a Likert Scale 1 - 5, with the minimum value of 1 being "Strongly Disagree" and 5the maximum value of 5 being "Strongly Agree." Higher scores mean a better outcome. Score will be assessed in aggregate with a total possible score of 15.	Baseline
Patient understanding of diagnosis and treatment - participant understanding of diagnosis and treatment plan	The effect of use of an AI Chatbot platform on participant understanding of diagnosis at the initial urogynecology visit. This will be measured by the incidence of agreement of participant-reported diagnosis with their physician-reported diagnosis. The participant will be provided with a three-item questionnaire (Post Consultation Participant Questionnaire). Each item will have multiple choice options. The participant will be asked asked to select the participant's primary diagnosis, treatment options, and selected management plan. There are no minimum or maximum values to this questionnaire, and there are no answers that would mean a better or worse outcome.	Baseline and three month follow-up
Patient understanding of diagnosis and treatment - understanding of diagnosis	The effect of use of an AI Chatbot platform on participant understanding of diagnosis at the urogynecology initial visit. This will be measured by rate of participant agreement to a medical decision making questionnaire (Understanding of Diagnosis Questionnaire). The questionnaire has three items, and will ask for responses on a Likert Scale 1 - 5, with the minimum value of 1 being "Strongly Disagree" and the maximum value of 5 being "Strongly Agree." Higher scores mean a better outcome. Score will be assessed in aggregate with a total possible score of 15.	Baseline and three month follow-up
Patient understanding of diagnosis and treatment - decision making	The effect of use of an AI Chatbot platform on participant decision making at the urogynecology initial visit. This will be measured by rate of participant agreement to a validated medical decision making questionnaire (Decisional Conflict Scale). The questionnaire has sixteen items, and will ask for responses on a Likert Scale 1 - 5, with the minimum value of 1 being "Strongly Disagree" and the maximum value of 5 being "Strongly Agree." Higher scores mean a better outcome. Score will be assessed in aggregate with a total possible score of 80.	Baseline and three month follow-up

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Patient satisfaction - medical visit	The effect of use of an AI Chatbot platform on participant satisfaction with the urogynecology visit. This will be measured by rate of participant agreement to a satisfaction questionnaire (Patient Satisfaction Questionnaire). The questionnaire has four items, and will ask for responses on a Likert Scale 1 - 5, with the minimum value of 1 being "Very Unsatisfied" and the maximum value of 5 being "Very Satisfied." Higher scores mean a better outcome. Score will be assessed in aggregate with a total possible score of 20.	Baseline and three month follow-up
Patient satisfaction - Chatbot	The effect of use of an AI Chatbot platform on participant satisfaction with the urogynecology visit. This will be measured by rate of participant agreement to a satisfaction questionnaire (Chatbot Satisfaction Questionnaire). The questionnaire has six items, and will ask for responses on a Likert Scale 1 - 5, with the minimum value of 1 being "Strongly Disagree" and the maximum value of 5 being "Strongly Agree." Higher scores mean a better outcome. Score will be assessed in aggregate with a total possible score of 30.	Baseline and three month follow-up
Patient Chatbot use after urogynecology visit	The participant use of the Chatbot platform at home after the urogynecology visit. This will be measured by one yes or no question asking if the participant used the Chatbot after the Urogynecology visit (Three Month Questionnaire). The questionnaire has one yes/no item. There are no minimum or maximum values to this questionnaire, and there are no answers that would mean a better or worse outcome.	Three month follow-up
Chatbot information accuracy	The accuracy of chatbot-provided diagnosis and counseling information via expert clinician review. This will be measured by expert review of all chatbot transcripts by two independent clinicians with a third for adjudication with assessment using the Patient Education Materials Assessment Tool (PEMAT). The questionnaire has twenty-six items, and will ask for responses on a two point scale of "Disagree" meaning 0 points, and "Agree" meaning 1 point. Higher scores mean a better outcome. Score will be assessed in aggregate with a total possible score of 26.	Baseline

Collaborators and Investigators

• Sponsor: Hartford Hospital
• Collaborators: Boston Scientific Corporation
• Investigators: Principal Investigator: Nicole J Wood, MD, Hartford Hosptial Division of Urogynecology; Principal Investigator: Elena Tuntisky-Bitton, MD, Hartford Hosptial Division of Urogynecology

Publications and helpful links

General Publications

• Daykan Y, O'Reilly BA. The role of artificial intelligence in the future of urogynecology. Int Urogynecol J. 2023 Aug;34(8):1663-1666. doi: 10.1007/s00192-023-05612-3. Epub 2023 Jul 24.
• Robinson CJ, Swift S, Johnson DD, Almeida JS. Prediction of pelvic organ prolapse using an artificial neural network. Am J Obstet Gynecol. 2008 Aug;199(2):193.e1-6. doi: 10.1016/j.ajog.2008.04.029. Epub 2008 Jun 4.
• Serati M, Salvatore S, Siesto G, Cattoni E, Braga A, Sorice P, Cromi A, Ghezzi F, Bolis P. Urinary symptoms and urodynamic findings in women with pelvic organ prolapse: is there a correlation? Results of an artificial neural network analysis. Eur Urol. 2011 Aug;60(2):253-60. doi: 10.1016/j.eururo.2011.03.010. Epub 2011 Mar 21.
• Grunebaum A, Chervenak J, Pollet SL, Katz A, Chervenak FA. The exciting potential for ChatGPT in obstetrics and gynecology. Am J Obstet Gynecol. 2023 Jun;228(6):696-705. doi: 10.1016/j.ajog.2023.03.009. Epub 2023 Mar 15.
• Johnson CM, Bradley CS, Kenne KA, Rabice S, Takacs E, Vollstedt A, Kowalski JT. Evaluation of ChatGPT for Pelvic Floor Surgery Counseling. Urogynecology (Phila). 2024 Mar 1;30(3):245-250. doi: 10.1097/SPV.0000000000001459.
• Acker A, Senapati S, Dokras A. Barriers to access: findings from an implementation study of an artificial intelligence-augmented 2-way chatbot for fertility care. Fertil Steril. 2023 Jul;120(1):199-201. doi: 10.1016/j.fertnstert.2023.04.016. Epub 2023 Apr 20. No abstract available. Erratum In: Fertil Steril. 2024 Apr;121(4):711. doi: 10.1016/j.fertnstert.2024.01.006.

Study record dates

Study Major Dates

• Study Start (Actual): July 22, 2024
• Primary Completion (Actual): December 16, 2024
• Study Completion (Estimated): April 1, 2025

Study Registration Dates

• First Submitted: June 11, 2024
• First Submitted That Met QC Criteria: June 27, 2024
• First Posted (Actual): July 1, 2024

Study Record Updates

• Last Update Posted (Actual): March 25, 2025
• Last Update Submitted That Met QC Criteria: December 17, 2024
• Last Verified: December 1, 2024

More Information

Terms related to this study

• Keywords: Artificial Intelligence
• Additional Relevant MeSH Terms: Urogenital Diseases; Male Urogenital Diseases; Pathological Conditions, Anatomical; Urologic Diseases; Female Urogenital Diseases; Female Urogenital Diseases and Pregnancy Complications; Urination Disorders; Urological Manifestations; Urinary Incontinence; Lower Urinary Tract Symptoms; Prolapse
• Other Study ID Numbers: HHC-2024-0097

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No