AI as an Aid for Weekly Symptom Intake in Radiotherapy

Evaluation of AI-Enhanced Symptom Summarization in Weekly Radiotherapy Consultations: A Comparative Study

The study investigates the use of artificial intelligence (AI) and large language models (LLMs) to enhance the efficiency and accuracy of weekly treatment consultations (OTVs) in radiotherapy. It hypothesizes that an AI-enabled symptom summary tool will match traditional medical review methods in accuracy while saving time. The study includes patients undergoing pelvic radiotherapy and excludes those with pelvic reirradiation or who have undergone surgery. Patients will receive both standard and AI-assisted weekly consultations, with AI summaries generated using the OpenAI GPT-4 API. Blinded oncologists will compare the accuracy and quality of the AI-generated and doctor-generated summaries, while patients and doctors will rate these summaries. The primary objective is to evaluate the accuracy and time efficiency of AI-assisted symptom summaries compared to traditional methods.

Study Overview

• Status: Recruiting
• Conditions: Pelvic Cancer; Radiotherapy Side Effect; Patient
• Intervention / Treatment: Other: Standard weekly symptom intake; Other: Generative Artificial Intelligence

Detailed Description

This clinical trial is a comparative study designed to evaluate the accuracy and time efficiency of an AI-enabled symptom summary tool in comparison to traditional medical review methods in patients undergoing radiotherapy in the pelvic region.

Hypothesis:

The AI-enabled symptom summary tool is hypothesized to be non-inferior in accuracy to traditional medical review methods and to save time in the process.

Primary Outcome:

Accuracy of Documentation: The quality of the documentation will be evaluated using the Physician Documentation Quality Instrument-9 (PDQI-9), a validated questionnaire that assesses nine key elements of documentation quality: completeness, correctness, consistency, comprehensibility, relevance, organization, conciseness, formatting, and overall impression. Blinded specialist doctors will use the PDQI-9 to evaluate both AI-generated and traditional summaries, assigning scores from 1 to 10.

Secondary Outcomes:

Time Efficiency: The time required to complete the AI-enabled and traditional consultations will be recorded and compared.

Physician Satisfaction: A custom-designed satisfaction questionnaire will be administered to the physicians participating in the study. This questionnaire will include Likert-scale questions to rate various aspects of satisfaction, including ease of use, time efficiency, accuracy perception, and overall satisfaction.

Patient Satisfaction: A custom-designed satisfaction questionnaire will be administered to the patients participating in the study. This questionnaire will include Likert-scale questions to rate various aspects of satisfaction, including clarity and understanding, perceived accuracy, engagement and interaction, and overall satisfaction.

Methodology:

Patient Selection: Patients meeting the inclusion criteria will be selected for participation. Exclusion criteria will be applied to eliminate cases of pelvic reirradiation or prior operations in the pelvic region.

Consultation Process: Patients will undergo a standard weekly consultation with a doctor. In the same week, each patient will also have a separate consultation with a different doctor. During this second consultation, a symptom questionnaire will be completed under medical supervision. The resulting summary from this questionnaire will be generated using the OpenAI GPT-4 API.

• Study Type: Interventional
• Enrollment (Estimated): 200
• Phase: Not Applicable

Contacts and Locations

• Study Contact: Name: Rachele Grazziotin, MD; Phone Number: (21)3207-4550; Email: cep@inca.gov.br

Study Locations

• Brazil
  • Rio De Janeiro, Brazil
    • Recruiting
    • Instituto Nacional de Câncer José Alencar Gomes da Silva - INCA
    • Contact: Rachele Rachele Grazziotin, MD

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: No

Description

Inclusion Criteria:

All patients undergoing radiotherapy in the pelvic region.

Exclusion Criteria:

Cases of pelvic reirradiation or operated cases.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Other
• Allocation: Non-Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Active Comparator: Standard weekly symptom assessment by physicians	Other: Standard weekly symptom intake; Standard weekly symptom intake performed by a physician
Experimental: AI-assisted symptom intake	Other: Generative Artificial Intelligence; Gen AI assisted symptom intake summarization

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
The Physician Documentation Quality Instrument-9 (PDQI-9)	The Physician Documentation Quality Instrument-9 (PDQI-9) will be used to evaluate the quality of the documentation. The PDQI-9 is a validated questionnaire that assesses nine key elements of documentation quality: completeness, correctness, consistency, comprehensibility, relevance, organization, conciseness, formatting, and overall impression.	2 months

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Time tracking	The total time spent with each patient and on documentation will be recorded for both the standard and AI-enabled consultations.	2 months
Accuracy	A team of blinded specialist oncologists will compare the summaries generated by both methods. Accuracy ratings will be assigned based on tabulated data from the completed questionnaires. Patients will be shown both the AI-generated summary and the doctor's summary and will be asked to rate their accuracy on a scale from 1 to 10.	2 months
Physician satisfaction	Physician Satisfaction Assessment: Physician satisfaction with the AI-enabled symptom summary tool compared to traditional documentation methods will be evaluated. Measurement Tool: A custom-designed satisfaction questionnaire will be administered to the physicians participating in the study. This questionnaire will include Likert-scale questions to rate various aspects of satisfaction, including: Ease of Use: Physicians will rate how easy they find the AI-enabled tool to use compared to traditional methods. Time Efficiency: Physicians will assess whether the AI tool saves time during patient consultations and documentation. Overall Satisfaction: An overall satisfaction score will be provided, reflecting the physician's general experience with the AI tool.	2 months
Patient satisfaction	Patient Satisfaction Assessment: Patient satisfaction with the AI-enabled symptom summary tool compared to traditional documentation methods will be evaluated as a secondary outcome of this trial. Measurement Tool: A custom-designed satisfaction questionnaire will be administered to the patients participating in the study. This questionnaire will include Likert-scale questions to rate various aspects of satisfaction, including: Clarity and Understanding: Patients will rate how clear and understandable they find the AI-generated summaries compared to traditional documentation. Engagement and Interaction: Patients will evaluate their engagement and interaction experience during consultations using the AI tool versus traditional methods. Satisfaction: An overall satisfaction score will be provided, reflecting the patient's general experience with the AI tool.	2 months

Collaborators and Investigators

• Sponsor: jaide
• Collaborators: National Cancer Institute, Brazil

Study record dates

Study Major Dates

• Study Start (Actual): July 22, 2024
• Primary Completion (Estimated): November 1, 2024
• Study Completion (Estimated): December 15, 2024

Study Registration Dates

• First Submitted: July 18, 2024
• First Submitted That Met QC Criteria: July 24, 2024
• First Posted (Actual): July 29, 2024

Study Record Updates

• Last Update Posted (Actual): October 10, 2024
• Last Update Submitted That Met QC Criteria: October 8, 2024
• Last Verified: October 1, 2024

More Information

Terms related to this study

• Keywords: Artificial Intelligence; Patient-Reported Outcome
• Additional Relevant MeSH Terms: Neoplasms; Neoplasms by Site; Pelvic Neoplasms
• Other Study ID Numbers: BR-001

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No