Evaluation of ChatGPT-4's Success Sonoanatomy

The Evaluation of ChatGPT-4's Success in Identifying Anatomical Landmarks in Ultrasound Images of Regional Anesthesia Techniques

Aim and Importance:

Regional anesthesia techniques have advanced significantly with the advent of ultrasound guidance. Peripheral nerve blocks and fascial plane blocks can now be performed safely and effectively under ultrasound visualization. Research has shown that ultrasound use significantly improves block success rates. However, accurate application requires in-depth knowledge of sonoanatomy, as failure to identify critical structures may result in incorrect anesthetic placement or failed blocks. While experienced anesthesiologists can easily identify these anatomical landmarks, those less familiar with sonoanatomy may find it challenging.

This study aims to evaluate the effectiveness of ChatGPT-4 in identifying sonoanatomical structures in ultrasound images. A secondary objective is to assess whether artificial intelligence can evaluate the accuracy of regional anesthesia applications.

Expected Benefits and Risks:

The primary benefit is to explore the potential of AI-based systems in improving the learning and application of sonoanatomy, which may help anesthesiologists perform more accurate and successful blocks. We believe that the findings could contribute to regional anesthesia training. The study poses no risks to participants.

Study Type, Scope, and Design:

This prospective, observational study will be conducted at Health Sciences University Istanbul Kanuni Sultan Süleyman Education and Research Hospital. Ultrasound images from patients aged 18 and older undergoing regional anesthesia under ultrasound guidance will be photographed, without collecting personal data. Detailed images of the ultrasound-guided block steps will be captured. The position and orientation of the ultrasound probe will be documented for the AI model.

A customized GPT-4 model will be developed to evaluate the sonoanatomical structures in the provided ultrasound images based on the probe's position and orientation. Additionally, the AI model will predict which block is being performed and assess the success of the block by analyzing the images. An experienced anesthesiologist will evaluate the accuracy of the AI's predictions.

Study Overview

• Status: Completed
• Conditions: Regional Anesthesia
• Intervention / Treatment: Other: Sonoanatomical Structure Identification; Other: Block Type Prediction; Other: Block Success Assessment

Detailed Description

Background and Rationale:

With the increasing use of ultrasound in regional anesthesia, techniques such as peripheral nerve blocks and fascial plane blocks have become more reliable and safer. Ultrasound guidance has significantly improved the success rate of regional anesthesia, reducing complications by enabling accurate visualization of relevant anatomical structures. However, successful ultrasound-guided blocks require thorough knowledge of sonoanatomy. Without this expertise, there is a risk of improper anesthetic placement, potentially leading to block failure or unintended complications.

Experienced anesthesiologists proficient in sonoanatomy can easily interpret ultrasound images, but those with limited experience often face difficulties. This highlights the need for educational tools that can aid in teaching and improving the identification of anatomical landmarks. The development of AI-based systems, such as ChatGPT-4, for this purpose could revolutionize the training of regional anesthesia techniques by providing real-time feedback on ultrasound images.

Primary Aim:

The primary objective of this study is to evaluate the accuracy and effectiveness of ChatGPT-4 in identifying sonoanatomical landmarks from ultrasound images during regional anesthesia procedures.

Secondary Aim:

A secondary goal is to assess whether the AI model can evaluate the accuracy of block applications by analyzing the ultrasound images and determining the success of the block based on sonoanatomical features and block placement.

Expected Benefits:

The study aims to explore whether AI-based systems can be integrated into educational settings to aid anesthesiologists in mastering sonoanatomy for regional anesthesia. By facilitating the accurate identification of anatomical structures, the AI could potentially enhance the learning curve and improve block success rates. The findings of this study may lead to the development of advanced tools for training and performing ultrasound-guided regional anesthesia, benefiting both novice and experienced anesthesiologists.

Potential Risks:

There are no anticipated risks for participants in this study, as no personal data will be collected, and the study involves only the analysis of ultrasound images.

Study Design:

This is a prospective, observational study that will be conducted at Health Sciences University Istanbul Kanuni Sultan Süleyman Education and Research Hospital. The study will include patients aged 18 years and older who are undergoing surgery and receiving regional anesthesia under ultrasound guidance for analgesia or anesthesia purposes. Consent will be obtained from all patients before participating in the study.

Data Collection:

Only ultrasound images from the procedures will be captured, without collecting any personal or identifiable patient data. Each regional anesthesia block will be documented step-by-step through ultrasound images. These images will include key steps such as probe position, orientation, and the anatomical structures visualized during the procedure. The positioning and orientation of the ultrasound probe during the block will also be recorded.

AI Model Configuration:

A customized GPT-4 model will be developed and trained to analyze the ultrasound images. Based on the probe's position, region of placement, and the anatomical plane, the AI will attempt to identify the sonoanatomical structures present in the ultrasound images. The model will also make predictions regarding the type of regional block being performed.

In addition to identifying anatomical landmarks, the AI model will assess the success of the block by analyzing the final images from each procedure. It will provide a prediction of whether the block was successfully applied based on the anatomical structures and positioning of the needle and anesthetic.

Evaluation of AI Predictions:

The accuracy of the AI's predictions regarding anatomical landmarks and block success will be evaluated by an experienced anesthesiologist with expertise in regional anesthesia. This expert will compare the AI's predictions with their own interpretations of the ultrasound images to assess the AI's performance.

Study Outcome:

The primary outcome will be the accuracy of ChatGPT-4 in identifying sonoanatomical structures in ultrasound images. The secondary outcome will be the accuracy of the AI model in evaluating the success of block applications. These results will be compared to the evaluations of the experienced anesthesiologist to determine the AI model's efficacy.

Conclusion:

This study seeks to explore the potential of artificial intelligence, specifically ChatGPT-4, in aiding the identification of anatomical landmarks during ultrasound-guided regional anesthesia. By evaluating the AI's accuracy, the study aims to contribute to the development of innovative training tools that could enhance the education and practice of regional anesthesia techniques.

• Study Type: Observational
• Enrollment (Actual): 147

Contacts and Locations

Study Locations

• Turkey
  • Istanbul, Turkey, 34303
    • Health Science University İstanbul Kanuni Sultan Süleyman Education and Training Hospital

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: No

• Sampling Method: Probability Sample

Study Population

The study will include adult patients aged 18 years or older who are undergoing surgery at Health Sciences University Istanbul Kanuni Sultan Süleyman Education and Research Hospital. These patients will receive regional anesthesia under ultrasound guidance, and those who sign an informed consent form will be included. The study will exclude patients under 18 years old, those without a surgical history, patients who did not receive regional anesthesia under ultrasound guidance, and those who do not sign the informed consent form. This population provides a representative sample of adult surgical patients undergoing regional anesthesia, making it suitable for evaluating the accuracy of AI in identifying sonoanatomical landmarks in ultrasound images.

Description

Inclusion Criteria:

• Patients aged 18 years or older.
• Patients undergoing surgery.
• Patients receiving any regional anesthesia technique under ultrasound guidance.
• Patients who have signed an informed consent form.

Exclusion Criteria:

• Patients under the age of 18.
• Patients without a history of surgery.
• Patients who have not received any regional anesthesia technique under ultrasound guidance.
• Patients who have not signed the required informed consent documents will not be included in the study.

Study Plan

How is the study designed?

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
accuracy of ChatGPT-4	Accuracy will be defined as the AI model's ability to correctly identify key anatomical landmarks (e.g., nerves, muscles, blood vessels, and fascial planes) that are crucial for successful block performance, compared to the gold standard interpretations provided by an experienced anesthesiologist.	immediately after procedure

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
ChatGPT-4's accuracy in predicting the specific block being performed	ChatGPT-4 will be tasked with identifying the type of regional anesthesia block (e.g., supraclavicular block, femoral nerve block) being performed, based on the sonoanatomical structures and probe position shown in the ultrasound images.	immediately after procedure
ChatGPT-4's accuracy in assessing the success of the block	ChatGPT-4 will analyze the ultrasound images post-block application and assess whether the block was successful, based on factors such as needle placement, local anesthetic spread, and proximity to target structures.	immediately after procedure

Collaborators and Investigators

• Sponsor: Kanuni Sultan Suleyman Training and Research Hospital
• Investigators: Principal Investigator: Engin ihsan Turan, Specialist, Health Science University İstanbul Kanuni Sultan Süleyman Education and Training Hospital

Study record dates

Study Major Dates

• Study Start (Actual): October 17, 2024
• Primary Completion (Actual): February 15, 2025
• Study Completion (Actual): February 16, 2025

Study Registration Dates

• First Submitted: October 12, 2024
• First Submitted That Met QC Criteria: October 12, 2024
• First Posted (Actual): October 15, 2024

Study Record Updates

• Last Update Posted (Actual): March 25, 2025
• Last Update Submitted That Met QC Criteria: February 18, 2025
• Last Verified: February 1, 2025

More Information

Terms related to this study

• Keywords: artificial intelligence; sonoanatomy
• Other Study ID Numbers: SONOANATOMY-AI

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: UNDECIDED

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No