Artificial Intelligence vs Endoscopist Identification in EUS Normal Anatomy

Comparative Evaluation of Artificial Intelligence and Endoscopists´ Accuracy in Endoscopic Ultrasound for Identifying Normal Anatomical Structures: A Multi-institutional, Cross-sectional Study

Endoscopic ultrasound (EUS) visual impression is operator-dependant and can hinder diagnostic accuracy, especially in less experienced endoscopists. The implementation of artificial intelligence can potentially mitigate operator dependency and interpretation variability, helping or improving the overall accuracy.

The investigators therefore aim to compare diagnostic accuracy between artificial intelligence (AI)-based model and the endoscopists when identifying normal anatomical structures in EUS-procedures.

Study Overview

• Status: Completed
• Conditions: Gastrointestinal Diseases
• Intervention / Treatment: Diagnostic test: Detection of structures

Detailed Description

EUS is an operator dependent procedure where accuracy depends on experience and skills. Nowadays, EUS-training can be achieved by a formal fellowship training in a center for 6-24 months or an informal training through didactic sessions with a short hands-on experience. However, parameters for a correct and complete learning experience measurement are yet to be defined. The implementation of artificial intelligence on EUS can potentially mitigate the operator-dependent variable and improve diagnostic accuracy.

Therefore, detection of normal anatomical structures on a separate basis using an AI-based model, expert and non-expert endoscopists to determine where the AI would be most helpful.

The investigators aim to compare the diagnostic accuracy of the AI-based model with the endoscopists identification of normal anatomical structures in EUS procedures.

• Study Type: Observational
• Enrollment (Actual): 30

Contacts and Locations

Study Locations

• Ecuador
  • Guayas
    • Guayaquil, Guayas, Ecuador, 090505
      • IECED

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: No

• Sampling Method: Non-Probability Sample

Study Population

Expert and non-expert gastrointestinal EUS-endoscopists.

Description

Inclusion Criteria:

• Expert gastrointestinal EUS-endoscopists.
• Non-expert gastrointestinal endoscopists training for EUS.
• Patients with chronic dyspepsia without other findings.
• Patients with previous CT images or upper digestive endoscopy reporting no other findings.
• Patients requiring EUS for surveillance due to family history of pancreatic cancer without findings on MRI.

Exclusion Criteria:

• Internet connection less than 100 MBs per second.
• Patients with abnormal structures or with visible lesions.

Study Plan

How is the study designed?

Number of groups / cohorts

3

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
AI-based model; AIWorks-EUS Convolutional Neural Network version 2 (CNNv2) (mdconsgroup, Guayaquil, Ecuador) applied on pre-recorded videos for the detection of normal anatomical structures.	Diagnostic test: Detection of structures; Pre-recorded videos, cropped according to the different windows (mediastinal, gastric, duodenal) will be analyzed by the AIWorks-EUS model and endoscopists on different times for recognition of the different normal anatomical structures.
Expert endoscopists; Endoscopists with >190 EUS procedures per year, including 75 pancreatobiliary and mucosal cancer staging procedures each, 40 subepithelial cases; and 50 cases of EUS-fine needle aspiration (FNA) (25 of them being pancreatic cases); following the American Society for Gastrointestinal Endoscopy (ASGE) recommendations.	Diagnostic test: Detection of structures; Pre-recorded videos, cropped according to the different windows (mediastinal, gastric, duodenal) will be analyzed by the AIWorks-EUS model and endoscopists on different times for recognition of the different normal anatomical structures.
Non-expert endoscopists; Endoscopists with <190 EUS procedures per year, including 75 pancreatobiliary and mucosal cancer staging procedures each, 40 subepithelial cases; and 50 cases of EUS-FNA (25 of them being pancreatic cases); following the American Society for Gastrointestinal Endoscopy (ASGE) recommendations.	Diagnostic test: Detection of structures; Pre-recorded videos, cropped according to the different windows (mediastinal, gastric, duodenal) will be analyzed by the AIWorks-EUS model and endoscopists on different times for recognition of the different normal anatomical structures.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Diagnostic accuracy	The true positive, true negative, false positive and false negative based on detection of anatomical structures according to the an external expert endoscopist as gold-standard.	5 months

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Interobserver agreement	Comparison of diagnostic accuracies between Artificial intelligence (AI)-based model and both groups (expert and non-expert endoscopists) using Fleiss Kappa.	5 months

Collaborators and Investigators

• Sponsor: Instituto Ecuatoriano de Enfermedades Digestivas
• Collaborators: Beth Israel Deaconess Medical Center; Universitair Ziekenhuis Brussel; The Methodist Hospital Research Institute; Larkin Community Hospital; mdconsgroup, Guayaquil, Ecuador; Hospital Clinico Universitario de Santiago; Carol Davila University of Medicine and Pharmacy; Baylor Saint Luke's Medical Center; Barra Life Medical Center, Brazil; Hospital Civil de Morelia, Michoacan; ELIAS Emergency University Hospital
• Investigators: Principal Investigator: Carlos Robles-Medranda, MD FASGE, Instituto Ecuatoriano de Enfermedades Digestivas (IECED)

Publications and helpful links

General Publications

• Han C, Nie C, Shen X, Xu T, Liu J, Ding Z, Hou X. Exploration of an effective training system for the diagnosis of pancreatobiliary diseases with EUS: A prospective study. Endosc Ultrasound. 2020 Sep-Oct;9(5):308-318. doi: 10.4103/eus.eus_47_20.
• Cho CM. Training in Endoscopy: Endoscopic Ultrasound. Clin Endosc. 2017 Jul;50(4):340-344. doi: 10.5946/ce.2017.067. Epub 2017 Jul 31.
• Finocchiaro M, Cortegoso Valdivia P, Hernansanz A, Marino N, Amram D, Casals A, Menciassi A, Marlicz W, Ciuti G, Koulaouzidis A. Training Simulators for Gastrointestinal Endoscopy: Current and Future Perspectives. Cancers (Basel). 2021 Mar 20;13(6):1427. doi: 10.3390/cancers13061427.
• Robles-Medranda C, Baquerizo-Burgos J, Puga-Tejada M, Del Valle R, Mendez JC, Egas-Izquierdo M, Arevalo-Mora M, Cunto D, Alcivar-Vasquez J, Pitanga-Lukashok H, Tabacelia D. Development of convolutional neural network models that recognize normal anatomic structures during real-time radial-array and linear-array EUS (with videos). Gastrointest Endosc. 2024 Feb;99(2):271-279.e2. doi: 10.1016/j.gie.2023.10.028. Epub 2023 Oct 10.

Study record dates

Study Major Dates

• Study Start (Actual): May 1, 2023
• Primary Completion (Actual): October 1, 2023
• Study Completion (Actual): January 26, 2024

Study Registration Dates

• First Submitted: February 16, 2024
• First Submitted That Met QC Criteria: February 22, 2024
• First Posted (Estimated): February 28, 2024

Study Record Updates

• Last Update Posted (Estimated): February 28, 2024
• Last Update Submitted That Met QC Criteria: February 22, 2024
• Last Verified: February 1, 2024

More Information

Terms related to this study

• Keywords: EUS; Gastrointestinal; AI
• Additional Relevant MeSH Terms: Gastrointestinal Diseases; Digestive System Diseases
• Other Study ID Numbers: IECED-12345

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No