Human-AI Collaborative Intelligence for Improving Fetal Flow Management

Human-AI Collaborative Intelligence for Improving Fetal Flow Management: A Randomized Trial

This randomized controlled study evaluates the effectiveness of explainable AI (XAI) in improving clinicians' interpretation of Doppler ultrasound images (UA and MCA) in obstetrics. It involves 92 clinicians, randomized into intervention and control groups. The intervention group receives XAI feedback, aiming to enhance accuracy in ultrasound interpretation and medical decision-making.

Objectives:

1. To develop an interpretable model for commonly used Doppler flows, specifically the Pulsatility Index (PI) of the umbilical artery (UA) and middle cerebral artery (MCA), with the aim to provide quality feedback on Doppler spectrum images and suggest potential gate placements.
2. To test the effects of providing Explainable AI (XAI)-feedback for clinicians compared with no feedback on their accuracy in ultrasound interpretation and management.

Study Overview

• Status: Recruiting
• Conditions: Healthy
• Intervention / Treatment: Behavioral: "XAI feedback on MCA/UA Doppler spectral curves and gate placement suggestions"

Detailed Description

Currently, Doppler ultrasound velocimetry serves as a crucial tool in obstetric practice, particularly for assessing the umbilical artery (UA) and middle cerebral artery (MCA) in uteroplacental-fetal circulation. While Doppler ultrasound is valuable for detecting conditions like fetal anemia and placental insufficiency, its accuracy relies on operator expertise. Artificial intelligence (AI) offers potential enhancements, especially in high-risk pregnancies. However, existing AI applications in fetal ultrasound often lack transparency, leading to user distrust. This study aims to address these limitations by developing an explainable AI model to assist clinicians in interpreting Doppler ultrasound images of UA and MCA for improved management of high-risk pregnancies.

The study's objectives are:

1. To develop an interpretable model for commonly used Doppler flows, specifically the Pulsatility Index (PI) of the umbilical artery (UA) and middle cerebral artery (MCA), with the aim to provide quality feedback on Doppler spectrum images and suggest potential gate placements.
2. To test the effects of providing Explainable AI (XAI)-feedback for clinicians compared with no feedback on their accuracy in ultrasound interpretation and management.

All participants will be instructed to provide gate placement for flow images of the umbilical artery and the MCA, and to evaluate the quality of the resulting flow curves. Each participant will be required to evaluate a total of 40 unique images (10 flow images for UA and MCA, 10 spectral doppler images for UA and MCA, respectively). From the four groups (UA-flow, UA-spectrum, MCA-flow & MCA-spectrum) the investigators will provide matched sets of 40 images that are provided to participants, who are matched for their level of experience within each hospital (PGY 1-2; PGY 3-5; board certified Obstetricians). For flow images, the participants will be instructed to identify the most appropriate gate placement. For the spectral flow curves, participants will be asked to evaluate whether the flow curves were of sufficient quality to inform medical management decisions.

The inclusions criteria for MCA and UA images will be images taken from the third trimester (>= week 28).

Study Design: Randomized controlled trial

Data Source: 1840 unique ultrasound scans including umbilical artery (UA) and middle cerebral artery (MCA) measurements. The 1840 unique images includes: 460 images of UA-flow images, 460 UA-spectrum images, 460 MCA-flow images and 460 MCA-spectrum images.

Participants: 92 clinicians with varying competence levels across four different university hospitals.

Intervention: XAI feedback on MCA/UA Doppler spectral curves and gate placement suggestions.

Control Group: No XAI feedback.

Procedure: Clinicians will be divided into two groups of 46 each, matched for experience levels across hospitals. The control group will place a gate on MCA/UA images and evaluate the Doppler spectrum without AI feedback, while the intervention group will perform the same tasks with access to AI feedback.

Outcome Measures: The participants' responses in the two groups are reviewed by two fetal medicine sonographers who evaluate the participants' answers independently of each other. In a disagreement, the two sonographers reach a consensus after discussion.

• Study Type: Interventional
• Enrollment (Estimated): 92
• Phase: Not Applicable

Contacts and Locations

• Study Contact: Name: Zahra Bashir, MD; Phone Number: 71871407; Email: zahra.bashir@regionh.dk

Study Locations

• Denmark
  • Capital Region Of Denmark
    • Copenhagen, Capital Region Of Denmark, Denmark, 2100
      • Recruiting
      • Rigshospitalet
      • Contact: Zahra Bashir, MD
      • Principal Investigator: Zahra Bashir, MD
  • Region Zealand
    • Slagelse, Region Zealand, Denmark, 4200
      • Recruiting
      • Slagelse Hospital
      • Contact: Zahra Bashir, MD
      • Principal Investigator: Zahra Bashir, MD

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

• The inclusion criterion is the use of ultrasound at least once per week

Exclusion Criteria:

• The exclusion criterion is the absence of experience in ultrasound scanning.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Double

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: "XAI feedback on MCA/UA Doppler spectral curves and gate placement suggestions"; The XAI feedback group will place a gate on MCA/UA images and evaluate the Doppler spectrum with AI feedback. N=46 clinicians (Clinicians will be divided into two groups (XAI feedback & No XAI feedback groups) of 46 each, matched for experience levels across hospitals)	Behavioral: "XAI feedback on MCA/UA Doppler spectral curves and gate placement suggestions"; This study includes 1840 ultrasound images, split into UA and MCA flow and spectrum images, each duplicated for a total of 3680 images to compare explainable AI (XAI) feedback vs. no feedback. The investigators will provide matched sets of 40 images (one for the XAI group and one for the non-XAI group) to participants. Participants are matched based on their level of experience within each hospital (Resident physicians, obstetricians, and gynecologists with obstetric ultrasound experience). All participants are instructed to place gates on the flow images of the umbilical artery and the middle cerebral artery and to assess the quality of the resulting flow curves. Specifically, for flow images, participants must identify the most appropriate gate placement. For spectral flow curves, they are to decide if the curves are of sufficient quality to guide medical management decisions.
No Intervention: "No XAI feedback"; The control group will place a gate on MCA/UA images and evaluate the Doppler spectrum without AI feedback. N=46 clinicians (Clinicians will be divided into two groups (XAI feedback & No XAI feedback groups) of 46 each, matched for experience levels across hospitals)

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Responses will be reviewed independently by two fetal medicine sonographers, and in case of disagreement between the two experts, a consensus will be reached.	The accuracy in each group (AI-feedback and without AI-feedback group) was defined as the percentage difference in the number of correctly managed flow images between the two groups, assessed by two fetal medicine sonographers. Correct management was defined as: Correct gate placement (multiple sites possible) AND Correct identification of flow curves that were of adequate quality to allow medical decision-making.	1 months

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Accuracy of flow image management among competence groups	The secondary outcome is to categorize participants into competence groups (trainees, obstetricians, and gynecologists with obstetric experience) and then examine the percentage difference in the accuracy of flow image management among these competence groups within both the AI-feedback and the non-AI-feedback groups.	1 months

Collaborators and Investigators

• Sponsor: Rigshospitalet, Denmark
• Collaborators: Slagelse Hospital; Technical University of Denmark

Study record dates

Study Major Dates

• Study Start (Estimated): April 15, 2024
• Primary Completion (Estimated): December 1, 2024
• Study Completion (Estimated): December 1, 2025

Study Registration Dates

• First Submitted: May 9, 2023
• First Submitted That Met QC Criteria: April 15, 2024
• First Posted (Actual): April 17, 2024

Study Record Updates

• Last Update Posted (Actual): April 23, 2024
• Last Update Submitted That Met QC Criteria: April 22, 2024
• Last Verified: April 1, 2024

More Information

Terms related to this study

• Other Study ID Numbers: 3-3031-2915/1

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No