AI-Assisted Anaerobic Threshold Interpretation in Cardiopulmonary Exercise Testing (PACE-Reader)
A Randomized Two-Period Crossover Reader Study of Artificial Intelligence-Assisted Anaerobic Threshold Interpretation in Cardiopulmonary Exercise Testing
This prospective randomized two-period crossover reader study will evaluate whether assistance from a frozen artificial intelligence model improves physicians' interpretation of the anaerobic threshold (AT) in cardiopulmonary exercise testing (CPET). Sixteen physicians, including 8 senior and 8 junior readers, will each interpret 150 de-identified CPET examinations once without AI assistance and once with AI assistance. The primary outcome is the within-physician difference in absolute AT localization error between the assisted and unassisted conditions. Secondary outcomes include signed timing error, agreement on whether AT is present, estimates within 15 and 30 seconds of the reference, inter-reader reliability, interpretation time, and whether the effect of AI assistance differs by physician seniority.
Tutkimuksen yleiskatsaus
Tila
Yksityiskohtainen kuvaus
The 150 de-identified CPET examinations will be divided into two non-overlapping case libraries, X and Y, with 75 examinations in each library. The libraries will be balanced, as far as feasible, by clinical severity and acquisition characteristics. None of the reader-study examinations will be used for model training, hyperparameter selection, or checkpoint selection.
Within each seniority stratum, physicians will be randomized in a 1:1 ratio to Sequence A or Sequence B. In period 1, Sequence A will interpret library X without AI assistance and library Y with AI assistance; Sequence B will receive the opposite conditions. Following a 14 ± 3 day washout, the assistance conditions will be reversed in period 2. Case order will be randomized within each period, and new blinded task identifiers will be assigned in period 2. Thus, every physician will interpret every examination once without AI assistance and once with AI assistance, yielding 4,800 interpretations.
In the assisted condition, the interface will display the frozen model's predicted AT time. In the unassisted condition, no model output will be shown. Readers will be masked to case source, routine-report and reference-standard labels, other readers' annotations, and their own annotations from the previous period. For each examination, the reader will classify AT as confirmed, not reached, or unable to confirm. When AT is confirmed, the reader will record the AT time and the interpretive method used. Interpretation time will also be recorded.
The reference standard will be prespecified from unassisted senior-physician annotations. AT will be considered present when at least 6 of 8 senior physicians confirm it, and the reference AT time will be the median of the confirming senior estimates. AT will be considered not reached when at least 6 of 8 senior physicians make that classification. Discordant cases will undergo adjudication by an additional senior physician who is masked to model output and reader identity. For analyses of senior readers, a leave-one-reader-out reference will be used.
The primary analysis will compare absolute AT localization error between assisted and unassisted interpretations using a crossed mixed-effects model. Fixed effects will include assistance condition, physician seniority, the assistance-by-seniority interaction, period, randomized sequence, and case library. Random intercepts will be included for physician and examination. Estimates will be reported with 95% confidence intervals. No simulated or pilot-result data will be included in the prospective study analysis.
Opintotyyppi
Ilmoittautuminen (Arvioitu)
Vaihe
- Ei sovellettavissa
Yhteystiedot ja paikat
Opiskeluyhteys
- Nimi: Cong Wang, MD, PhD
- Puhelinnumero: +86-21-64041990
- Sähköposti: wang.cong1@zs-hospital.sh.cn
Opiskelupaikat
-
-
Shanghai Municipality
-
Shanghai, Shanghai Municipality, Kiina, 200032
- Zhongshan Hospital, Fudan University
-
Ottaa yhteyttä:
- Cong Wang, MD, PhD
- Puhelinnumero: +86-21-64041990
- Sähköposti: wang.cong1@zs-hospital.sh.cn
-
-
Osallistumiskriteerit
Kelpoisuusvaatimukset
Opintokelpoiset iät
Hyväksyy terveitä vapaaehtoisia
Sukupuolet, jotka voivat opiskella
Kuvaus
Inclusion Criteria:
- Age 18 years or older.
- Licensed physician or physician enrolled in a recognized postgraduate clinical training program at the study institution.
- Eligible for one of the two prespecified seniority strata: senior physicians are attending physicians or above with at least 5 years of independent CPET interpretation experience; junior physicians are residents or specialty trainees with less than 5 years of independent CPET interpretation experience.
- Able to interpret CPET examinations and use the study interface after standardized orientation.
- Willing and able to complete both study periods and the prespecified washout interval.
- Provides written informed consent.
Exclusion Criteria:
- Direct involvement in development, training, hyperparameter selection, or checkpoint selection of the PACE-Former model.
- Prior access to the study case libraries, reference-standard annotations, or model outputs for the study examinations.
- Unable to complete the required interpretation tasks or the 14 ± 3 day washout interval.
- Withdrawal of consent before completion of study procedures.
Opintosuunnitelma
Miten tutkimus on suunniteltu?
Suunnittelun yksityiskohdat
- Ensisijainen käyttötarkoitus: Diagnostiikka
- Jako: Satunnaistettu
- Inventiomalli: Crossover-tehtävä
- Naamiointi: Ei mitään (avoin tarra)
Aseet ja interventiot
Osallistujaryhmä / Arm |
Interventio / Hoito |
|---|---|
|
Kokeellinen: Sequence A
Period 1: library X interpreted without AI assistance and library Y interpreted with AI assistance. Period 2, after a 14 ± 3 day washout: library X interpreted with AI assistance and library Y interpreted without AI assistance. |
A frozen PACE-Former model provides a predicted anaerobic threshold time within the CPET review interface. Physicians retain responsibility for the final classification and AT time annotation. No other model output or reference label is displayed. The physician reviews the same de-identified CPET data using the study interface without display of any model prediction or AI-generated suggestion and independently classifies AT status and records the AT time when confirmed. |
|
Kokeellinen: Sequence B
Period 1: library X interpreted with AI assistance and library Y interpreted without AI assistance. Period 2, after a 14 ± 3 day washout: library X interpreted without AI assistance and library Y interpreted with AI assistance. |
A frozen PACE-Former model provides a predicted anaerobic threshold time within the CPET review interface. Physicians retain responsibility for the final classification and AT time annotation. No other model output or reference label is displayed. The physician reviews the same de-identified CPET data using the study interface without display of any model prediction or AI-generated suggestion and independently classifies AT status and records the AT time when confirmed. |
Mitä tutkimuksessa mitataan?
Ensisijaiset tulostoimenpiteet
Tulosmittaus |
Toimenpiteen kuvaus |
Aikaikkuna |
|---|---|---|
|
Difference in Absolute Anaerobic Threshold Localization Error
Aikaikkuna: Recorded immediately after each case interpretation and analyzed after completion of both study periods, up to 2 months
|
Absolute difference, in seconds, between the physician-selected AT time and the prespecified expert reference AT time. The primary estimand is the within-physician difference between AI-assisted and unassisted interpretations, analyzed across the same examinations using a crossed mixed-effects model. |
Recorded immediately after each case interpretation and analyzed after completion of both study periods, up to 2 months
|
Toissijaiset tulostoimenpiteet
Tulosmittaus |
Toimenpiteen kuvaus |
Aikaikkuna |
|---|---|---|
|
Signed Anaerobic Threshold Localization Error
Aikaikkuna: Recorded immediately after each case interpretation and analyzed after completion of both study periods, up to 2 months
|
Signed difference, in seconds, between the physician-selected AT time and the prespecified expert reference AT time. Negative values indicate earlier placement and positive values indicate later placement relative to the reference. |
Recorded immediately after each case interpretation and analyzed after completion of both study periods, up to 2 months
|
|
Proportion of Anaerobic Threshold Estimates Within 15 Seconds of the Reference
Aikaikkuna: Recorded immediately after each case interpretation and analyzed after completion of both study periods, up to 2 months
|
Proportion of interpretable examinations for which the physician-selected AT time is within 15 seconds, in either direction, of the prespecified expert reference AT time. |
Recorded immediately after each case interpretation and analyzed after completion of both study periods, up to 2 months
|
|
Proportion of Anaerobic Threshold Estimates Within 30 Seconds of the Reference
Aikaikkuna: Recorded immediately after each case interpretation and analyzed after completion of both study periods, up to 2 months
|
Proportion of interpretable examinations for which the physician-selected AT time is within 30 seconds, in either direction, of the prespecified expert reference AT time. |
Recorded immediately after each case interpretation and analyzed after completion of both study periods, up to 2 months
|
|
Agreement With the Reference Anaerobic Threshold Status
Aikaikkuna: Recorded immediately after each case interpretation and analyzed after completion of both study periods, up to 2 months
|
Agreement between the physician classification (AT confirmed, AT not reached, or unable to confirm) and the prespecified expert reference classification. Results will be summarized as overall agreement and condition-specific classification performance. |
Recorded immediately after each case interpretation and analyzed after completion of both study periods, up to 2 months
|
|
Inter-Reader Reliability
Aikaikkuna: Recorded immediately after each case interpretation and analyzed after completion of both study periods, up to 2 months
|
Reliability among physicians for AT timing and AT-status classification under AI-assisted and unassisted conditions, summarized using appropriate intraclass correlation and chance-corrected agreement statistics with 95% confidence intervals. |
Recorded immediately after each case interpretation and analyzed after completion of both study periods, up to 2 months
|
|
Interpretation Time per Examination
Aikaikkuna: Recorded immediately after each case interpretation and analyzed after completion of both study periods, up to 2 months
|
Elapsed time, in seconds, from opening the blinded CPET task to submission of the physician's AT classification and, when applicable, the selected AT time. |
Recorded immediately after each case interpretation and analyzed after completion of both study periods, up to 2 months
|
|
Interaction Between AI Assistance and Physician Seniority
Aikaikkuna: Recorded immediately after each case interpretation and analyzed after completion of both study periods, up to 2 months
|
Difference in the effect of AI assistance between senior and junior physicians, estimated from the prespecified assistance-by-seniority interaction term in the crossed mixed-effects model for absolute AT localization error. |
Recorded immediately after each case interpretation and analyzed after completion of both study periods, up to 2 months
|
Yhteistyökumppanit ja tutkijat
Sponsori
Tutkijat
- Päätutkija: Cong Wang, MD, PhD, Fudan University
Opintojen ennätyspäivät
Opi tärkeimmät päivämäärät
Opiskelun aloitus (Arvioitu)
Ensisijainen valmistuminen (Arvioitu)
Opintojen valmistuminen (Arvioitu)
Opintoihin ilmoittautumispäivät
Ensimmäinen lähetetty
Ensimmäinen toimitettu, joka täytti QC-kriteerit
Ensimmäinen Lähetetty (Todellinen)
Tutkimustietojen päivitykset
Viimeisin päivitys julkaistu (Todellinen)
Viimeisin lähetetty päivitys, joka täytti QC-kriteerit
Viimeksi vahvistettu
Lisää tietoa
Tähän tutkimukseen liittyvät termit
Avainsanat
Muut tutkimustunnusnumerot
- PACE-AT-READER-01
Yksittäisten osallistujien tietojen suunnitelma (IPD)
Aiotko jakaa yksittäisten osallistujien tietoja (IPD)?
Lääke- ja laitetiedot, tutkimusasiakirjat
Tutkii yhdysvaltalaista FDA sääntelemää lääkevalmistetta
Tutkii yhdysvaltalaista FDA sääntelemää laitetuotetta
Nämä tiedot haettiin suoraan verkkosivustolta clinicaltrials.gov ilman muutoksia. Jos sinulla on pyyntöjä muuttaa, poistaa tai päivittää tutkimustietojasi, ota yhteyttä register@clinicaltrials.gov. Heti kun muutos on otettu käyttöön osoitteessa clinicaltrials.gov, se päivitetään automaattisesti myös verkkosivustollemme .