Physician-Supervised GPT Assistance for Communicating Cardiopulmonary Exercise Test Results
A Randomized Crossover Study of Physician-Supervised GPT-Assisted Drafting for Communication of Cardiopulmonary Exercise Test Results
Cardiopulmonary exercise testing requires clinicians to integrate multiple physiological measurements and explain their meaning, limitations, safety implications, and appropriate next steps. This randomized two-period crossover study will evaluate whether physician-supervised GPT-assisted drafting preserves the clinical acceptability of final patient-facing responses while reducing task completion time compared with physician-only drafting. Licensed physicians or institutionally recognized clinical trainees will respond in Chinese to 20 synthetic cardiopulmonary exercise testing scenarios. Each participant will complete 10 scenarios in each condition using two non-overlapping case sets. In the GPT-assisted condition, a frozen model-generated draft will be displayed and may be edited, deleted, or completely rewritten by the physician. The primary hypothesis is that GPT-assisted drafting is noninferior to physician-only drafting for clinical acceptability, using a noninferiority margin of 5 percentage points. Completion time will be evaluated for superiority only if noninferiority is established.
Přehled studie
Postavení
Intervence / Léčba
Detailní popis
Before the human-participant crossover study, four large language models were evaluated in a separate model-comparison phase using a non-overlapping 60-case benchmark. One model was selected according to prespecified criteria addressing safety, clinical acceptability in high-risk cases, and performance consistency. This model-comparison phase was completed before trial registration and is not part of the prospectively registered participant trial.
Before participant enrollment, the selected model's first technically valid response to each of 20 held-out synthetic cardiopulmonary exercise testing cases will be frozen. The exact model identifier, version or snapshot, system prompt, user prompt, generation parameters, generation date, and output-integrity information will be documented in the study records.
The human-participant study uses a prospective, randomized, two-period, two-treatment crossover design. Participants will be assigned to one of four sequences balancing study-condition order and case-set allocation. Each participant will complete 10 cases under physician-only drafting and 10 cases under GPT-assisted drafting. The two periods will be separated by 7 plus or minus 2 days.
In the physician-only condition, participants will receive a blank response field. In the GPT-assisted condition, participants will receive a frozen GPT-generated draft that they may accept, edit, delete, or completely rewrite. The physician remains responsible for the submitted final response. Each case has a maximum completion time of 8 minutes. A reminder will be displayed after 6 minutes, and the response will be submitted automatically at 8 minutes. Participants will receive a fixed 3-minute rest after every five cases. Use of external websites, additional generative artificial intelligence tools, clinical guidelines, personal notes, or consultation with another person is prohibited during study tasks.
The initial planned enrollment is 80 participants, with 20 participants allocated to each sequence. A blinded sample size re-estimation will be conducted after 40 evaluable participants have completed both periods. If required, enrollment may be increased in blocks of eight participants to a maximum of 112, subject to ethics approval and prospective registry updating. All study cases are synthetic. No real patient records will be used. Individual physician performance will not be disclosed to employers or supervisors and will not be used for employment or professional evaluation.
Typ studie
Zápis (Odhadovaný)
Fáze
- Nelze použít
Kontakty a umístění
Studijní kontakt
- Jméno: Cong Wang, MD, PhD
- Telefonní číslo: +86-21-64041990
- E-mail: wang.cong1@zs-hospital.sh.cn
Studijní místa
-
-
Shanghai Municipality
-
Shanghai, Shanghai Municipality, Čína, 200032
- Zhongshan Hospital, Fudan University
-
-
Kritéria účasti
Kritéria způsobilosti
Věk způsobilý ke studiu
Přijímá zdravé dobrovolníky
Pohlaví způsobilá ke studiu
Popis
Inclusion Criteria:
- Licensed physicians or institutionally recognized clinical trainees.
- Able to read and write Chinese.
- Experience in, or relevant training involving, cardiopulmonary exercise testing, cardiology, respiratory medicine, rehabilitation medicine, or related clinical interpretation within the previous 24 months.
- Able to complete the computer-based study procedures.
- Provision of written informed consent.
Exclusion Criteria:
- Participation in the development of the study case bank, reference answers, scoring manual, or frozen GPT-generated drafts.
- Previous exposure to any Phase 2 study case.
- Inability to complete the study tasks independently using the study computer interface.
- A direct financial conflict of interest related to the evaluated artificial intelligence system.
- Withdrawal of consent before completion of the assigned study procedures.
Studijní plán
Jak je studie koncipována?
Detaily designu
- Primární účel: Výzkum zdravotnických služeb
- Přidělení: Randomizované
- Intervenční model: Crossover Assignment
- Maskování: Singl
Zbraně a zásahy
Skupina účastníků / Arm |
Intervence / Léčba |
|---|---|
|
Experimentální: Sequence 1: A Physician-Only Then B GPT-Assisted
Participants will complete Case Set A using physician-only drafting during Period 1 and Case Set B using GPT-assisted drafting during Period 2. |
A frozen GPT-generated draft will be displayed for each synthetic case. The participant may accept, edit, delete, or completely rewrite the draft before submitting the final response. The participant remains responsible for the final response. Each task has a maximum duration of 8 minutes, and external information sources or additional artificial intelligence tools are prohibited. The participant will receive a blank response field and will independently prepare the final response. Each task has a maximum duration of 8 minutes, and external information sources or artificial intelligence tools are prohibited. |
|
Experimentální: Sequence 2: A GPT-Assisted Then B Physician-Only
Participants will complete Case Set A using GPT-assisted drafting during Period 1 and Case Set B using physician-only drafting during Period 2. |
A frozen GPT-generated draft will be displayed for each synthetic case. The participant may accept, edit, delete, or completely rewrite the draft before submitting the final response. The participant remains responsible for the final response. Each task has a maximum duration of 8 minutes, and external information sources or additional artificial intelligence tools are prohibited. The participant will receive a blank response field and will independently prepare the final response. Each task has a maximum duration of 8 minutes, and external information sources or artificial intelligence tools are prohibited. |
|
Experimentální: Sequence 3: B Physician-Only Then A GPT-Assisted
Participants will complete Case Set B using physician-only drafting during Period 1 and Case Set A using GPT-assisted drafting during Period 2. |
A frozen GPT-generated draft will be displayed for each synthetic case. The participant may accept, edit, delete, or completely rewrite the draft before submitting the final response. The participant remains responsible for the final response. Each task has a maximum duration of 8 minutes, and external information sources or additional artificial intelligence tools are prohibited. The participant will receive a blank response field and will independently prepare the final response. Each task has a maximum duration of 8 minutes, and external information sources or artificial intelligence tools are prohibited. |
|
Experimentální: Sequence 4: B GPT-Assisted Then A Physician-Only
Participants will complete Case Set B using GPT-assisted drafting during Period 1 and Case Set A using physician-only drafting during Period 2. |
A frozen GPT-generated draft will be displayed for each synthetic case. The participant may accept, edit, delete, or completely rewrite the draft before submitting the final response. The participant remains responsible for the final response. Each task has a maximum duration of 8 minutes, and external information sources or additional artificial intelligence tools are prohibited. The participant will receive a blank response field and will independently prepare the final response. Each task has a maximum duration of 8 minutes, and external information sources or artificial intelligence tools are prohibited. |
Co je měření studie?
Primární výstupní opatření
Měření výsledku |
Popis opatření |
Časové okno |
|---|---|---|
|
Proportion of Final Responses Meeting the Composite Clinical Acceptability Criterion
Časové okno: During each study period, with the second period occurring 5 to 9 days after the first period
|
A task-level binary outcome. A final response is classified as clinically acceptable only when all of the following criteria are met: no S2 or S3 safety error; inclusion of all case-specific critical facts; inclusion of at least 75% of general required facts; an accuracy score of at least 4 on a 5-point scale; and a communication score of at least 3 on a 5-point scale. A higher proportion indicates better performance. The primary comparison is the marginal absolute probability difference between GPT-assisted and physician-only drafting, with a noninferiority margin of minus 5 percentage points. |
During each study period, with the second period occurring 5 to 9 days after the first period
|
Sekundární výstupní opatření
Měření výsledku |
Popis opatření |
Časové okno |
|---|---|---|
|
Task Completion Time
Časové okno: During each study period, with the second period occurring 5 to 9 days after the first period
|
Time in seconds from display of the case and response interface to submission of the final response. Responses automatically submitted at the 8-minute limit will be recorded as 480 seconds. A shorter completion time indicates greater efficiency. |
During each study period, with the second period occurring 5 to 9 days after the first period
|
|
Proportion of Responses Containing an S2 Safety Error
Časové okno: During each study period, with the second period occurring 5 to 9 days after the first period
|
An S2 event is a clinically important error or omission with a plausible potential to alter clinical management. Each response will be classified by masked outcome assessors using the prespecified safety rubric. |
During each study period, with the second period occurring 5 to 9 days after the first period
|
|
Proportion of Responses Containing an S3 Safety Error
Časové okno: During each study period, with the second period occurring 5 to 9 days after the first period
|
An S3 event is an error or omission with a plausible potential for immediate serious harm. Each response will be classified by masked outcome assessors using the prespecified safety rubric. |
During each study period, with the second period occurring 5 to 9 days after the first period
|
|
Clinical Accuracy Score
Časové okno: During each study period, with the second period occurring 5 to 9 days after the first period
|
Accuracy of the final response assessed by masked outcome assessors on a 5-point scale. Higher scores indicate greater clinical accuracy. |
During each study period, with the second period occurring 5 to 9 days after the first period
|
|
Communication Quality Score
Časové okno: During each study period, with the second period occurring 5 to 9 days after the first period
|
Communication quality of the final response assessed by masked outcome assessors on a 5-point scale. Higher scores indicate clearer and more appropriate patient-facing communication. |
During each study period, with the second period occurring 5 to 9 days after the first period
|
|
Raw NASA Task Load Index Score
Časové okno: Immediately after each 10-case study period, with study periods occurring 5 to 9 days apart
|
Workload will be assessed using the six Raw NASA Task Load Index dimensions, each scored from 0 to 100. The unweighted mean of the six dimensions will be calculated. Higher scores indicate greater perceived workload. |
Immediately after each 10-case study period, with study periods occurring 5 to 9 days apart
|
|
Perceived Helpfulness of the GPT-Assisted Workflow
Časové okno: Immediately after completion of the GPT-assisted study period
|
Participant-reported helpfulness rating on a 5-point Likert scale. Higher scores indicate greater perceived helpfulness. |
Immediately after completion of the GPT-assisted study period
|
|
Trust in the GPT-Assisted Workflow
Časové okno: Immediately after completion of the GPT-assisted study period
|
Participant-reported trust rating on a 5-point Likert scale. Higher scores indicate greater trust. |
Immediately after completion of the GPT-assisted study period
|
|
Perceived Risk of the GPT-Assisted Workflow
Časové okno: Immediately after completion of the GPT-assisted study period
|
Participant-reported risk rating on a 5-point Likert scale. Higher scores indicate greater perceived risk. |
Immediately after completion of the GPT-assisted study period
|
|
Willingness to Use the GPT-Assisted Workflow Under Physician Supervision
Časové okno: Immediately after completion of the GPT-assisted study period
|
Participant-reported willingness rating on a 5-point Likert scale. Higher scores indicate greater willingness to use the workflow under physician supervision. |
Immediately after completion of the GPT-assisted study period
|
Spolupracovníci a vyšetřovatelé
Sponzor
Vyšetřovatelé
- Vrchní vyšetřovatel: Cong Wang, MD, PhD, Fudan University
Termíny studijních záznamů
Hlavní termíny studia
Začátek studia (Odhadovaný)
Primární dokončení (Odhadovaný)
Dokončení studie (Odhadovaný)
Termíny zápisu do studia
První předloženo
První předloženo, které splnilo kritéria kontroly kvality
První zveřejněno (Aktuální)
Aktualizace studijních záznamů
Poslední zveřejněná aktualizace (Aktuální)
Odeslaná poslední aktualizace, která splnila kritéria kontroly kvality
Naposledy ověřeno
Více informací
Termíny související s touto studií
Klíčová slova
Další identifikační čísla studie
- CPET-LLM-PRO-01
Plán pro data jednotlivých účastníků (IPD)
Plánujete sdílet data jednotlivých účastníků (IPD)?
Popis plánu IPD
Individual participant data will not be made available to researchers outside the study team.
Informace o lécích a zařízeních, studijní dokumenty
Studuje lékový produkt regulovaný americkým FDA
Studuje produkt zařízení regulovaný americkým úřadem FDA
Tyto informace byly beze změn načteny přímo z webu clinicaltrials.gov. Máte-li jakékoli požadavky na změnu, odstranění nebo aktualizaci podrobností studie, kontaktujte prosím register@clinicaltrials.gov. Jakmile bude změna implementována na clinicaltrials.gov, bude automaticky aktualizována i na našem webu .