- ICH GCP
- US Clinical Trials Registry
- Clinical Trial NCT06792890
A Randomized Controlled Trial of Ambient Artificial Intelligence Scribe Technologies (AIScribe RCT)
A Randomized Controlled Trial of Two Ambient Artificial Intelligence Scribe Technologies to Improve Documentation Efficiency and Reduce Physician Burnout
This is a three-arm pragmatic RCT of 238 outpatient physicians at a large academic health system, randomized 1:1:1 to one of two AI scribe tools or a usual-care control group. The two-month study will observe and compare the effects of each tool prior to system-wide roll out of selected tool (anticipated Spring 2025). We will use covariate-constrained randomization to balance the arms in terms of physician baseline time in notes, survey-measured level of burnout, and clinic days per week.
The primary purpose of the initiative is to improve quality, efficiency, and business operations at University of California, Los Angeles (UCLA) Health, and this initiative is not being done for research purposes. The results of this operational initiative will inform the widespread roll out of AI scribe tools across all providers within the UCLA Health System. Nevertheless, the UCLA study team plans to rigorously examine and publish the impact of this intervention across the health system, which is why the study team pre-registered the initiative.
Study Overview
Status
Conditions
Intervention / Treatment
Detailed Description
This study will assess operational-oriented outcomes across all groups. Notably, all groups will eventually receive all interventions over time in this observational study of a randomized roll out of a QI initiative. Moreover, the primary purpose of this initiative is operational. In other words, based on the results of this initiative, one of these tools will be eventually selected and operationalized widely across the health system.
Enrolled participants are randomized to one of three groups. Randomization was needed to overcome secular trends, seasonal and holiday effects in December, and other factors confounding the relationship between exposure to the AI tools and the outcomes.
The primary aim of this study is to evaluate the impact of two ambient AI scribe technologies on clinician change from baseline time spent on EHR documentation, comparing each scribe to a control group. Secondary objectives include assessing the AI scribes' impact on clinician metrics such as burnout, physician satisfaction, and productivity. Additionally, the study team intends to perform an economic evaluation analysis of the tools to guide business decision making. The study team will also analyze physician reported effects of the AI tools on patient safety, equity, and any unintended consequences of the initiative.
Study Type
Enrollment (Actual)
Phase
- Not Applicable
Contacts and Locations
Study Locations
-
-
California
-
Los Angeles, California, United States, 90024
- UCLA Health System
-
-
Participation Criteria
Eligibility Criteria
Ages Eligible for Study
- Child
- Adult
- Older Adult
Accepts Healthy Volunteers
Description
Inclusion Criteria:
- Ambulatory care physicians within the UCLA Health system who held at least one half-day of clinic per week
Exclusion Criteria:
- Trainee providers (e.g., residents, medical students) and allied healthcare professionals (e.g., RNs, PAs)
- Attendings who work exclusively with trainees
Study Plan
How is the study designed?
Design Details
- Primary Purpose: Health Services Research
- Allocation: Randomized
- Interventional Model: Single Group Assignment
- Masking: Single
Arms and Interventions
Participant Group / Arm |
Intervention / Treatment |
|---|---|
|
Other: Nabla, Vendor of virtual AI scribe technology
Participants in this arm will utilize AI scribe tools from Nabla and will continue their usual clinical documentation processes, supported by the scribe software, which integrates with the EHR and automatically adds the generated text to the note.
The Nabla AI scribe tool is transcriptional and does not provide clinical decision support.
|
AI Scribe technologies capture physician-patient conversations to create a transcript, then summarize the transcript in the form of a clinical notes. These tools are integrated into the EHR and automatically adds the generated text to the provider note. All physicians must inform patients about the recording and obtain their verbal consent, and instances of patients declining to consent are tracked. Nabla leverages its proprietary speech-to-text to transform the conversation into a written context, combined with HIPAA compliant Large Language Models (LLM) like Azure OpenAI's GPT-4. Nabla does not store any audio. |
|
Other: Vendor B of virtual AI scribe technology
Participants in this arm will utilize AI scribe tools from Vendor B and will continue their usual clinical documentation processes, supported by the scribe software, which integrates with the EHR and automatically adds the generated text to the note.
The AI scribe tool is transcriptional and does not provide clinical decision support.
|
AI Scribe technologies capture physician-patient conversations to create a transcript, then summarize the transcript in the form of a clinical notes.
These tools are integrated into the EHR and automatically adds the generated text to the provider note.
All physicians must inform patients about the recording and obtain their verbal consent, and instances of patients declining to consent are tracked.
|
|
No Intervention: No Scribe
Participants in this arm will not have access to AI scribe tools and will continue their usual clinical documentation processes
|
What is the study measuring?
Primary Outcome Measures
Outcome Measure |
Measure Description |
Time Frame |
|---|---|---|
|
Change in the Time in Notes Per Note
Time Frame: Study month 2
|
The primary outcome measure is the change in provider mean time in notes per note in the second month of the trial from the providers baseline mean time in notes per note for the six months prior to enrollment.
This change will be computed on the natural log scale.
No patient level information will be collected for this outcome measure.
|
Study month 2
|
Secondary Outcome Measures
Outcome Measure |
Measure Description |
Time Frame |
|---|---|---|
|
Provider Burnout Score
Time Frame: Study month 2
|
The Mini Z 2.0 Survey is a validated 10-item instrument designed to measure key factors influencing workplace satisfaction and burnout among healthcare professionals.
Each item is scored on a Likert scale (1-5), with higher scores generally indicating more positive outcomes - greater job satisfaction, sufficiency of time for electronic medical record documentation, and lower levels of stress.
For negatively framed items (e.g., stress due to the job or frustration with the electronic medical record), higher scores indicate lower levels of dissatisfaction.
The total score ranges from 10 to 50, with scores ≥40 representing a joyful workplace.
No patient level information will be collected for this outcome measure.
|
Study month 2
|
|
Change in EHR Signal (Activity) Data - Pajama Time
Time Frame: Study month 2
|
We will examine change from a retrospective baseline 6 months prior to enrollment in Signal metrics including pajama time per scheduled day.
Using this data will determine how a providers time is utilized in the EHR.
No patient level information will be collected for this outcome measure.
|
Study month 2
|
|
Provider Task Load Score
Time Frame: Study month 2
|
Provider task load adapted from the NASA Task Load Index (TLX), a validated tool for assessing perceived workload across six sub-scales: mental demand, physical demand, temporal demand, performance, effort, and frustration.
For this study, we adapted the TLX to focus on note-writing workload, including four sub-scales (mental demand, temporal demand, physical demand, and effort) as done previously.
Each sub-scale is rated from 0 (low task load) to 100 (high task load) and summed together for a total score scale of 0 (low task load) to 400 (high task load), lower is better.
No patient level information will be collected for this outcome measure.
|
Study month 2
|
|
Provider Professional Fulfillment
Time Frame: Study month 2
|
The Professional Fulfillment Index (PFI) is a validated 16-item instrument that uses a 5-point Likert scale (0-4) to measure professional fulfillment, work exhaustion, and interpersonal disengagement.
For this study, we utilize the 4-item work exhaustion subscale which is a mean of the 4-items within that subscale, where a low score (0) indicates a lower level of exhaustion and a high (4) score indicates greater level of exhaustion.
No patient level information will be collected for this outcome measure.
|
Study month 2
|
|
Number of Physicians Who Are Considered Detractors, Passive, or Promoters
Time Frame: Study month 2
|
Self-reported satisfaction survey that asks physicians to consider note accuracy, patient safety, equity, and other potential unintended consequences and rate their overall likelihood to recommend use of the tool on a 1-10 scale.
Higher scores (10) indicate greater satisfaction and likelihood to recommend, whereas lower scores (1) indicate dissatisfaction and unlikelihood to recommend.
Providers are grouped as "Promoters" if they respond 9-10, "Passive" if they respond 7-8, and "Detractors" if they respond with a value less than or equal to 6.
This grouping matches commonly accepted "Net Promoter Score" groupings.
No patient level information will be collected for this outcome measure.
|
Study month 2
|
|
Change in Provider RVU
Time Frame: Study month 2
|
The study team will use physician-level billing information via RVU to determine their change in productivity from a retrospective baseline 6 months prior to enrollment.
No patient level information will be collected for this outcome measure.
|
Study month 2
|
|
Change in EHR Signal (Activity) Data - Time Outside Scheduled Hours
Time Frame: Study month 2
|
We will examine change from a retrospective baseline 6 months prior to enrollment in Signal metrics including time outside scheduled hours per scheduled day.
Using this data will determine how a providers time is utilized in the EHR.
No patient level information will be collected for this outcome measure.
|
Study month 2
|
|
Change in EHR Signal (Activity) Data - Time on Unscheduled Days
Time Frame: Study month 2
|
We will examine change from a retrospective baseline 6 months prior to enrollment in Signal metrics including time spent in the system on unscheduled days where .
Using this data will determine how a providers time is utilized in the EHR.
No patient level information will be collected for this outcome measure.
|
Study month 2
|
Collaborators and Investigators
Publications and helpful links
General Publications
- Trockel M, Bohman B, Lesure E, Hamidi MS, Welle D, Roberts L, Shanafelt T. A Brief Instrument to Assess Both Burnout and Professional Fulfillment in Physicians: Reliability and Validity, Including Correlation with Self-Reported Medical Errors, in a Sample of Resident and Practicing Physicians. Acad Psychiatry. 2018 Feb;42(1):11-24. doi: 10.1007/s40596-017-0849-3. Epub 2017 Dec 1.
- Sinsky C, Colligan L, Li L, Prgomet M, Reynolds S, Goeders L, Westbrook J, Tutty M, Blike G. Allocation of Physician Time in Ambulatory Practice: A Time and Motion Study in 4 Specialties. Ann Intern Med. 2016 Dec 6;165(11):753-760. doi: 10.7326/M16-0961. Epub 2016 Sep 6.
- Linzer M, McLoughlin C, Poplau S, Goelz E, Brown R, Sinsky C; AMA-Hennepin Health System (HHS) burnout reduction writing team. The Mini Z Worklife and Burnout Reduction Instrument: Psychometrics and Clinical Implications. J Gen Intern Med. 2022 Aug;37(11):2876-2878. doi: 10.1007/s11606-021-07278-3. Epub 2022 Jan 19. No abstract available.
- Gianfrancesco MA, Tamang S, Yazdany J, Schmajuk G. Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data. JAMA Intern Med. 2018 Nov 1;178(11):1544-1547. doi: 10.1001/jamainternmed.2018.3763.
- Garcia P, Ma SP, Shah S, Smith M, Jeong Y, Devon-Sand A, Tai-Seale M, Takazawa K, Clutter D, Vogt K, Lugtu C, Rojo M, Lin S, Shanafelt T, Pfeffer MA, Sharp C. Artificial Intelligence-Generated Draft Replies to Patient Inbox Messages. JAMA Netw Open. 2024 Mar 4;7(3):e243201. doi: 10.1001/jamanetworkopen.2024.3201.
- Cruz Rivera S, Liu X, Chan AW, Denniston AK, Calvert MJ; SPIRIT-AI and CONSORT-AI Working Group; SPIRIT-AI and CONSORT-AI Steering Group; SPIRIT-AI and CONSORT-AI Consensus Group. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Nat Med. 2020 Sep;26(9):1351-1363. doi: 10.1038/s41591-020-1037-7. Epub 2020 Sep 9.
- McCoy LG, Manrai AK, Rodman A. Large Language Models and the Degradation of the Medical Record. N Engl J Med. 2024 Oct 31;391(17):1561-1564. doi: 10.1056/NEJMp2405999. Epub 2024 Oct 26. No abstract available.
- Hendrix N, Veenstra DL, Cheng M, Anderson NC, Verguet S. Assessing the Economic Value of Clinical Artificial Intelligence: Challenges and Opportunities. Value Health. 2022 Mar;25(3):331-339. doi: 10.1016/j.jval.2021.08.015. Epub 2021 Oct 9.
- Rotenstein L, Melnick ER, Iannaccone C, Zhang J, Mugal A, Lipsitz SR, Healey MJ, Holland C, Snyder R, Sinsky CA, Ting D, Bates DW. Virtual Scribes and Physician Time Spent on Electronic Health Records. JAMA Netw Open. 2024 May 1;7(5):e2413140. doi: 10.1001/jamanetworkopen.2024.13140.
- Cao DY, Silkey JR, Decker MC, Wanat KA. Artificial intelligence-driven digital scribes in clinical documentation: Pilot study assessing the impact on dermatologist workflow and patient encounters. JAAD Int. 2024 Feb 20;15:149-151. doi: 10.1016/j.jdin.2024.02.009. eCollection 2024 Jun. No abstract available.
- Owens LM, Wilda JJ, Grifka R, Westendorp J, Fletcher JJ. Effect of Ambient Voice Technology, Natural Language Processing, and Artificial Intelligence on the Patient-Physician Relationship. Appl Clin Inform. 2024 Aug;15(4):660-667. doi: 10.1055/a-2337-4739. Epub 2024 Jun 4.
- Haberle T, Cleveland C, Snow GL, Barber C, Stookey N, Thornock C, Younger L, Mullahkhel B, Ize-Ludlow D. The impact of nuance DAX ambient listening AI documentation: a cohort study. J Am Med Inform Assoc. 2024 Apr 3;31(4):975-979. doi: 10.1093/jamia/ocae022.
- Liu TL, Hetherington TC, Stephens C, McWilliams A, Dharod A, Carroll T, Cleveland JA. AI-Powered Clinical Documentation and Clinicians' Electronic Health Record Experience: A Nonrandomized Clinical Trial. JAMA Netw Open. 2024 Sep 3;7(9):e2432460. doi: 10.1001/jamanetworkopen.2024.32460.
- Blackley SV, Huynh J, Wang L, Korach Z, Zhou L. Speech recognition for clinical documentation from 1990 to 2018: a systematic review. J Am Med Inform Assoc. 2019 Apr 1;26(4):324-338. doi: 10.1093/jamia/ocy179.
- Heckman J, Mukamal KJ, Christensen A, Reynolds EE. Medical Scribes, Provider and Patient Experience, and Patient Throughput: a Trial in an Academic General Internal Medicine Practice. J Gen Intern Med. 2020 Mar;35(3):770-774. doi: 10.1007/s11606-019-05352-5. Epub 2019 Dec 5.
- Bates DW, Landman AB. Use of Medical Scribes to Reduce Documentation Burden: Are They Where We Need to Go With Clinical Documentation? JAMA Intern Med. 2018 Nov 1;178(11):1472-1473. doi: 10.1001/jamainternmed.2018.3945. No abstract available.
- Mishra P, Kiang JC, Grant RW. Association of Medical Scribes in Primary Care With Physician Workflow and Patient Experience. JAMA Intern Med. 2018 Nov 1;178(11):1467-1472. doi: 10.1001/jamainternmed.2018.3956.
- Steinkamp J, Kantrowitz JJ, Airan-Javia S. Prevalence and Sources of Duplicate Information in the Electronic Medical Record. JAMA Netw Open. 2022 Sep 1;5(9):e2233348. doi: 10.1001/jamanetworkopen.2022.33348.
- Guille C, Sen S. Burnout, Depression, and Diminished Well-Being among Physicians. N Engl J Med. 2024 Oct 24;391(16):1519-1527. doi: 10.1056/NEJMra2302878. No abstract available.
- Lou SS, Lew D, Harford DR, Lu C, Evanoff BA, Duncan JG, Kannampallil T. Temporal Associations Between EHR-Derived Workload, Burnout, and Errors: a Prospective Cohort Study. J Gen Intern Med. 2022 Jul;37(9):2165-2172. doi: 10.1007/s11606-022-07620-3. Epub 2022 Jun 16.
- Moy AJ, Schwartz JM, Chen R, Sadri S, Lucas E, Cato KD, Rossetti SC. Measurement of clinical documentation burden among physicians and nurses using electronic health records: a scoping review. J Am Med Inform Assoc. 2021 Apr 23;28(5):998-1008. doi: 10.1093/jamia/ocaa325.
- Peccoralo LA, Kaplan CA, Pietrzak RH, Charney DS, Ripp JA. The impact of time spent on the electronic health record after work and of clerical work on burnout among clinical faculty. J Am Med Inform Assoc. 2021 Apr 23;28(5):938-947. doi: 10.1093/jamia/ocaa349.
- Sittig DF, Singh H. Recommendations to Ensure Safety of AI in Real-World Clinical Care. JAMA. 2025 Feb 11;333(6):457-458. doi: 10.1001/jama.2024.24598.
- Lukac PJ, Turner W, Vangala S, Chin AT, Khalili J, Shih YT, Sarkisian C, Cheng EM, Mafi JN. Ambient AI Scribes in Clinical Practice: A Randomized Trial. NEJM AI. 2025 Dec;2(12):10.1056/aioa2501000. doi: 10.1056/aioa2501000. Epub 2025 Nov 26.
- Lukac PJ, Turner W, Vangala S, Chin AT, Khalili J, Shih YT, Sarkisian C, Cheng EM, Mafi JN. A Randomized-Clinical Trial of Two Ambient Artificial Intelligence Scribes: Measuring Documentation Efficiency and Physician Burnout. medRxiv [Preprint]. 2025 Jul 11:2025.07.10.25331333. doi: 10.1101/2025.07.10.25331333.
Study record dates
Study Major Dates
Study Start (Actual)
Primary Completion (Actual)
Study Completion (Actual)
Study Registration Dates
First Submitted
First Submitted That Met QC Criteria
First Posted (Actual)
Study Record Updates
Last Update Posted (Actual)
Last Update Submitted That Met QC Criteria
Last Verified
More Information
Terms related to this study
Keywords
Other Study ID Numbers
- NPR 40703
Plan for Individual participant data (IPD)
Plan to Share Individual Participant Data (IPD)?
Drug and device information, study documents
Studies a U.S. FDA-regulated drug product
Studies a U.S. FDA-regulated device product
This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.
Clinical Trials on Physician Workflow
-
Emory UniversityWithdrawnPhysician WorkflowUnited States
-
University of California, Los AngelesRecruitingPhysician Workflow | Resource UtilizationUnited States
-
University of PennsylvaniaRecruitingPhysician Workflow | Transitions of Care | Electronic Medical Record | Large Language Model | Artificial Intelligence (AI)United States
-
University Hospital, GenevaRecruiting
-
University of RochesterWeill Medical College of Cornell University; State University of New York at...CompletedOperations Research WorkflowUnited States
-
University of Missouri-ColumbiaCompleted
-
Philips Clinical & Medical Affairs GlobalPhilips Healthcare (Suzhou) Co., Ltd.Completed
-
University Hospital, Basel, SwitzerlandCompletedLaparoscopy | Cholecystectomy | Costs | WorkflowSwitzerland
-
Boston Children's HospitalHarvard Risk Management FoundationCompletedPatient Safety | Resident Workflow | Resident ExperienceUnited States
-
Mayo ClinicCompletedPhysician Well-being | Physician Job SatisfactionUnited States
Clinical Trials on Use Nabla AI Scribe tool provided
-
University of WashingtonCompletedPhysician Work Environment | Physician Burnout | Artificial Intelligence (AI)United States