Ambient Scribe in General Practice: a Multi-perspective Before-after Longitudinal Mixed-methods Study (AI Scribe)

General practitioners (GPs) in the Netherlands are under unsustainable pressure. Recent surveys show that 68% of general practitioners find the workload too high and 18% find their work extremely or very stressful. The pressure on GPs significantly harms patient care, as reduced physician well-being can negatively impact patient experiences, treatment adherence, patient-provider communication, healthcare costs, care quality, and patient safety.

A key contributor to the stress is the increasing time commitment associated with clinical documentation. The documentation process has evolved into a time-intensive task, which is a significant obstacle to efficient patient care. Large language models (LLMs) are promising artificial intelligence (AI) solutions to reduce the documentation in general practice. In this project, the investigators aim to study an AI-based transcription and reporting tool in general practice.

Study Overview

• Status: Completed
• Conditions: Workload
• Intervention / Treatment: Other: LLM-based transcription and reporting tool

Detailed Description

Introduction Over the years the introduction of the electronic health record (EHR) and escalating demands for documentation have led to a mounting burden on GPs. Clinical documentation is a major barrier to efficient patient care as last year more than half the GPs spend more than 20% of their time on administrative duties. This burden leads to reduced job satisfaction and well-being, increased rates of burnout, and employee attrition. The documentation burden also negatively influences patient-provider communication. It leads to GPs making less eye contact, having a more closed body posture, and conveying less information to their patients. The documentation in the EHR however also has positive effects, such as reduced cognitive load and improvements in patient safety and care. Reducing the provider-computer interaction during the consultation may improve patient-provider interaction and provider well-being while retaining the benefits of EHRs.

In fact, studies showed that employing medical assistants for documentation during consultations leads to an increase in face-to-face time and improves patient satisfaction. Moreover, speech-to-text technologies for dictation after the consultation alleviated the documentation burden by increasing documentation speed, patient experience, and provider satisfaction. These interventions may lower burnout and attrition rates, strengthening the well-being of the providers as well as that of the healthcare sector.

The rapid advancement of large language models (LLMs) has opened new avenues to reduce documentation burden. LLMs, such as ChatGPT, are artificial intelligence (AI) models that can interpret and generate text. These models can transcribe and summarize a consultation with the GP in real-time removing the need for medical assistants or dictation after the consultation. However, if the LLM makes mistakes, this may lead to increased administrative workload. The investigators aim to assess the effect of a AI-based transcription and reporting tool in general practice.

Objectives Our primary objective is to assess the effect of a transcription and reporting tool on time spent on clinical documentation in general practice.

Secondary objectives are to assess the effect of a transcription and reporting tool in general practice on

• Total consultation time
• GP experience
• Patient experience
• Documentation length and quality

To assess the usage rates of the tool To assess the acceptability of use of the tool

Design This is a longitudinal before-after study. For each GP the investigators will observe two days of consultations without the tool. This will be done two weeks before implementation of the tool (baseline period). The investigators will also observe two days of consultations with the tool. This occurs two weeks after implementation of the tool (intervention period). Consultations from the intervention period will be compared with the baseline period to assess effectiveness of tool on the various outcomes.

Sample size A sample size simulation showed the investigators would need 30 observations to have enough power for the primary outcome. To improve the generalizability of our research and increase the sample for the qualitative outcomes, the investigators aim to observe in total four days of consultations for 12 GPs from at least five different practices. Practices will be sampled purposively for differences in patient population and practice organization.

Outcomes The investigators will measure time outcomes through continuous observation. An observer will monitor the time spent on various tasks during a consultation, including taking the medical history, conducting the physical examination, explaining the diagnosis or treatment plan, consulting a colleague, clinical documentation, and administrative duties like prescribing or referring.

Patient experience of the consultation will be measured with a validated patient experience questionnaire (PEQ) for general practice. Patient attitude on the use of the tool during the consultation will be expanded upon with a semi-structured interview based on prior research. The experiences of the GP with and impact of the tool on clinical duties will be measured with semi-structured interviews based on prior research. The acceptability and use of the tool by GPs will be assessed according to the unified theory of acceptance and use of technology (UTAUT).

Usage rates of the tool will be measured by assessing the proportion of consultations in which the tool is used. Documentation volume will be measured by determining the length of documentation in words. The documentation's information quantity will be measured by the number of relevant clinical variables in the note.

• Study Type: Observational
• Enrollment (Actual): 800

Contacts and Locations

Study Locations

• Netherlands
  • South Holland
    • Rotterdam, South Holland, Netherlands, 3015GD
      • Erasmus MC

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: No

• Sampling Method: Non-Probability Sample

Study Population

• General practitioners (in training) in the Southwest of the Netherlands
• Patients seeing general practitioners (in training) in the Southwest of the Netherlands

Description

General practitioners:

Inclusion:

• Planning to implement the AI-based transcription and reporting tool
• Give informed consent for observation and/or interview and/or questionnaire

Exclusion:

- Insufficient knowledge of the Dutch language to be interviewed

Patients:

Inclusion:

• Give informed consent for questionnaire and/or interview
• For questionnaire: had a consultation with the GP
• For interview: had a consultation with the GP in which the tool was used

Exclusion:

- Insufficient knowledge of the Dutch language to be interviewed

Study Plan

How is the study designed?

Number of groups / cohorts

12

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
GP 01; General practitioner working with/without the tool	Other: LLM-based transcription and reporting tool; The LLM-based transcription and reporting tool works in three steps: The conversation between patient and GP is transcribed to text with a speech-to-text LLM; The text is summarized according to the Subjective, Objective, Assessment, Plan (SOAP) rules; The SOAP summary is imported in the electronic health record of the GP; Other Names: Juvoly QuickConsult
GP 02; General practitioner working with/without the tool	Other: LLM-based transcription and reporting tool; The LLM-based transcription and reporting tool works in three steps: The conversation between patient and GP is transcribed to text with a speech-to-text LLM; The text is summarized according to the Subjective, Objective, Assessment, Plan (SOAP) rules; The SOAP summary is imported in the electronic health record of the GP; Other Names: Juvoly QuickConsult
GP 03; General practitioner working with/without the tool	Other: LLM-based transcription and reporting tool; The LLM-based transcription and reporting tool works in three steps: The conversation between patient and GP is transcribed to text with a speech-to-text LLM; The text is summarized according to the Subjective, Objective, Assessment, Plan (SOAP) rules; The SOAP summary is imported in the electronic health record of the GP; Other Names: Juvoly QuickConsult
GP 04; General practitioner working with/without the tool	Other: LLM-based transcription and reporting tool; The LLM-based transcription and reporting tool works in three steps: The conversation between patient and GP is transcribed to text with a speech-to-text LLM; The text is summarized according to the Subjective, Objective, Assessment, Plan (SOAP) rules; The SOAP summary is imported in the electronic health record of the GP; Other Names: Juvoly QuickConsult
GP 05; General practitioner working with/without the tool	Other: LLM-based transcription and reporting tool; The LLM-based transcription and reporting tool works in three steps: The conversation between patient and GP is transcribed to text with a speech-to-text LLM; The text is summarized according to the Subjective, Objective, Assessment, Plan (SOAP) rules; The SOAP summary is imported in the electronic health record of the GP; Other Names: Juvoly QuickConsult
GP 06; General practitioner working with/without the tool	Other: LLM-based transcription and reporting tool; The LLM-based transcription and reporting tool works in three steps: The conversation between patient and GP is transcribed to text with a speech-to-text LLM; The text is summarized according to the Subjective, Objective, Assessment, Plan (SOAP) rules; The SOAP summary is imported in the electronic health record of the GP; Other Names: Juvoly QuickConsult
GP 07; General practitioner working with/without the tool	Other: LLM-based transcription and reporting tool; The LLM-based transcription and reporting tool works in three steps: The conversation between patient and GP is transcribed to text with a speech-to-text LLM; The text is summarized according to the Subjective, Objective, Assessment, Plan (SOAP) rules; The SOAP summary is imported in the electronic health record of the GP; Other Names: Juvoly QuickConsult
GP 08; General practitioner working with/without the tool	Other: LLM-based transcription and reporting tool; The LLM-based transcription and reporting tool works in three steps: The conversation between patient and GP is transcribed to text with a speech-to-text LLM; The text is summarized according to the Subjective, Objective, Assessment, Plan (SOAP) rules; The SOAP summary is imported in the electronic health record of the GP; Other Names: Juvoly QuickConsult
GP 09; General practitioner working with/without the tool	Other: LLM-based transcription and reporting tool; The LLM-based transcription and reporting tool works in three steps: The conversation between patient and GP is transcribed to text with a speech-to-text LLM; The text is summarized according to the Subjective, Objective, Assessment, Plan (SOAP) rules; The SOAP summary is imported in the electronic health record of the GP; Other Names: Juvoly QuickConsult
GP 10; General practitioner working with/without the tool	Other: LLM-based transcription and reporting tool; The LLM-based transcription and reporting tool works in three steps: The conversation between patient and GP is transcribed to text with a speech-to-text LLM; The text is summarized according to the Subjective, Objective, Assessment, Plan (SOAP) rules; The SOAP summary is imported in the electronic health record of the GP; Other Names: Juvoly QuickConsult
GP 11; General practitioner working with/without the tool	Other: LLM-based transcription and reporting tool; The LLM-based transcription and reporting tool works in three steps: The conversation between patient and GP is transcribed to text with a speech-to-text LLM; The text is summarized according to the Subjective, Objective, Assessment, Plan (SOAP) rules; The SOAP summary is imported in the electronic health record of the GP; Other Names: Juvoly QuickConsult
GP 12; General practitioner working with/without the tool	Other: LLM-based transcription and reporting tool; The LLM-based transcription and reporting tool works in three steps: The conversation between patient and GP is transcribed to text with a speech-to-text LLM; The text is summarized according to the Subjective, Objective, Assessment, Plan (SOAP) rules; The SOAP summary is imported in the electronic health record of the GP; Other Names: Juvoly QuickConsult

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Time spent on documentation	This outcome refers to the time spent on documentation for a clinical consultation. The investigators will measure time outcomes through continuous observation. An external observer will monitor the time spent on various tasks during a consultation, including taking the medical history, conducting the physical examination, explaining the diagnosis or treatment plan, consulting a colleague, clinical documentation, and administrative duties like prescribing or referring.	Measured during the consultation (baseline and intervention)

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Total consultation time	This outcome refers to the total time spent on the clinical consultation. The investigators will measure time outcomes through continuous observation. An external observer will monitor the time spent on various tasks during a consultation, including taking the medical history, conducting the physical examination, explaining the diagnosis or treatment plan, consulting a colleague, clinical documentation, and administrative duties like prescribing or referring.	Measured during the consultation (baseline and intervention)
GP experience with the tool	The experiences of the GP and impact of the tool on clinical duties will be measured with semi-structured interviews based on prior research with the following topic guide: What do you think about Juvoly QuickConsult?; Does the system help you? How? Workload? Focus? Job satisfaction?; Does the system help the patient? How? More time to explain or ask questions?; Does the system hinder the you? How? Distraction?; Does the system hinder the patient? Distraction? Don't want to talk about certain issues/share certain information? Less room for asking questions?	Measured within one week after the two-day intervention period
Patient experience with the consultation	Patient experience of the consultation and communication with the GP will be measured with a patient experience questionnaire (PEQ) for general practice (doi: 10.1093/fampra/18.4.410). The PEQ contains five scales: Outcomes of the visit (4-20, higher is better); Communication experiences (4-20, higher is better); Communication barriers (4-20, higher is worse); Experience with the auxiliary staff (2-10, higher is better); Emotions (4-28, higher is better)	Measured within 1 week after the consultation
Patient experience with the tool	Patient experience and attitude with the tool will be assessed using a semi-structured interview based on prior research with the following topic guide: How did you experience the consultation?; What do you think about the system the GP used to write down the consultation?; Does the system help the GP? How? More explanation time?; Does the system help you? How? More elaboration time?; Does the system hinder the GP? How? Distraction for the GP?; Does the system hinder you? Distraction? Don't want to talk about certain issues/share certain information? Less room for asking questions?	Measured within 1 week after the consultation
Usage rates	Usage rates of the tool will be measured by assessing the proportion of consultations in which the tool is used.	Measured directly after the consultation
Documentation volume	Documentation volume will be measured by determining the length of documentation in words.	Measured directly after the consultation
Documentation information density	The documentation's information quantity will be measured by the number of relevant clinical variables in the note. They will be categorized according to their type: Signs and related variables such as site, onset, character, or time course; Contextual factors such as self-care, risk behavior or family history; Symptoms; (Differential) diagnosis; Plan	Measured directly after the consultation
GP acceptability and use of the tool	The acceptability and use of the tool by GPs will be assessed according to the unified theory of acceptance and use of technology (UTAUT). The questionnaire has five domains. Performance expectancy: 4-20 points, higher is better Effort expectancy: 4-20 points, higher is better Social influence: 4-20 points, higher is better Facilitating conditions: 4-20 points, higher is better Behavioral intention: 3-15 points, no value judgement	Measured 1 week before the baseline period, 1 week before the start of the intervention period, and 1 week after intervention period

Collaborators and Investigators

• Sponsor: Erasmus Medical Center

Study record dates

Study Major Dates

• Study Start (Actual): December 9, 2024
• Primary Completion (Actual): July 2, 2025
• Study Completion (Actual): July 2, 2025

Study Registration Dates

• First Submitted: October 25, 2024
• First Submitted That Met QC Criteria: November 14, 2024
• First Posted (Actual): November 15, 2024

Study Record Updates

• Last Update Posted (Actual): December 11, 2025
• Last Update Submitted That Met QC Criteria: December 4, 2025
• Last Verified: December 1, 2025

More Information

Terms related to this study

• Keywords: artificial intelligence; general practice; workload; electronic health record; transcription; documentation; large language model
• Other Study ID Numbers: PaNaMa-12075; MEC-2024-0286 (Other Identifier: Medical Ethical Review Committee Erasmus MC)

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: YES
• IPD Plan Description: Participants will be asked whether their data may be used in future research projects. Metadata of the project will be shared on a data repository. External investigators can apply for use of the data through this data repository. Only data of participants that allow for re-use of their data will be eligible for sharing.
• IPD Sharing Time Frame: Unlimited through a data repository.
• IPD Sharing Access Criteria: Participants will be asked whether their data may be used in future research projects. Metadata of the project will be shared on a data repository. External investigators can apply for use of the data through this data repository. Only data of participants that allow for re-use of their data will be eligible for sharing.
• IPD Sharing Supporting Information Type: STUDY_PROTOCOL; SAP; ICF; ANALYTIC_CODE

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No
• product manufactured in and exported from the U.S.: No