Optimizing Video Communication Assessment for Teaching Error Disclosure Skills

The goal of this randomized, single-blinded, educational study is to test the effect of providing crowdsourced ratings and feedback to second-year (PGY2) internal medicine (IM) and family medicine (FM) resident physicians' about their adverse event communication skills. The main question it aims to answer is:

- Is the intervention of providing reports with personal performance feedback and recommendations for effective error disclosure associated with higher ratings of resident error disclosure skills?

Participants will perform simulated error disclosure with a software tool called the Video-based Communication Assessment (VCA). Participants will be randomized to receive feedback reports (intervention) or not (control). Participants receiving the intervention will be asked to review their feedback and all participants will use the VCA again approximately 4 weeks later with different patient cases.

Study Overview

• Status: Completed
• Conditions: Error Disclosure
• Intervention / Treatment: Other: VCA feedback report available

Detailed Description

Participating residency programs assigned all eligible post-graduate year 2 (PGY2s) to attend a 75-minute teaching session at Time 1, consisting of 50 minutes of lecture about communication with patients after medical harm, 20 minutes of VCA practice with two cases (containing 4 and 3 sequenced vignettes, respectively), and 5 minutes of debrief. At Time 2, residents attended a session consisting of 25 minutes of lecture about institutional programs to support clinicians with error disclosure and 20 minutes of VCA practice with two additional cases (3 sequenced vignettes each). The recommended duration between Time 1 and Time 2 was four weeks, although the conference schedule at two residencies required an interval of 5 to 8 weeks for some residents.

Residents who completed the VCA at Time 1 were randomized in 1:1 fashion to either receive feedback before Time 2 (intervention) or after Time 2 (control). Intervention residents received emails when their feedback was available, instructing them to review it in the app before the next teaching session and VCA practice. Feedback was typically provided two weeks after VCA use to allow for completion of rating and data quality checks. Reports presented an interactive feedback display within the VCA app for each vignette.

Residents provided audio responses to each vignette through the VCA software. Audio responses were bundled into rating tasks on MTurk for raters who were US residents over 18 years old and able to speak and read English. Raters answered demographic questions, read a vignette description in lay language, viewed the patient video, and listened to resident responses. They rated each response on six items covering domains of error disclosure. We averaged ratings across items and raters to create an overall rating of each response. We then averaged response ratings across all 7 vignettes at Time 1 to create an overall Time 1 score, and across all 6 vignettes at Time 2 to create a Time 2 score.

Residents completed questionnaires in the VCA application before proceeding to cases. The survey at Time 1 asked about age, gender, race, the number of times the resident had personally participated in disclosure of a harmful error to a patient or family, and the highest level of involvement they've had during disclosure of a harmful medical error. Before Time 2, residents who had received feedback were asked "approximately how many minutes did you spend reviewing your feedback" (response options in 5-min ranges), and "how many of your own responses did you replay", "how many of the exemplar (highly rated peer) responses did you play", (response options of none, 1-2, 3-4, 5 or more). Residents responded to four additional items (Table 2) about the usefulness of each feedback component (scores, personal recordings, exemplar recordings, learning points) using a 5-point scale with labels from "not at all" to "extremely"

To address our primary study question about the effect of the intervention, i.e., access to VCA feedback, we conducted a factorial analysis of covariance (ANCOVA) examining the impact that the intervention and prior disclosure exposure had on Time 2 scores, while adjusting for Time 1 scores. We used logistic regression to investigate whether Time 1 scores could predict the likelihood participants returned for Time 2.

• Study Type: Interventional
• Enrollment (Actual): 154
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • Idaho
    • Boise, Idaho, United States, 83702
      • University of Washington at Boise
  • Massachusetts
    • Worcester, Massachusetts, United States, 01655
      • University of Massachussets
  • Michigan
    • Dearborn, Michigan, United States, 48124
      • Beaumont Health
  • Missouri
    • Saint Louis, Missouri, United States, 63141
      • Washington University in St. Louis
  • New Hampshire
    • Lebanon, New Hampshire, United States, 03756
      • Dartmouth Hitchcock Medica Center
  • Washington
    • Everett, Washington, United States, 98201
      • Washington State University at Everett
    • Seattle, Washington, United States, 98105
      • University of Washington

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

• Enrolled in the second post-graduate year (PGY2) of an Internal Medicine or Family Medicine Residency

Exclusion Criteria:

• Declines research use of audio recordings

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Other
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Single

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: Intervention; Participants will be provided with access to a feedback report in the VCA app that contains 1) crowdsourced ratings of their error disclosure communication, 2) access to recordings of their responses to the completed VCA cases, 3) a recording of a highly-rated peer response and 4) learning points summarizing what laypeople would like the doctor to say in response to the case.	Other: VCA feedback report available; Access to feedback report described in the intervention
No Intervention: Control; Participants will not have access to a feedback report.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Summary crowdsourced rating	A numerical score representing an average of scores of a participant's communication skills provided by panels of crowdsourced laypeople who listened to audio recordings of participants responding to a VCA case.	The second of two time points for VCA use (4 weeks after first use)

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Survey about prior disclosure experience	Survey with items asking about prior experience with error disclosure, provided to all participants.	Before the first VCA use
Survey about VCA feedback use in intervention arm	Survey asking about self-reported use of feedback components, provided to the intervention arm participants only.	Before the second VCA use (4 weeks after first use)

Collaborators and Investigators

• Sponsor: University of Washington
• Collaborators: The National Board of Medical Examiners
• Investigators: Principal Investigator: Andrew A White, MD, SFHM, University of Washington

Study record dates

Study Major Dates

• Study Start (Actual): July 1, 2022
• Primary Completion (Actual): May 15, 2023
• Study Completion (Actual): June 30, 2023

Study Registration Dates

• First Submitted: January 22, 2024
• First Submitted That Met QC Criteria: January 30, 2024
• First Posted (Estimated): January 31, 2024

Study Record Updates

• Last Update Posted (Estimated): January 31, 2024
• Last Update Submitted That Met QC Criteria: January 30, 2024
• Last Verified: January 1, 2024

More Information

Terms related to this study

• Other Study ID Numbers: STUDY00015707

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO
• IPD Plan Description: The primary data is confidential audio recordings, which we do not have participant permission to share with other researchers.

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No