Effects of Performance Feedback on Imaging Use in the Emergency Department

DESIGN: An IRB-approved, HIPAA-compliant prospective randomized controlled trial conducted from January 1, 2012-December 31, 2013.

SETTING: The ED of an urban Level-I adult trauma center with existing CDS for CTPE.

PARTICIPANTS: All attending physicians were stratified into quartiles by 2012 CTPE use and randomized to receive feedback reporting or not.

INTERVENTION: Quarterly performance feedback reports consisting of individual and anonymized group data on EBG adherence (using the Wells criteria), CTPE use (CTPEs per 1,000 patients), and yield (percentage of CTPEs positive for PE) beginning January 2013.

Study Overview

• Status: Completed
• Conditions: Pulmonary Embolism
• Intervention / Treatment: Behavioral: Physician Feedback Reporting

Detailed Description

Setting and Subjects This IRB-approved and HIPAA-compliant prospective randomized controlled trial was conducted from January 1, 2012 to December 31, 2013 in the ED of an urban Level-I adult trauma center. Imaging CDS was deployed for all CTPE requests throughout the study period as previously reported. Ordering providers could ignore the evidence presented in CDS and proceed with CTPE requests deviating from evidence-based guidelines without interference. The investigators included all attending emergency physicians and, prior to randomization, stratified them into quartiles by 2012 CTPE use. Physicians were randomized by quartile into two groups using a random number generator: the intervention group received individualized feedback reports on CTPE adherence to EBG, use (defined as number of CTPEs per 1,000 patients), and yield (percentage of CTPEs positive for PE), and the control group did not.

Data Collection Use of CTPEs for each physician was calculated using the number of completed CTPE examinations and the total number of patients seen during the quarter. Yield of CTPEs for acute PE was determined using a previously validated natural language processing tool and reported as a percentage of total CTPEs completed. Adherence to EBG was determined by applying the Wells Criteria and reviewing the serum D-dimer levels (if obtained). The discrete criteria making up the Wells Criteria were prospectively documented in the investigators computerized order entry (CPOE) system at the time of order entry, as previously reported.

In order to determine whether any differences observed in guideline adherence were the result of "gaming" (erroneous data entry to either avoid potentially onerous CDS interactions or to enhancing the physician's apparent performance on feedback reports) the investigators performed manual chart reviews of 100 randomly chosen charts from each of the two groups. These chart reviews were performed by an attending physician to assess concordance between adherence to EBG calculated from the CDS data, and adherence to EBG calculated from data documented in the ED visit clinical note. The sample size was determined using a baseline concordance of 90% as previously reported, and chosen to detect a difference in concordance of 15% between groups with a power of 80% and an alpha error rate of 5%. Demographic data (including gender and both age and years since residency training as measured at the beginning of the study period) was also captured for all the attending physicians in the study.

• Study Type: Interventional
• Enrollment (Actual): 43
• Phase: Not Applicable

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 15 years and older (Child, Adult, Older Adult)
• Accepts Healthy Volunteers: Yes
• Genders Eligible for Study: All

Description

Inclusion Criteria:

• All attending physicians in the emergency department

Exclusion Criteria:

• None

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: Randomized
• Interventional Model: Single Group Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
No Intervention: Control
Experimental: Intervention; The intervention consisted of quarterly performance feedback reports sent via e-mail.	Behavioral: Physician Feedback Reporting; The performance feedback reports displayed both individual physicians' statistics as well as their performance compared to anonymized results for the entire group of emergency physicians.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Adherence to Evidence-Based Guidelines	Percentage of CTPE orders that adhere to evidence-based guidelines for the ordering of CTs for PE	2 years

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Use of Computed Tomography for Pulmonary Embolism	#CTPEs/1,000 ED visits for each physician	2 years
Yield of Computed Tomography for Pulmonary Embolism	Percentage of CTPE orders that are positive for acute PE	2 years

Collaborators and Investigators

• Sponsor: Brigham and Women's Hospital
• Investigators: Principal Investigator: Ramin Khorasani, MD, Brigham and Women's Hospital

Study record dates

Study Major Dates

• Study Start: January 1, 2012
• Primary Completion (Actual): December 1, 2013
• Study Completion (Actual): December 1, 2013

Study Registration Dates

• First Submitted: November 2, 2014
• First Submitted That Met QC Criteria: November 4, 2014
• First Posted (Estimate): November 5, 2014

Study Record Updates

• Last Update Posted (Estimate): November 5, 2014
• Last Update Submitted That Met QC Criteria: November 4, 2014
• Last Verified: November 1, 2014

More Information

Terms related to this study

• Keywords: Quality Improvement; Physician Feedback
• Additional Relevant MeSH Terms: Cardiovascular Diseases; Vascular Diseases; Respiratory Tract Diseases; Lung Diseases; Embolism and Thrombosis; Embolism; Pulmonary Embolism
• Other Study ID Numbers: 2009-P001587