Making Data Count Intervention Evaluation

A Randomized Controlled Evaluation a Training Intervention to Increase the Use of Statistical Process Control Charts for National Health Service Hospitals in England

Hospitals collect copious amounts of data to share with their board for quality assurance and improvement purposes. The way these data are presented can influence board members' decisions. For example, time-series charts highlight the highest and lowest data but do not clarify whether those data lie outside expected or 'common cause' variation. Statistical process control charts make this clarification and, in so doing, guide quality assurance and improvements in a more targeted fashion.

A previous study showed that data suitable for presentation as a control chart are seldom presented in that format. A training intervention called 'Making Data Count' was created to improve the uptake of statistical process control charts by hospitals in England. The current study will use a randomized design to evaluate whether the intervention increases control charts in hospitals that were not early adopters of the training intervention.

This study is a parallel cluster randomized trial (with baseline-line measurements) across 20 National Health Service (NHS) hospitals in England. The hospitals will be randomly split into two groups. One group will be scheduled to experience the training intervention, and the other group will be placed on a waiting list to experience the training later. The primary analysis will compare the difference in the use of control charts between waitlist and intervention hospitals (adjusting for pre-intervention use) reported with 95% confidence intervals. A qualitative thematic analysis of feedback forms will be conducted.

The present research will evaluate the impact of the training intervention on the use of control charts. The results will apply to institutions that are not early adopters of this training intervention.

Study Overview

• Status: Completed
• Conditions: Statistical Process Control Charts
• Intervention / Treatment: Behavioral: Making Data Count Training intervention

• Study Type: Interventional
• Enrollment (Actual): 15
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United Kingdom
  • Coventry, United Kingdom
    • University of Warwick

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

• NHS Hospital
• Placed on a red-list for predominantly using R-A-G or two-point data presentations in their board papers.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
No Intervention: Waitlist control group; Hospitals assigned to this group will be placed in on a waitlist to receive the training intervention at a later date
Experimental: "Making data count" intervention group; The Making Data Count training intervention was designed by NHS-Improvement to improve knowledge about SPC charts and to increase their uptake. Training sessions are tailored for two sets of attendees: board members and data analysts. Board member and analyst training sessions are delivered as close as possible in time, typically within the same month. Training sessions for board members are usually delivered over about one-and-a-half hours and focus more heavily on the benefits of control charts compared to other charts. Training sessions for analysts are usually delivered over three hours and focus more heavily on the structure and interpretation of the individual and moving range charts (X-mR charts).	Behavioral: Making Data Count Training intervention; The Making Data Count training intervention was designed by NHS-Improvement to improve knowledge about SPC charts and to increase their uptake. Training sessions are tailored for two sets of attendees: board members and data analysts. Board member and analyst training sessions are delivered as close as possible in time, typically within the same month. Training sessions for board members are usually delivered over about one-and-a-half hours and focus more heavily on the benefits of control charts compared to other charts. Training sessions for analysts are usually delivered over three hours and focus more heavily on the structure and interpretation of the individual and moving range charts (X-mR charts).

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Change in the proportion of SPC charts out of all quality and safety charts from baseline to five months post intervention	Change in the proportion of SPC charts (the numerator) out of all charts indicated in each board paper (the denominator)	One month pre-intervention and then five months post-intervention.

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Chart types	counts of whether the charts are time series charts, between subjects charts, time and between charts, or other (e.g., pie)	One month pre-intervention and then five months post-intervention.
SPC elements	counts of whether the charts include recalculated control lines, highlight special cause variation, use R-A-G colouring, and whether the accesses are labeled.	One month pre-intervention and then five months post-intervention.
SPC supporting text	counts of whether the charts include recalculated control lines, highlight special counts of whether SPC charts are explained in supporting text, regarding where control lines are set, reasons for variations, and suggestions for improvements.	One month pre-intervention and then five months post-intervention.

Other Outcome Measures

Outcome Measure	Measure Description	Time Frame
Feedback forms	optional forms participants fill out after training to improve future sessions. Intervention group only.	immediately post intervention

Collaborators and Investigators

• Sponsor: University of Warwick
• Collaborators: University of Birmingham

Study record dates

Study Major Dates

• Study Start (Actual): October 1, 2021
• Primary Completion (Actual): January 31, 2023
• Study Completion (Actual): June 30, 2023

Study Registration Dates

• First Submitted: July 14, 2021
• First Submitted That Met QC Criteria: July 16, 2021
• First Posted (Actual): July 27, 2021

Study Record Updates

• Last Update Posted (Estimated): December 11, 2023
• Last Update Submitted That Met QC Criteria: December 7, 2023
• Last Verified: December 1, 2023

More Information

Terms related to this study

• Other Study ID Numbers: BSREC 100/20-21

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No