Evaluation of the Effectiveness of an Audit and Feedback Intervention With Quality Improvement Toolbox in Intensive Care

Increasing the Effectiveness and Understanding of Audit and Feedback Interventions in Intensive Care: Protocol for a Mixed-methods Study

This study evaluates the addition of a quality improvement toolbox to an online audit and feedback intervention in Dutch intensive care units. The toolbox comprises for each quality indicator (e.g., percentage of patients per shift whose pain is measured) a list of potential bottlenecks in the care process (e.g., staff is unaware of the prevailing guidelines for measuring pain every shift), associated recommendations for actions to solve mentioned bottlenecks (e.g., organize an educational training session), and supporting materials to facilitate implementation of the actions (e.g., a slide show presentation discussing the importance and relevance of measuring pain every shift). Half of the participating intensive care units will only receive online feedback, while the other half will additionally gain access to the integrated toolbox to facilitate planning and executing actions.

Study Overview

• Status: Completed
• Conditions: Pain Management; Critical Care; Quality Improvement
• Intervention / Treatment: Behavioral: Audit and Feedback; Behavioral: Toolbox

• Study Type: Interventional
• Enrollment (Actual): 21
• Phase: Not Applicable

Contacts and Locations

Study Locations

• Netherlands
  • Noord-Holland
    • Amsterdam, Noord-Holland, Netherlands, 1100 EC
      • National Intensive Care Evaluation foundation

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 16 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

Description

Inclusion Criteria:

• ICUs treating adult (18 years and above) patients
• ICUs willing and able to submit data monthly
• ICUs with a local quality improvement team of at least 1 intensivist and 1 nurse

Exclusion Criteria:

• Nil

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Single

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: Audit and Feedback with toolbox; Access to an online dashboard that provides insight into clinical performance on quality indicators for pain management; including a quality improvement toolbox to facilitate planning and executing actions. Participating ICUs will receive one educational outreach visit and brief monthly telephone calls.	Behavioral: Audit and Feedback; An online dashboard that provides insight into clinical performance on pain management quality indicators. It also incorporates an empty action plan, in which participating intensive care units can define potential bottlenecks in the care process and what actions they intend to undertake to improve.; Other Names: Feedback; A&F; Behavioral: Toolbox; A quality improvement toolbox, incorporated within the action plan, that comprises for each pain management quality indicator (e.g., percentage of patients per shift whose pain is measured) a list of potential bottlenecks in the care process (e.g., staff is unaware of the prevailing guidelines for measuring pain every shift), associated recommendations for actions to solve mentioned bottlenecks (e.g., organize an educational training session), and supporting materials to facilitate implementation of the actions (e.g., a slide show presentation discussing the importance and relevance of measuring pain every shift).; Other Names: Quality Improvement Toolbox
Active Comparator: Audit and Feedback without toolbox; Same intervention, but without access to the quality improvement toolbox.	Behavioral: Audit and Feedback; An online dashboard that provides insight into clinical performance on pain management quality indicators. It also incorporates an empty action plan, in which participating intensive care units can define potential bottlenecks in the care process and what actions they intend to undertake to improve.; Other Names: Feedback; A&F

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Change from proportion of patient shifts during which pain has been adequately managed	"Adequately managed" is defined: pain measured AND (acceptable pain score OR (unacceptable pain score AND normalized within 1 hour)). The primary outcome measure is a composite score of four quality indicators (= secondary outcome measures).The measure is limited to only patients' first 12 shifts to prevent bias from patients with a long stay. There are 3 shifts in 1 day.	9 months

Secondary Outcome Measures

Outcome Measure	Time Frame
Change from proportion of patient shifts during which pain was measured at least once	9 months
Change from proportion of patient shifts during which pain was measured and no unacceptable pain scores were observed	9 months
Change from proportion of patient shifts during which an unacceptable pain score was measured, and pain was timely re-measured	9 months
Change from proportion of patient shifts during which an unacceptable pain score was measured, and pain was timely re-measured indicating that the pain score was normalized	9 months

Collaborators and Investigators

• Sponsor: Academisch Medisch Centrum - Universiteit van Amsterdam (AMC-UvA)
• Collaborators: National Intensive Care Evaluation Foundation
• Investigators: Principal Investigator: Nicolette F de Keizer, PhD, Academisch Medisch Centrum - Universiteit van Amsterdam (AMC-UvA)

Publications and helpful links

General Publications

• Gude WT, Roos-Blom MJ, van der Veer SN, Dongelmans DA, de Jonge E, Peek N, de Keizer NF. Facilitating action planning within audit and feedback interventions: a mixed-methods process evaluation of an action implementation toolbox in intensive care. Implement Sci. 2019 Sep 18;14(1):90. doi: 10.1186/s13012-019-0937-8.
• Roos-Blom MJ, Gude WT, de Jonge E, Spijkstra JJ, van der Veer SN, Peek N, Dongelmans DA, de Keizer NF. Impact of audit and feedback with action implementation toolbox on improving ICU pain management: cluster-randomised controlled trial. BMJ Qual Saf. 2019 Dec;28(12):1007-1015. doi: 10.1136/bmjqs-2019-009588. Epub 2019 Jul 1.
• Gude WT, Roos-Blom MJ, van der Veer SN, Dongelmans DA, de Jonge E, Francis JJ, Peek N, de Keizer NF. Health professionals' perceptions about their clinical performance and the influence of audit and feedback on their intentions to improve practice: a theory-based study in Dutch intensive care units. Implement Sci. 2018 Feb 17;13(1):33. doi: 10.1186/s13012-018-0727-8.
• Gude WT, Roos-Blom MJ, van der Veer SN, de Jonge E, Peek N, Dongelmans DA, de Keizer NF. Electronic audit and feedback intervention with action implementation toolbox to improve pain management in intensive care: protocol for a laboratory experiment and cluster randomised trial. Implement Sci. 2017 May 25;12(1):68. doi: 10.1186/s13012-017-0594-8.

Study record dates

Study Major Dates

• Study Start: September 1, 2016
• Primary Completion (Actual): October 1, 2018
• Study Completion (Actual): October 1, 2018

Study Registration Dates

• First Submitted: September 26, 2016
• First Submitted That Met QC Criteria: October 3, 2016
• First Posted (Estimate): October 4, 2016

Study Record Updates

• Last Update Posted (Actual): January 10, 2019
• Last Update Submitted That Met QC Criteria: January 9, 2019
• Last Verified: January 1, 2019

More Information

Terms related to this study

• Other Study ID Numbers: METC_W16_271

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: Yes
• IPD Plan Description: The National Intensive Care Evaluation (NICE) foundation has a policy for sharing data among its participating ICUs.