Facilitating action planning within audit and feedback interventions: a mixed-methods process evaluation of an action implementation toolbox in intensive care

Wouter T Gude, Marie-José Roos-Blom, Sabine N van der Veer, Dave A Dongelmans, Evert de Jonge, Niels Peek, Nicolette F de Keizer, Wouter T Gude, Marie-José Roos-Blom, Sabine N van der Veer, Dave A Dongelmans, Evert de Jonge, Niels Peek, Nicolette F de Keizer

Abstract

Background: Audit and feedback (A&F) is more effective if it facilitates action planning, but little is known about how best to do this. We developed an electronic A&F intervention with an action implementation toolbox to improve pain management in intensive care units (ICUs); the toolbox contained suggested actions for improvement. A head-to-head randomised trial demonstrated that the toolbox moderately increased the intervention's effectiveness when compared with A&F only.

Objective: To understand the mechanisms through which A&F with action implementation toolbox facilitates action planning by ICUs to increase A&F effectiveness.

Methods: We extracted all individual actions from action plans developed by ICUs that received A&F with (n = 10) and without (n = 11) toolbox for 6 months and classified them using Clinical Performance Feedback Intervention Theory. We held semi-structured interviews with participants during the trial. We compared the number and type of planned and completed actions between study groups and explored barriers and facilitators to effective action planning.

Results: ICUs with toolbox planned more actions directly aimed at improving practice (p = 0.037) and targeted a wider range of practice determinants compared to ICUs without toolbox. ICUs with toolbox also completed more actions during the study period, but not significantly (p = 0.142). ICUs without toolbox reported more difficulties in identifying what actions they could take. Regardless of the toolbox, all ICUs still experienced barriers relating to the feedback (low controllability, accuracy) and organisational context (competing priorities, resources, cost).

Conclusions: The toolbox helped health professionals to broaden their mindset about actions they could take to change clinical practice. Without the toolbox, professionals tended to focus more on feedback verification and exploring solutions without developing intentions for actual change. All feedback recipients experienced organisational barriers that inhibited eventual completion of actions.

Trial registration: ClinicalTrials.gov, NCT02922101 . Registered on 26 September 2016.

Keywords: Feedback; Intensive care; Medical audit; Quality improvement; Quality indicators.

Conflict of interest statement

The authors declare that they have no competing interests.

Figures

Fig. 1
Fig. 1
The NICE dashboard displayed an overview of pain management performance (upper part) and four types of pages specific to the selected indicator (lower part). The difference between study groups was only in the action plan page. The feedback only group received a blank structured action plan and could record and update potential barriers and intended actions. The action plan for the feedback with toolbox group was supplemented with a pre-filled list of potential barriers and suggested actions (indicated by the NICE icon). Some actions included supporting materials (indicated by a wrench icon) available for download. Users could add suggested actions to their action plan and specify their description (plus sign) or hide them if they were not relevant (minus sign)
Fig. 2
Fig. 2
Key findings from the qualitative analysis of the telephone interviews explained by Clinical Performance Feedback Intervention Theory [18]. The identified factors had facilitating (green), inhibiting (red) or mixed (orange) impact on specific processes of the feedback cycle and exerted their effects through various mechanisms. ICU intensive care unit

References

    1. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;6:CD000259. doi: 10.1002/14651858.CD000259.pub3.
    1. No authors listed. Medical audit in general practice. I: Effects on doctors’ clinical behaviour for common childhood conditions. North of England Study of Standards and Performance in General Practice. BMJ 1992;304:1480–1484.
    1. Gardner B, Whittington C, McAteer J, Eccles MP, Michie S. Using theory to synthesise evidence from behaviour change interventions: the example of audit and feedback. Soc Sci Med. 2010;70:1618–1625. doi: 10.1016/j.socscimed.2010.01.039.
    1. Ivers NM, Tu K, Young J, Francis JJ, Barnsley J, Shah BR, et al. Feedback GAP: pragmatic, cluster-randomized trial of goal setting and action plans to increase the effectiveness of audit and feedback interventions in primary care. Implement Sci. 2013;8:142. doi: 10.1186/1748-5908-8-142.
    1. Ivers NM, Grimshaw JM, Jamtvedt G, Flottorp S, O’Brien MA, French SD, et al. Growing literature, stagnant science? Systematic review, meta-regression and cumulative analysis of audit and feedback interventions in health care. J Gen Intern Med. 2014;29:1534–1541. doi: 10.1007/s11606-014-2913-y.
    1. Gude WT, van Engen-Verheul MM, van der Veer SN, Kemps HMC, Jaspers MWM, de Keizer NF, et al. Effect of a web-based audit and feedback intervention with outreach visits on the clinical performance of multidisciplinary teams: a cluster-randomized trial in cardiac rehabilitation. Implement Sci. 2016;11:160. doi: 10.1186/s13012-016-0516-1.
    1. Ivers NM, Sales A, Colquhoun H, Michie S, Foy R, Francis JJ, et al. No more “business as usual” with audit and feedback interventions: towards an agenda for a reinvigorated intervention. Implement Sci. 2014;9:14. doi: 10.1186/1748-5908-9-14.
    1. Ivers NM, Grimshaw JM. Reducing research waste with implementation laboratories. Lancet (London, England) 2016;388:547–548. doi: 10.1016/S0140-6736(16)31256-9.
    1. Colquhoun HL, Carroll K, Eva KW, Grimshaw JM, Ivers N, Michie S, et al. Advancing the literature on designing audit and feedback interventions: identifying theory-informed hypotheses. Implement Sci. 2017;12:117. doi: 10.1186/s13012-017-0646-0.
    1. Ivers N, Barnsley J, Upshur R, Tu K, Shah B, Grimshaw J, et al. My approach to this job is. One person at a time: perceived discordance between population-level quality targets and patient-Centred care. Can Fam Physician. 2014;60:258–266.
    1. Gude WT, Roos-Blom MJ, van der Veer SN, de Jonge E, Peek N, Dongelmans DA, et al. Electronic audit and feedback intervention with action implementation toolbox to improve pain management in intensive care: protocol for a laboratory experiment and cluster randomised trial. Implement Sci. 2017;12:68. doi: 10.1186/s13012-017-0594-8.
    1. Roos-Blom M-J, Gude WT, Spijkstra JJ, de Jonge E, Dongelmans D, de Keizer NF. Measuring quality indicators to improve pain management in critically ill patients. J Crit Care. 2018;49:136–142. doi: 10.1016/j.jcrc.2018.10.027.
    1. Roos-Blom MJ, Gude WT, de Jonge E, Spijkstra JJ, van der Veer SN, Peek N, Dongelmans DA, de Keizer NF. Impact of audit and feedback with action implementation toolbox on improving ICU pain management: cluster-randomised controlled trial. BMJ Qual Saf. 2019. 10.1136/bmjqs-2019-009588. PMID: 31263017.
    1. Carver CS, Scheier MF. Control theory: a useful conceptual framework for personality-social, clinical, and health psychology. Psychol Bull. 1982;92:111–135. doi: 10.1037/0033-2909.92.1.111.
    1. Brehaut JC, Colquhoun HL, Eva KW, Carroll K, Sales A, Michie S, et al. Practice feedback interventions: 15 suggestions for optimizing effectiveness. Ann Intern Med. 2016;164:435–441. doi: 10.7326/M15-2248.
    1. Gude WT, Van Engen-Verheul MM, Van Der Veer SN, De Keizer NF, Peek N. How does audit and feedback influence intentions of health professionals to improve practice? A laboratory experiment and field study in cardiac rehabilitation. BMJ Qual Saf. 2017;26:279–287. doi: 10.1136/bmjqs-2015-004795.
    1. van de Klundert N, Holman R, Dongelmans DA, de Keizer NF. Data resource profile: the Dutch National Intensive Care Evaluation (NICE) registry of admissions to adult intensive care units. Int J Epidemiol. 2015;44:1850–1850h. doi: 10.1093/ije/dyv291.
    1. Brown B, Gude WT, Blakeman T, van der Veer SN, Ivers N, Francis JJ, et al. Clinical Performance Feedback Intervention Theory (CP-FIT): a new theory for designing, implementing, and evaluating feedback in health care based on a systematic review and meta-synthesis of qualitative research. Implement Sci. 2019;14:40. doi: 10.1186/s13012-019-0883-5.
    1. Birt L, Scott S, Cavers D, Campbell C, Walter F. Member checking: a tool to enhance trustworthiness or merely a nod to validation? Qual Health Res. 2016;26:1802–1811. doi: 10.1177/1049732316654870.
    1. Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M, et al. A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. 2013;8:35. doi: 10.1186/1748-5908-8-35.
    1. Morrell C, Harvey G, Kitson A. Practitioner based quality improvement: a review of the Royal College of Nursing’s dynamic standard setting system. Qual Health Care. 1997;6:29–34. doi: 10.1136/qshc.6.1.29.
    1. Grant AM, Guthrie B, Dreischulte T. Developing a complex intervention to improve prescribing safety in primary care: mixed methods feasibility and optimisation pilot study. BMJ Open. 2014;4:e004153. doi: 10.1136/bmjopen-2013-004153.
    1. Tierney S, Kislov R, Deaton C. A qualitative study of a primary-care based intervention to improve the management of patients with heart failure: the dynamic relationship between facilitation and context. BMC Fam Pract. 2014;15:153. doi: 10.1186/1471-2296-15-153.
    1. Sinuff T, Muscedere J, Rozmovits L, Dale CM, Scales DC. A qualitative study of the variable effects of audit and feedback in the ICU. BMJ Qual Saf. 2015;24:393–399. doi: 10.1136/bmjqs-2015-003978.
    1. De Vos MLG, Van Der Veer SN, Graafmans WC, De Keizer NF, Jager KJ, Westert GP, et al. Process evaluation of a tailored multifaceted feedback program to improve the quality of intensive care by using quality indicators. BMJ Qual Saf. 2013;22:233–241. doi: 10.1136/bmjqs-2012-001375.
    1. Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud P-AC, et al. Why don’t physicians follow clinical practice guidelines? J Am Med Assoc. 1999;282:1458. doi: 10.1001/jama.282.15.1458.
    1. Lilford R, Mohammed MA, Spiegelhalter D, Thomson R. Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma. Lancet. 2004;363:1147–1154. doi: 10.1016/S0140-6736(04)15901-1.
    1. Guidet B, González-Romá V. Climate and cultural aspects in intensive care units. Crit Care. 2011;15:312. doi: 10.1186/cc10361.
    1. van der Sluijs AF, van Slobbe-Bijlsma ER, Chick SE, Vroom MB, Dongelmans DA, Vlaar APJ. The impact of changes in intensive care organization on patient outcome and cost-effectiveness—a narrative review. J Intensive Care. 2017;5:13. doi: 10.1186/s40560-016-0207-7.
    1. Hysong SJ, Best RG, Pugh JA. Audit and feedback and clinical practice guideline adherence: making feedback actionable. Implement Sci. 2006;1:9. doi: 10.1186/1748-5908-1-9.
    1. Hysong SJ. Meta-analysis: audit and feedback features impact effectiveness on care quality. Med Care. 2009;47:356–363. doi: 10.1097/MLR.0b013e3181893f6b.
    1. O'Brien MA, Rogers S, Jamtvedt G, Oxman AD, Odgaard-Jensen J, Kristoffersen DT, Forsetlund L, Bainbridge D, Freemantle N, Davis DA, Haynes RB, Harvey EL. Educational outreach visits: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2007;(4):CD000409. .
    1. Reszel Jessica, Dunn Sandra I, Sprague Ann E, Graham Ian D, Grimshaw Jeremy M, Peterson Wendy E, Ockenden Holly, Wilding Jodi, Quosdorf Ashley, Darling Elizabeth K, Fell Deshayne B, Harrold JoAnn, Lanes Andrea, Smith Graeme N, Taljaard Monica, Weiss Deborah, Walker Mark C. Use of a maternal newborn audit and feedback system in Ontario: a collective case study. BMJ Quality & Safety. 2019;28(8):635–644. doi: 10.1136/bmjqs-2018-008354.
    1. Gude WT, van der Veer SN, de Keizer NF, Coiera E, Peek N. Optimizing digital health informatics interventions through unobtrusive quantitative process evaluations. Stud Health Technol Inform. 2016;228:594–598.
    1. Scott PJ, Rigby M, Ammenwerth E, McNair JB, Georgiou A, Hyppönen H, et al. Evaluation considerations for secondary uses of clinical data: principles for an evidence-based approach to policy and implementation of secondary analysis. Yearb Med Inform. 2017;26:59–67. doi: 10.15265/IY-2017-010.
    1. Grant A, Treweek S, Dreischulte T, Foy R, Guthrie B. Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials. 2013;14:15. doi: 10.1186/1745-6215-14-15.

Source: PubMed

Спонсоры и соавторы

Заболевания

Медикаментозные вмешательства

Подписаться