Growing literature, stagnant science? Systematic review, meta-regression and cumulative analysis of audit and feedback interventions in health care

Noah M Ivers, Jeremy M Grimshaw, Gro Jamtvedt, Signe Flottorp, Mary Ann O'Brien, Simon D French, Jane Young, Jan Odgaard-Jensen, Noah M Ivers, Jeremy M Grimshaw, Gro Jamtvedt, Signe Flottorp, Mary Ann O'Brien, Simon D French, Jane Young, Jan Odgaard-Jensen

Abstract

Background: This paper extends the findings of the Cochrane systematic review of audit and feedback on professional practice to explore the estimate of effect over time and examine whether new trials have added to knowledge regarding how optimize the effectiveness of audit and feedback.

Methods: We searched the Cochrane Central Register of Controlled Trials, MEDLINE, and EMBASE for randomized trials of audit and feedback compared to usual care, with objectively measured outcomes assessing compliance with intended professional practice. Two reviewers independently screened articles and abstracted variables related to the intervention, the context, and trial methodology. The median absolute risk difference in compliance with intended professional practice was determined for each study, and adjusted for baseline performance. The effect size across studies was recalculated as studies were added to the cumulative analysis. Meta-regressions were conducted for studies published up to 2002, 2006, and 2010 in which characteristics of the intervention, the recipients, and trial risk of bias were tested as predictors of effect size.

Results: Of the 140 randomized clinical trials (RCTs) included in the Cochrane review, 98 comparisons from 62 studies met the criteria for inclusion. The cumulative analysis indicated that the effect size became stable in 2003 after 51 comparisons from 30 trials. Cumulative meta-regressions suggested new trials are contributing little further information regarding the impact of common effect modifiers. Feedback appears most effective when: delivered by a supervisor or respected colleague; presented frequently; featuring both specific goals and action-plans; aiming to decrease the targeted behavior; baseline performance is lower; and recipients are non-physicians.

Discussion: There is substantial evidence that audit and feedback can effectively improve quality of care, but little evidence of progress in the field. There are opportunity costs for patients, providers, and health care systems when investigators test quality improvement interventions that do not build upon, or contribute toward, extant knowledge.

Conflict of interest statement

The authors declare that they do not have a conflict of interest.

Figures

Figure 1
Figure 1
Cumulative number of randomized trials featuring audit and feedback as a core component of a quality improvement intervention.
Figure 2
Figure 2
Study flow diagram.
Figure 3
Figure 3
Cumulative analysis–effect size* of audit and feedback interventions over time (AF: audit and feedback; *absolute difference in compliance with intended professional behaviors).

References

    1. Davis DA, Mazmanian PE, Fordis M, Van Harrison RTKE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA. 2006;296(9):1094. doi: 10.1001/jama.296.9.1094.
    1. Hysong SJ, Teal CR, Khan MJ, Haidet P. Improving quality of care through audit and feedback. Implement Sci. 2012;7:45. doi: 10.1186/1748-5908-7-45.
    1. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;6:CD000259.
    1. Jamtvedt G, Young JM, Kristoffersen DT, Thomson O’Brien MA, Oxman AD. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2003;3(3):CD000259.
    1. Jamtvedt G, Young JM, Kristoffersen DT, O’Brien MA, Oxman AD. Does telling people what they have been doing change what they do? A systematic review of the effects of audit and feedback. Qual Saf Health Care. 2006;15(6):433–6. doi: 10.1136/qshc.2006.018549.
    1. Foy R, Eccles MP, Jamtvedt G, Young J, Grimshaw JM, Baker R. What do we know about how to do audit and feedback? Pitfalls in applying evidence from a systematic review. BMC Health Serv Res. 2005;5:50. doi: 10.1186/1472-6963-5-50.
    1. Gardner B, Whittington C, Mcateer J, Eccles MP, Michie S. Using theory to synthesise evidence from behaviour change interventions: the example of audit and feedback. Soc Sci Med. 2010;70(10):1618–25. doi: 10.1016/j.socscimed.2010.01.039.
    1. Hysong SJ. Meta-analysis: audit and feedback features impact effectiveness on care quality. Med Care. 2009;47(3):356–63. doi: 10.1097/MLR.0b013e3181893f6b.
    1. Colquhoun HL, Brehaut JC, Sales A, Ivers N, Grimshaw J, Michie S, et al. A systematic review of the use of theory in randomized controlled trials of audit and feedback. Implement Sci. 2013;8(1):66. doi: 10.1186/1748-5908-8-66.
    1. Suggested Risk Of Bias Criteria For EPOC Review. Cochrane Effective Practice And Organisation Of Care (EPOC) Group. 2013. Available At: (Accessed 2 May 2014).
    1. Higgins JPT, Green S. Cochrane Handbook For Systematic Reviews Of Interventions. Cochrane Database Of Systematic Reviews. 2008. Available at: (Accessed 2 May 2014).
    1. Grimshaw JM, Thomas RE, Maclennan G, Fraser C, Ramsay CR, Vale L, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004;8(6):1–72. doi: 10.3310/hta8060.
    1. Flodgren G, Parmelli E, Doumit G, Gattellari M, O’Brien MA, Grimshaw J. Local opinion leaders: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2011;8(8):CD000125.
    1. Ivers NM, Tu K, Francis J, Barnsley J, Shah B, Upshur R, et al. Feedback GAP: study protocol for a cluster-randomized trial of goal setting and action plans to increase the effectiveness of audit and feedback interventions in primary care. Implement Sci. 2010;5:98. doi: 10.1186/1748-5908-5-98.
    1. Gilbody S, Bower P, Fletcher J, Richards D, Sutton AJ. Collaborative care for depression: a cumulative meta-analysis and review of longer-term outcomes. Arch Intern Med. 2006;166(21):2314–21. doi: 10.1001/archinte.166.21.2314.
    1. Dent L, Taylor R, Jolly K, Raftery J. “Flogging dead horses” evaluating when have clinical trials achieved sufficiency and stability? A case study in cardiac rehabilitation. Trials. 2011;12:83. doi: 10.1186/1745-6215-12-83.
    1. Pogue JM, Yusuf S. Cumulating evidence from randomized trials: utilizing sequential monitoring boundaries for cumulative meta-analysis. Control Clin Trials. 1997;18(6):580–93. doi: 10.1016/S0197-2456(97)00051-2.
    1. Lau J, Antman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers TC. Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med. 1992;327(4):248–54. doi: 10.1056/NEJM199207233270406.
    1. Van Der Veer SN, De Keizer NF, Ravelli AC, Tenkink S, Jager KJ. Improving quality of care. A systematic review on how medical registries provide information feedback to health care providers. Int J Med Inform. 2010;79(5):305–23. doi: 10.1016/j.ijmedinf.2010.01.011.
    1. Hysong SJ, Best RG, Pugh JA. Audit and feedback and clinical practice guideline adherence: making feedback actionable. Implement Sci. 2006;1:9. doi: 10.1186/1748-5908-1-9.
    1. Ivers NM, Sales A, Colquhoun HL, Michie S, Foy R, Francis JJ, et al. No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention. Implement Sci. 2014;9:14. doi: 10.1186/1748-5908-9-14.
    1. Collins LM, Murphy SA, Nair V, Strecher VJ. A strategy for optimizing and evaluating behavioural interventions. Ann Behav Med. 2005;30(1):65–73. doi: 10.1207/s15324796abm3001_8.
    1. Chokshi DA, Avorn J, Kesselheim AS. Designing comparative effectiveness research on prescription drugs: lessons from the clinical trial literature. Health Aff (Millwood) 2010;29(10):1842–8. doi: 10.1377/hlthaff.2010.0843.
    1. Auerbach AD, Landefeld CS, Shojania KG. The tension between needing to improve care and knowing how to do it. N Engl J Med. 2007;357(6):608–13. doi: 10.1056/NEJMsb070738.
    1. Macleod MR, Michie S, Roberts I, Dirnagl U, Chalmers I, Ionnadis JP, et al. Biomedical value: increasing value, reducing waste. Lancet. 2014;383(9912):101–4. doi: 10.1016/S0140-6736(13)62329-6.
    1. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, et al. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:A1655. doi: 10.1136/bmj.a1655.
    1. Goudie AC, Sutton AJ, Jones DR, Donald A. Empirical assessment suggests that existing evidence could be used more fully in designing randomized controlled trials. J Clin Epidemiol. 2010;63(9):983–91. doi: 10.1016/j.jclinepi.2010.01.022.
    1. O’Leary M, Krailo M, Anderson JR, Reaman GH, Children’s Oncology Group Progress in childhood cancer: 50 years of research collaboration, a report from the Children’s Oncology Group. Semin Oncol. 2008;35(5):484–93. doi: 10.1053/j.seminoncol.2008.07.008.

Source: PubMed

Patrocinadores e Colaboradores

Condições médicas

Intervenções de drogas

3
Se inscrever