Quality improvement needed in quality improvement randomised trials: systematic review of interventions to improve care in diabetes

Noah M Ivers, Andrea C Tricco, Monica Taljaard, Ilana Halperin, Lucy Turner, David Moher, Jeremy M Grimshaw, Noah M Ivers, Andrea C Tricco, Monica Taljaard, Ilana Halperin, Lucy Turner, David Moher, Jeremy M Grimshaw

Abstract

Objective: Despite the increasing numbers of published trials of quality improvement (QI) interventions in diabetes, little is known about the risk of bias in this literature.

Design: Secondary analysis of a systematic review.

Data sources: Medline, the Cochrane Effective Practice and Organisation of Care (EPOC) database (from inception to July 2010) and references of included studies.

Eligibility criteria: Randomised trials assessing 11 predefined QI strategies or financial incentives targeting health systems, healthcare professionals or patients to improve the management of adult outpatients with diabetes.

Analysis: Risk of bias (low, unclear or high) was assessed for the 142 trials in the review across nine domains using the EPOC version of the Cochrane Risk of Bias Tool. We used Cochran-Armitage tests for trends to evaluate the improvement over time.

Results: There was no significant improvement over time in any of the risk of bias domains. Attrition bias (loss to follow-up) was the most common source of bias, with 24 trials (17%) having high risk of bias due to incomplete outcome data. Overall, 69 trials (49%) had at least one domain with high risk of bias. Inadequate reporting frequently hampered the risk of bias assessment: allocation sequence was unclear in 82 trials (58%) and allocation concealment was unclear in 78 trials (55%). There were significant reductions neither in the proportions of studies at high risk of bias over time nor in the adequacy of reporting of risk of bias domains.

Conclusions: Nearly half of the included QI trials in this review were judged to have high risk of bias. Such trials have serious limitations that put the findings in question and therefore inhibit evidence-based QI. There is a need to limit the potential for bias when conducting QI trials and improve the quality of reporting of QI trials so that stakeholders have adequate evidence for implementation.

Figures

Figure 1
Figure 1
Study flow diagram.
Figure 2
Figure 2
Percentage of studies judged to be at low, unclear or high risk of bias in each risk of bias domain.
Figure 3
Figure 3
Cumulative number of diabetes quality improvement trial publications at high risk of bias in any domain, 1990–2010.

References

    1. Tricco AC, Ivers NM, Grimshaw JM, et al. Effectiveness of quality improvement strategies on the management of diabetes: a systematic review and meta-analysis. Lancet 2012;379:2252–61
    1. Pildal J, Hrobjartsson A, Jorgensen KJ, et al. Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials. Int J Epidemiol 2007;36:847–57
    1. Fan E, Laupacis A, Pronovost PJ, et al. How to use an article about quality improvement. JAMA 2010;304:2279–87
    1. Higgins JP, Altman DG, Gotzsche PC, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928.
    1. Falagas ME, Grigori T, Ioannidou E. A systematic review of trends in the methodological quality of randomized controlled trials in various research fields. J Clin Epidemiol 2009;62:227–31, 31 e1–9
    1. Cochrane Effective Practice and Organisation of Care Group Suggested risk of bias criteria for EPOC reviews, 2012
    1. SJR—SCImago Journal & Country Rank, 2007 [program]
    1. Falagas ME, Kouranos VD, Arencibia-Jorge R, et al. Comparison of SCImago journal rank indicator with journal impact factor. FASEB J 2008;22:2623–8
    1. Higgins JPT, Altman DG. Assessing risk of bias in included studies. In: Higgins JPT, Green S. (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from .
    1. Hartling L, Ospina M, Liang Y, et al. Risk of bias versus quality assessment of randomised controlled trials: cross sectional study. BMJ 2009;339:b4012.
    1. Ivers NM, Halperin IJ, Barnsley J, et al. Allocation techniques for balance at baseline in cluster randomized trials: a methodological review. Trials 2012;13:120.
    1. Moher D, Schulz KF, Altman D, et al. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA 2001;285:1987–91
    1. Shojania KG, Ranji SR, McDonald KM, et al. Effects of quality improvement strategies for type 2 diabetes on glycemic control: a meta-regression analysis. JAMA 2006;296:427–40
    1. Armijo-Olivo S, Stiles CR, Hagen NA, et al. Assessment of study quality for systematic reviews: a comparison of the Cochrane Collaboration Risk of Bias Tool and the Effective Public Health Practice Project Quality Assessment Tool: methodological research. J Eval Clin Pract 2012;18:12–18
    1. Odgaard-Jensen J, Vist GE, Timmer A, et al. Randomisation to protect against selection bias in healthcare trials. Cochrane Database of Systematic Reviews 2011;4: MR000012. doi: 10.1002/14651858.MR000012.pub3.
    1. Hempel S, Suttorp MJ, Miles JNV, et al. Empirical Evidence of Associations Between Trial Quality and Effect Size [Internet]. Rockville, MD: Agency for Healthcare Research and Quality (US); 2011
    1. SAS 9.2 [program]: Cary, NC: SAS Institute Inc., 2008
    1. Simera I, Moher D, Hirst A, et al. Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR network. BMC Med 2010;8:24.
    1. Davidoff F, Batalden P, Stevens D, et al. Publication guidelines for quality improvement studies in health care: evolution of the SQUIRE project. BMJ 2009;338:a3152.
    1. Ivers NM, Taljaard M, Dixon S, et al. Impact of CONSORT extension for cluster randomised trials on quality of reporting and study methodology: review of random sample of 300 trials, 2000–8. BMJ 2011;343:d5886.
    1. Campbell MK, Elbourne DR, Altman DG, Group C . CONSORT statement: extension to cluster randomised trials. BMJ 2004;328:702–8
    1. Michie S, Fixsen D, Grimshaw JM, et al. Specifying and reporting complex behaviour change interventions: the need for a scientific method. Implement Sci 2009;4:40.
    1. Alexander JA, Hearld LR. What can we learn from quality improvement research? A critical review of research methods. Med Care Res Rev 2009;66:235–71
    1. Eccles M, Grimshaw J, Walker A, et al. Changing the behavior of healthcare professionals: the use of theory in promoting the uptake of research findings. J Clin Epidemiol 2005;58:107–12
    1. Hempel S, Rubenstein LV, Shanman RM, et al. Identifying quality improvement intervention publications—a comparison of electronic search strategies. Implement Sci 2011;6:85.

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe