Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group

David Atkins, Martin Eccles, Signe Flottorp, Gordon H Guyatt, David Henry, Suzanne Hill, Alessandro Liberati, Dianne O'Connell, Andrew D Oxman, Bob Phillips, Holger Schünemann, Tessa Tan-Torres Edejer, Gunn E Vist, John W Williams Jr, GRADE Working Group, David Atkins, Martin Eccles, Signe Flottorp, Gordon H Guyatt, David Henry, Suzanne Hill, Alessandro Liberati, Dianne O'Connell, Andrew D Oxman, Bob Phillips, Holger Schünemann, Tessa Tan-Torres Edejer, Gunn E Vist, John W Williams Jr, GRADE Working Group

Abstract

Background: A number of approaches have been used to grade levels of evidence and the strength of recommendations. The use of many different approaches detracts from one of the main reasons for having explicit approaches: to concisely characterise and communicate this information so that it can easily be understood and thereby help people make well-informed decisions. Our objective was to critically appraise six prominent systems for grading levels of evidence and the strength of recommendations as a basis for agreeing on characteristics of a common, sensible approach to grading levels of evidence and the strength of recommendations.

Methods: Six prominent systems for grading levels of evidence and strength of recommendations were selected and someone familiar with each system prepared a description of each of these. Twelve assessors independently evaluated each system based on twelve criteria to assess the sensibility of the different approaches. Systems used by 51 organisations were compared with these six approaches.

Results: There was poor agreement about the sensibility of the six systems. Only one of the systems was suitable for all four types of questions we considered (effectiveness, harm, diagnosis and prognosis). None of the systems was considered usable for all of the target groups we considered (professionals, patients and policy makers). The raters found low reproducibility of judgements made using all six systems. Systems used by 51 organisations that sponsor clinical practice guidelines included a number of minor variations of the six systems that we critically appraised.

Conclusions: All of the currently used approaches to grading levels of evidence and the strength of recommendations have important shortcomings.

References

    1. Canadian Task Force on the Periodic Health Examination The periodic health examination. Can Med Assoc J. 1979;121:1193–254.
    1. Sackett DL. Rules of evidence and clinical recommendations on the use of antithrombotic agents. Chest. 1986;89:2S–3S.
    1. Sackett DL. Rules of evidence and clinical recommendations on the use of antithrombotic agents. Archives Int Med. 1986;146:464–465.
    1. Sackett DL. Rules of evidence and clinical recommendations on the use of antithrombotic agents. Chest. 1989;95:2S–4S.
    1. Cook DJ, Guyatt GH, Laupacis A, Sackett DL. Rules of evidence and clinical recommendations on the use of antithrombotic agents. Antithrombotic Therapy Consensus Conference. Chest. 1992;102:305S–311S.
    1. US Department of Health and Human Services, Public Health Service, Agency Health Care Policy and Research Acute Pain Management: Operative or Medical Procedures and Trauma. Agency for Health Care Policy and Research Publications, Rockville, MD (AHCPR Pub 92-0038) 1992.
    1. Gyorkos TW, Tannenbaum TN, Abrahamowicz M, Oxman AD, Scott EA, Millson ME, Rasooly I, Frank JW, Riben PD, Mathias RG. An approach to the development of practice guidelines for community health interventions. Can J Public Health. 1994;85:S8–S13.
    1. Hadorn DC, Baker D. Development of the AHCPR-sponsored heart failure guideline: methodologic and procedural issues. J Quality Improvement. 1994;20:539–54.
    1. Cook DJ, Guyatt GH, Laupacis A, Sackett DL, Goldberg RJ. Clinical recommendations using levels of evidence for antithrombotic agents. Chest. 1995;108:227S–230S.
    1. Guyatt GH, Sackett DL, Sinclair JC, Hayward R, Cook DJ, Cook RJ, for the Evidence-Based Medicine Working Group User's guides to the medical literature.1X. A method for grading health care recommendations. Evidence-Based medicine working group. JAMA. 1995;274:1800–4. doi: 10.1001/jama.274.22.1800.
    1. Petrie J, Barnwell E, Grimshaw J. Criteria for appraisal for national use. Pilot Edition. Scottish Intercollegiate Guidelines Network. 1995.
    1. US Preventive Services Task Force . Guide to Clinical Preventive Services. 2. Baltimore: Williams & Wilkins; 1996. pp. xxxix–lv.
    1. Eccles M, Clapp Z, Grimshaw J, Adams PC, Higgins B, Purves I, Russell I. North of England evidence based guidelines development project: methods of guideline development. BMJ. 1996;312:760–2.
    1. Centro per la Valutazione della Efficacia della Assistenza Sanitaria (CeVEAS) Linee Guida per il trattamento del tumore della mammella nella provincia di Modena (Luglio 2000) accessed December 29, 2002.
    1. Guyatt GH, Cook DJ, Sackett DL, Eckman M, Pauker S. Grades of recommendation for antithrombotic agents. Chest. 1998;114:441S–4S.
    1. Ball C, Sackett D, Phillips B, Straus S, Haynes B. Levels of evidence and grades of recommendations. Last revised 17 September 1998. Centre for Evidence-Based Medicine;
    1. National Health and Medical Research Council How to use the evidence: assessment and application of scientific evidence. Commonwealth of Australia. 2000.
    1. Harbour R, Miller J. A new system for grading recommendations in evidence based guidelines. BMJ. 2001;323:334–6. doi: 10.1136/bmj.323.7308.334.
    1. Roman SH, Silberzweig SB, Siu AL. Grading the evidence for diabetes performance measures [see comments] Eff Clin Pract. 2000;3:85–91.
    1. Woloshin S. Arguing about grades. Eff Clin Pract. 2000;3:94–5.
    1. Guyatt GH, Schünemann H, Cook D, Pauker S, Sinclair J, Bucher H, Jaeschke R. Grades of recommendation for antithrombotic agents. Chest. 2001;119:3S–7S. doi: 10.1378/chest.119.1_suppl.3S.
    1. Atkins D, Best D, Shapiro EN. The third U.S. Preventive Services Task Force : background, methods and first recommendations. Am J Preventive Medicine. 2001;20:1–108.
    1. Woolf SH, Atkins D. The evolving role of prevention in health care: Contributions of the U.S. Preventive Services Task Force. Am J Preventive Medicine. 2001;20:13–20. doi: 10.1016/S0749-3797(01)00262-8.
    1. Harris RP, Helfand M, Woolf SH, Lohr KN, Mulrow CD, Teutsch SM, Atkins D, for the Methods Work Group of the Third U.S. Preventive Services Task Force Current methods of the U.S. Preventive Services Task Force: A review of the process. Am J Preventive Medicine. 2001;20:21–35. doi: 10.1016/S0749-3797(01)00261-6.
    1. Briss PA, Zaza S, Pappaioanou M, Fielding J, Wright-De Aguero L, Truman BI, Hopkins DP, Mullen PD, Thompson RS, Woolf SH, Carande-Kulis VG, Anderson L, Hinman AR, McQueen DV, Teutsch SM, Harris JR. Developing an evidence-based Guide to Community Preventive Services – methods. The Task Force on Community Preventive Services. Am J Preventive Medicine. 2000;18:35–43. doi: 10.1016/S0749-3797(99)00119-1.
    1. Zaza S, Wright-De Aguero LK, Briss PA, Truman BI, Hopkins DP, Hennessy MH, Sosin DM, Anderson L, Carande-Kulis VG, Teutsch SM, Pappaioanou M. Data collection instrument and procedure for systematic reviews in the Guide to Community Preventive Services. Task Force on Community Preventive Services. American Journal of Preventive Medicine. 2000;18:44–74. doi: 10.1016/S0749-3797(99)00122-1.
    1. Greer N, Mosser G, Logan G, Halaas GW. A practical approach to evidence grading. Joint Commission J Qual Improv. 2000;26:700–12.
    1. West S, King V, Carey TS, Lohr KN, McKoy N, Sutton SF, Lux L. AHRQ Publication No 02-E016. Rockville, MD: Agency for Healthcare Research and Quality; 2002. Systems to Rate the Strength of Scientific Evidence. Evidence Report/Technology Assessment No. 47 (Prepared by the Research Triangle Institute-University of North Carolina Evidence-based Practice Center under Contract No. 290-97-0011) pp. 64–88.
    1. Feinstein AR. Clinimetrics. New Haven, CT: Yale University Press; 1987. pp. 141–66.
    1. National Guidelines Clearing House Accessed April 19, 2001.
    1. Guyatt G, Drummond R, eds . Users' Guide to the Medical Literature. Chicago, IL: AMA Press; 2002. pp. 55–154.
    1. West S, King V, Carey TS, Lohr KN, McKoy N, Sutton SF, Lux L. AHRQ Publication No 02-E016. Rockville, MD: Agency for Healthcare Research and Quality; 2002. Systems to Rate the Strength of Scientific Evidence. Evidence Report/Technology Assessment No. 47 (Prepared by the Research Triangle Institute-University of North Carolina Evidence-based Practice Center under Contract No. 290-97-0011) pp. 51–63.
    1. Atkins D, Briss PA, Eccles M, Flottorp S, Guyatt GH, Harbour RT, Hill S, Jaeschke R, Liberati A, Magrini N, Mason J, O'Connell D, Oxman AD, Phillips B, Schunemann HJ, Edejer TT, Vist GE, Williams JW, Jr, GRADE Working Group Systems for grading the quality of evidence and the strength of recommendations II: Pilot study of a new system. BioMed Central.
    1. Atkins D, Best D, Briss PA, Eccles M, Falck Ytter Y, Flottorp S, Guyatt GH, Harbour RT, Haugh MC, Henry D, Hill S, Jaeschke R, Leng G, Liberati A, Magrini N, Mason J, Middleton P, Mrukowicz J, O'Connell D, Oxman AD, Phillips B, Schunemann HJ, Edejer TT, Varonen H, Vist GE, Williams JW, Jr, Zaza S, Grade Working Group Grading quality of evidence and strength of recommendations. BMJ. p. 1490. 2004 Jun 19.

Source: PubMed

Спонсоры и соавторы

Заболевания

Медикаментозные вмешательства

Подписаться