Defining certainty of net benefit: a GRADE concept paper

Brian S Alper, Peter Oettgen, Ilkka Kunnamo, Alfonso Iorio, Mohammed Toseef Ansari, M Hassan Murad, Joerg J Meerpohl, Amir Qaseem, Monica Hultcrantz, Holger J Schünemann, Gordon Guyatt, GRADE Working Group, Brian S Alper, Peter Oettgen, Ilkka Kunnamo, Alfonso Iorio, Mohammed Toseef Ansari, M Hassan Murad, Joerg J Meerpohl, Amir Qaseem, Monica Hultcrantz, Holger J Schünemann, Gordon Guyatt, GRADE Working Group

Abstract

Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology is used to assess and report certainty of evidence and strength of recommendations. This GRADE concept article is not GRADE guidance but introduces certainty of net benefit, defined as the certainty that the balance between desirable and undesirable health effects is favourable. Determining certainty of net benefit requires considering certainty of effect estimates, the expected importance of outcomes and variability in importance, and the interaction of these concepts. Certainty of net harm is the certainty that the net effect is unfavourable. Guideline panels using or testing this approach might limit strong recommendations to actions with a high certainty of net benefit or against actions with a moderate or high certainty of net harm. Recommendations may differ in direction or strength from that suggested by the certainty of net benefit or harm when influenced by cost, equity, acceptability or feasibility.

Keywords: clinical decision making; decision analysis; evidence synthesis; evidence-based medicine; guideline development.

Conflict of interest statement

Competing interests: All authors are members of the GRADE Working Group and conduct scholarly activity or professional services related to the concepts in this article. BSA and PO are employed by EBSCO Information Services and IK is employed by Duodecim Medical Publications Ltd.

© Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Figures

Figure 1
Figure 1
Certainty across the evidence-to-decision framework*.
Figure 2
Figure 2
A stepwise approach to determining the certainty of the net effect estimate.
Figure 3
Figure 3
Classification of precision of net effect estimate.

References

    1. [Accessed 27 Nov 2016].
    1. Atkins D, Best D, Briss PA, et al. . Grading quality of evidence and strength of recommendations. BMJ 2004;328:1490 10.1136/bmj.328.7454.1490
    1. Balshem H, Helfand M, Schünemann HJ, et al. . GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol 2011;64:401–6. 10.1016/j.jclinepi.2010.07.015
    1. Hultcrantz M, Rind D, Akl EA, et al. . The GRADE Working Group clarifies the construct of certainty of evidence. J Clin Epidemiol 2017;87:4–13. 10.1016/j.jclinepi.2017.05.006
    1. Andrews JC, Schünemann HJ, Oxman AD, et al. . GRADE guidelines: 15. Going from evidence to recommendation-determinants of a recommendation’s direction and strength. J Clin Epidemiol 2013;66:726–35. 10.1016/j.jclinepi.2013.02.003
    1. Schunemann H, Brozek J, Guyatt G, et al. . Handbook for grading the quality of evidence and the strength of recommendations using the GRADE approach. Updated October 2013. Section 6.3.2 Confidence in best estimates of magnitude of effects (quality of evidence). Available: [Accessed 27 Nov 2016].
    1. Guyatt GH, Oxman AD, Kunz R, et al. . Going from evidence to recommendations. BMJ 2008;336:1049–51. 10.1136/
    1. Schunemann H, Brozek J, Guyatt G, et al. . Handbook for grading the quality of evidence and the strength of recommendations using the GRADE approach. Updated October 2013. Section 6.3.3 Confidence in values and preferences. Available: [Accessed 27 Nov 2016].
    1. Schunemann H, Brozek J, Guyatt G, et al. . Handbook for grading the quality of evidence and the strength of recommendations using the GRADE approach. Updated October 2013. Section 6.2.1 Balance of desirable and undesirable consequences. Available: [Accessed 27 Nov 2016].
    1. Schunemann H, Brozek J, Guyatt G, et al. . Handbook for grading the quality of evidence and the strength of recommendations using the GRADE approach. Updated October 2013. Section 6.3.4 Resource use (cost). Available: [Accessed 27 Nov 2016].
    1. Schunemann H, Brozek J, Guyatt G, et al. . Handbook for grading the quality of evidence and the strength of recommendations using the GRADE approach. Updated October 2013. Section 6.5 The Evidence-to-Decision framework. Available: [Accessed 27 Nov 2016].
    1. Sawaya GF, Guirguis-Blake J, LeFevre M, et al. . Update on the methods of the U.S. Preventive Services Task Force: estimating certainty and magnitude of net benefit. Ann Intern Med 2007;147:871–5. 10.7326/0003-4819-147-12-200712180-00007
    1. Krist AH, Wolff TA, Jonas DE, et al. . Update on the Methods of the U.S. Preventive Services Task Force: Methods for Understanding Certainty and Net Benefit When Making Recommendations. Am J Prev Med 2018;54:S11–S18. 10.1016/j.amepre.2017.09.011
    1. Schunemann H, Brozek J, Guyatt G, et al. . Handbook for grading the quality of evidence and the strength of recommendations using the GRADE approach. Updated October 2013. Section 3 Selecting and rating the importance of outcomes. Available: [Accessed 6 Apr 2019].
    1. Hazlewood GS, Bombardier C, Tomlinson G, et al. . A Bayesian model that jointly considers comparative effectiveness research and patients' preferences may help inform GRADE recommendations: an application to rheumatoid arthritis treatment recommendations. J Clin Epidemiol 2018;93:56–65. 10.1016/j.jclinepi.2017.10.003
    1. Fain KM, Yu T, Li T, et al. . Evidence selection for a prescription drug’s benefit-harm assessment: challenges and recommendations. J Clin Epidemiol 2016;74:151–7. 10.1016/j.jclinepi.2016.02.018
    1. MacLean S, Mulla S, Akl EA, et al. . Patient values and preferences in decision making for antithrombotic therapy: a systematic review: Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. Chest 2012;141:e1S–e23S. 10.1378/chest.11-2290
    1. Richards T, Schroter S, Price A, et al. . Better together: patient partnership in medical journals. BMJ 2018;362:k3798 10.1136/bmj.k3798
    1. Schünemann HJ. Interpreting GRADES’s levels of certainty or quality of the evidence: GRADE for statisticians, considering review information size or less emphasis on imprecision? J Clin Epidemiol 2016;75:6–15. 10.1016/j.jclinepi.2016.03.018
    1. Efron B, Tibshirani RJ. An Introduction to the Bootstrap, 1st edn New York, NY: Chapman and Hall/CRC, 1993.
    1. Spiegelhalter DJ, Myles JP, Jones DR, et al. . An introduction to Bayesian methods in health technology assessment. BMJ 1999;319:508–12.
    1. Zhang Y, Alonso-Coello P, Guyatt GH, et al. . GRADE Guidelines: 19. Assessing the certainty of evidence in the importance of outcomes or values and preferences-Risk of bias and indirectness. J Clin Epidemiol 2018. 10.1016/j.jclinepi.2018.01.013
    1. Smith CM. Origin and uses of primum all do no harm!. J Clin Pharmacol 2005;45:371–7. 10.1177/0091270004273680
    1. Schünemann HJ. Guidelines 2.0: do no net harm-the future of practice guideline development in asthma and other diseases. Curr Allergy Asthma Rep 2011;11:261–8. 10.1007/s11882-011-0185-8

Source: PubMed

Patrocinadores e Colaboradores

Condições médicas

Intervenções de drogas

3
Se inscrever