Good practices for real-world data studies of treatment and/or comparative effectiveness: Recommendations from the joint ISPOR-ISPE Special Task Force on real-world evidence in health care decision making

Marc L Berger, Harold Sox, Richard J Willke, Diana L Brixner, Hans-Georg Eichler, Wim Goettsch, David Madigan, Amr Makady, Sebastian Schneeweiss, Rosanna Tarricone, Shirley V Wang, John Watkins, C Daniel Mullins, Marc L Berger, Harold Sox, Richard J Willke, Diana L Brixner, Hans-Georg Eichler, Wim Goettsch, David Madigan, Amr Makady, Sebastian Schneeweiss, Rosanna Tarricone, Shirley V Wang, John Watkins, C Daniel Mullins

Abstract

Purpose: Real-world evidence (RWE) includes data from retrospective or prospective observational studies and observational registries and provides insights beyond those addressed by randomized controlled trials. RWE studies aim to improve health care decision making.

Methods: The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the International Society for Pharmacoepidemiology (ISPE) created a task force to make recommendations regarding good procedural practices that would enhance decision makers' confidence in evidence derived from RWD studies. Peer review by ISPOR/ISPE members and task force participants provided a consensus-building iterative process for the topics and framing of recommendations.

Results: The ISPOR/ISPE Task Force recommendations cover seven topics such as study registration, replicability, and stakeholder involvement in RWE studies. These recommendations, in concert with earlier recommendations about study methodology, provide a trustworthy foundation for the expanded use of RWE in health care decision making.

Conclusion: The focus of these recommendations is good procedural practices for studies that test a specific hypothesis in a specific population. We recognize that some of the recommendations in this report may not be widely adopted without appropriate incentives from decision makers, journal editors, and other key stakeholders.

Keywords: comparative effectiveness; decision making; guidelines; pharmacoepidemiology; real-world data; treatment effectiveness.

© 2017 The Authors. Pharmacoepidemiology & Drug Safety published by John Wiley & Sons Ltd.

Figures

Figure 1
Figure 1
Recommendations for good procedural practices for Hypothesis Evaluating Treatment Effectiveness Studies

References

    1. Garrison LP, Neumann PJ, Erickson P, Marshall D, Mullins CD. Using real‐world data for coverage and payment decisions: The ISPOR real‐world data task force report. Value Health. 2006;10(5):326‐335.
    1. Makady A, de Boer A, Hillege H, Klungel O, Goettsch W. What is real‐world data? A review of definitions based on literature and stakeholder interviews. Value Health. 2017;20(7):858‐865.
    1. Makady A, Ham RT, de Boer A, Hillege H, Klungel O, Goettsch W. GetReal Workpackage. Policies for use of real‐world data in health technology assessment (HTA): A comparative study of six HTA agencies. Value Health. 2017;20(4):520‐532.
    1. Sherman RE, Anderson SA, Dal Pan GJ, et al. Real‐world evidence—what is it and what can it tell us? N Engl J Med. 2016;375(23):2293‐2297.
    1. Brixner DI, Holtorf AP, Neumann PJ, Malone DC, Watkins JB. Standardizing quality assessment of observational studies for decision making in health care. J Manag Care Pharm. 2009;15(3):275‐283. Available at:
    1. U.S. Food and Drug Administration . Guidance for industry and FDA staff. Best practices for conducting and reporting pharmacoepidemiologic safety studies using electronic healthcare data. May 2013. Available at:
    1. The PLOS Medicine Editors . Observational studies: Getting clear about transparency. PLoS Med. 2014;11(8):e1001711. ). Accessed December 10, 2015
    1. Segal JB, Kallich JD, Oppenheim ER, et al. Using certification to promote uptake of real‐world evidence by payers. J Manag Care Pharm. 2016;22(3):191‐196.
    1. International Committee of Medical Journal Editors. (accessed July 6, 2017)
    1. US Food and Drug Administration. (accessed July 6, 2017)
    1. Motheral B, Brooks J, Clark MA, et al. A checklist for retroactive database studies—report of the ISPOR task force on retrospective databases. Value Health. 2003;6(2):90‐97.
    1. Berger ML, Mamdani M, Atkins D, Johnson ML. Good research practices for comparative effectiveness research: Defining, reporting and interpreting nonrandomized studies of treatment effects using secondary data sources: the ISPOR good research practices for retrospective database analysis task force report—part I. Value Health. 2009;12(8):1044‐1052.
    1. Cox E, Martin BC, Van Staa T, Garbe E, Siebert U, Johnson ML. Good research practices for comparative effectiveness research: Approaches to mitigate bias and confounding in the design of non‐randomized studies of treatment effects using secondary data sources: The ISPOR good research practices for retrospective database analysis task force—part II. Value Health. 2009;12(8):1053‐1061.
    1. Johnson ML, Crown W, Martin BC, Dartmouth CR, Siebert U, Normand SL. Good research practices for comparative effectiveness research: analytic methods to improve causal inference from nonrandomized studies of treatment effects using secondary data sources: The ISPOR good research practices for retrospective database analysis task force report—part III. Value Health. 2009;12:1062‐1073.
    1. Berger ML, Dreyer N, Anderson F, Towse A, Sedrakyan A, Normand SL. Prospective observational studies to assess comparative effectiveness: the ISPOR good research practices task force report. Value Health. 2012;15:217‐230.
    1. Garrison LP, Towse A, Briggs A, et al. Performance‐based risk‐sharing arrangements—good practices for design, implementation, and evaluation: report of the ISPOR good practices for performance‐based risk‐sharing arrangements task force, Value Health. 2013;16:703–709.
    1. Berger M, Martin B, Husereau D, et al. A questionnaire to assess the relevance and credibility of observational studies to inform healthcare decision making: an ISPOR‐AMCP‐NPC good practice task force report. Value Health. 2014;17(2):143‐156.
    1. Public Policy Committee . International Society for Pharmacoepidemiology. Guidelines for good pharmacoepidemiology practices (GPP). Pharmacoepidemiol Drug Saf. 2016;25:2‐10.
    1. U.S. FDA : Guidance for industry and FDA staff: Best practices for conducting and reporting pharmacoepidemiologic safety studies using electronic healthcare data. May 2013. URL:
    1. U. S. FDA : Guidance for industry: Good Pharmacovigilance Practices and Pharmacoepidemiologic Assessment. March 2005
    1. European Medicines Agency (EMA) : Guideline on good pharmacovigilance practices (GVP) module VIII—post‐authorisation safety studies (Rev 2). August 2016. URL:
    1. .
    1. European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP) . The European Union electronic register of post‐authorisation studies (EU PAS Register). (Accessed July 13, 2017).
    1. Ford I, Norrie J. Pragmatic clinical trials. NEJM. 2016;375:454‐463.
    1. Drummond M, Griffin A, Tarricone R. Economic evaluation for devices: Same or different? Value Health. 2009;12(4):402‐404.
    1. Wang SV, Brown J, Gagne JJ, Rassen JA, Klungel O, Douglas I, Smeeth L, Nguyen M, Sturkenboom M, Berger M, Eichler HG, Mullins D, Gini R, Schneeweiss S. Reporting to facilitate assessment of validity and improve reproducibility of healthcare database analyses 1.0. Working paper (May 3, 2017).
    1. Benchimol EI, Smeeth L, Guttmann A, et al., RECORD Working Committee . The REporting of studies Conducted using Observational Routinely‐collected health Data (RECORD) statement. PLoS Med. 2015.
    1. Vandenbroucke JP, von Elm E, Altman DG, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): Explanation and elaboration. PLoS Med. 2007;4(10):e297.
    1. (Accessed 4/29/2017)
    1. Tarricone R, Torbica A, Drummond M, Schreyögg J. Assessment of medical devices: Challenges and solutions. Health Econ. 2017;S1:1‐152. Accessed
    1. (Accessed 4/29/2017)
    1. Eichler HG, Baird LG, Barker R, et al. From adaptive licensing to adaptive pathways: Delivering a flexible life‐span approach to bring new drugs to patients. Clin Pharmacol Ther. 2015. Mar;97(3):234‐246.
    1. International conference on harmonisation of technical requirements for registration of pharmaceuticals for human use: ICH Harmonised Tripartite Guideline Statistical Principles for Clinical Trials E9." (PDF). ICH. 1998–02‐05.
    1. Wagenmakers EJ, Wetzels R, Borsboom D, van der Maas HL, Kievit RA. An agenda for purely confirmatory research. Perspect Psychol Sci. 2012. Nov;7(6):632‐638.
    1. Kimmelman J, Mogil JS, Dirangl U. Distinguishing between exploratory and confirmatory preclinical research will improve translation. PLoS Biol. 2014. May;12(5):e1001863.
    1. . . Accessed March 21, 2017. 32.HSRProj. . (Accessed March 21, 2017).
    1. HSRPoj . . (Accessed March 21, 2017).
    1. University hospital Medical Information Network–Clinical Trials Registry. (Accessed July 5, 2017).
    1. Blair, G. , Cooper, J. , Coppock, A. , & Humphreys, M. Declaring and diagnosing research designs. 2006. (Accessed July 5, 2017).
    1. Wang SV, Schneeweiss S. on behalf of the joint ISPE-ISPOR Special Task Force on Real World Evidence in Health Care Decision Making . Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies V1.0. Pharmacoepidemiol Drug Saf 2017;26(9):1018–1032.
    1. Chen AW, Tetzlaff JM, Altman DG, et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med. 2013;158(3):200‐207.
    1. Madigan D, Ryan P, Schuemie M, et al. Evaluating the impact of database heterogeneity on observational studies. Am J Epidemiol. 2013.
    1. Walker AM. Orthogonal predictions: Follow‐up questions for suggestive data. Pharmacoepidemiol Drug Saf. 2010;19(5):529‐532.
    1. Kirwan JR, de Wit M, Frank L, et al. Emerging guidelines for patient engagement in research. Value Health. 2017;20(3):481‐486.
    1. Selby JV, Forsydth LP, Sox HC. Stakeholder‐driven comparative effectiveness research: an update from PCORI. JAMA. 2015;314:2335‐2336.
    1. Forsythe LP, Frank LB, Workman TA, Hilliard T, Harwell D, Fayish L. Patient, caregiver and clinician views on engagement in comparative effectiveness research. J Comp Eff Res. 2017. Feb 8. . [Epub ahead of print] PubMed; 28173732
    1. Forsythe LP, Frank LB, Workman TA, Borsky A, Hilliard T, Harwell D, Fayish L. Health researcher views on comparative effectiveness research and research engagement. J Comp Eff Res. 2017. Feb 8. . [Epub ahead of print] PubMed PMID: 28173710.
    1. Frank LB, Forsythe LP, Workman TA, Hilliard T, Lavelle M, Harwell D, Fayish L. Patient, caregiver and clinician use of comparative effectiveness research findings in care decisions: Results from a national study. J Comp Eff Res. 2017. Feb 8. . [Epub ahead of print] PubMed PMID: 28173724.
    1. Hill AB. Observation and experiment. N Engl J Med. 1953;248:995‐1001.
    1. Loannidis JP. Why most published research findings are false. PLoS Med. 2005;2(8): e124
    1. Simmons JP, Nelson LD, Simonsohn U. False‐positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci. 2011;22(11):1359‐1366.

Source: PubMed

Sponsoren und Mitarbeiter

Krankheiten

Drogeninterventionen

3
Abonnieren