Regression Discontinuity Design to Evaluate of Drotrecogin Alpha Effectiveness

Regression Discontinuity Designs to Evaluate Real-world Effectiveness: a Case Study of Drotrecogin Alpha

Many health care interventions and medications found to have benefits ("efficacy") in experimental, tightly-controlled, human research trials are later found to lack real-world health benefits ("effectiveness"). Inadequate surveillance of real-world clinical effectiveness may falsely reassure clinicians and those who monitor healthcare quality, propagating unrecognized ineffective or harmful treatments at high costs to patients and society. The failure to translate potential health benefits into realized gains, or to detect unexpected harms in healthcare delivery, stems from a lack of methods with which to robustly measure real-world (in)effectiveness. Current methods to detect changes in outcomes 'before and after' implementation may be biased by secular trends in healthcare practice and outcomes; other methods to compare outcomes for treated and untreated patients may be biased by unmeasured factors.

The current project aims to develop and demonstrate - as a proof-of-concept - the use of a quasi-experimental research method called 'regression discontinuity design (RDD)' in surveillance of real-world clinical effectiveness. RDD had previously found use in the evaluation of educational programs in which students scoring below a threshold were assigned an intervention. The US Department of Education considers RDD designs to have quality similar to randomized trials.

Study Overview

• Status: Completed
• Conditions: Sepsis
• Intervention / Treatment: Drug: Drotrecogin alfa activated

Detailed Description

Many health care interventions and medications found to have benefits ("efficacy") in experimental, tightly-controlled, human research trials are later found to lack real-world health benefits ("effectiveness"). Inadequate surveillance of real-world clinical effectiveness may falsely reassure clinicians and those who monitor healthcare quality, propagating unrecognized ineffective or harmful treatments at high costs to patients and society. The failure to translate potential health benefits into realized gains, or to detect unexpected harms in healthcare delivery, stems from a lack of methods with which to robustly measure real-world (in)effectiveness. Current methods to detect changes in outcomes 'before and after' implementation may be biased by secular trends in healthcare practice and outcomes; other methods to compare outcomes for treated and untreated patients may be biased by unmeasured factors.

The current project aims to develop and demonstrate - as a proof-of-concept - the use of a quasi-experimental research method called 'regression discontinuity design (RDD)' in surveillance of real-world clinical effectiveness. RDD had previously found use in the evaluation of educational programs in which students scoring below a threshold were assigned an intervention. The US Department of Education considers RDD designs to have quality similar to randomized trials. However, RDD has not been rigorously evaluated in the context of evaluating clinical effectiveness. RDD can be used whenever an intervention is given to patients scoring above a threshold on a continuous biomarker or risk score. This scenario often arises in clinical practice, in which thresholds are used to identify and treat 'high risk' patients. In RDD, outcomes are compared for patients just above and just below the threshold, who are similar, but receive different treatments.

The project will study the use of RDD in evaluating the real-world effectiveness of drotrecogin alpha, a medication that was recommended by the FDA to be given to critically ill patients with severe sepsis at high risk for mortality (APACHE score > 25). Drotrecogin alpha was shown to potentially have "effectiveness" using traditional methods of real-world research, but was eventually shown to not be clinically efficacious in subsequent large randomized trials. The present proposal is a 'proof-of-concept' study that will allow evaluation of effect estimates derived from RDD methods to those of gold standard, pooled randomized trial results. The demonstration of feasibility for a new research method, such as RDD, to evaluate real-world clinical effectiveness would be a major leap forward in the ability monitor for potential real world benefits and harms of new treatments.

• Study Type: Observational
• Enrollment (Actual): 12492

Contacts and Locations

Study Locations

• United States
  • Massachusetts
    • Boston, Massachusetts, United States, 02118
      • Boston Medical Center

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

• Sampling Method: Non-Probability Sample

Study Population

An International Observational Study Among Severe Sepsis Patients Treated in the Intensive Care Unit (PROGRESS severe sepsis registry).

The PROGRESS registry of severe sepsis cases is a unique resource that contains data linking APACHE scores, drotrecogin alpha administration, and hospital mortality outcomes, providing an opportunity to study RDD as a novel method of comparative effectiveness research, and compare results derived from RDD to prior studies using more traditional comparative effectiveness designs (i.e., propensity score adjustment).

Description

Inclusion Criteria:

• Included in PROGRESS severe sepsis registry

Exclusion Criteria:

• None

Study Plan

How is the study designed?

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Relative risk reduction for mortality	The study will evaluate the change in the relationship between APACHE II scores and hospital mortality rate at the APACHE II threshold score of 25	Duration of hospitalization, on average approximately 14 days

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
APACHE II scores at which drotrecogin alpha was used	Evaluation of drotrecogin alpha practice patterns at APACHE II score of 25	Baseline
Change in mortality risk over time	The study will assess interaction between year and relative risk for mortality associated with drotrecogin alpha	3 years

Collaborators and Investigators

• Sponsor: Boston Medical Center

Publications and helpful links

General Publications

• Martin G, Brunkhorst FM, Janes JM, Reinhart K, Sundin DP, Garnett K, Beale R. The international PROGRESS registry of patients with severe sepsis: drotrecogin alfa (activated) use and patient outcomes. Crit Care. 2009;13(3):R103. doi: 10.1186/cc7936. Epub 2009 Jun 30.
• Beale R, Reinhart K, Brunkhorst FM, Dobb G, Levy M, Martin G, Martin C, Ramsey G, Silva E, Vallet B, Vincent JL, Janes JM, Sarwat S, Williams MD; PROGRESS Advisory Board. Promoting Global Research Excellence in Severe Sepsis (PROGRESS): lessons from an international sepsis registry. Infection. 2009 Jun;37(3):222-32. doi: 10.1007/s15010-008-8203-z. Epub 2009 Apr 28.
• Walkey AJ, Bor J. Risk-based Heterogeneity of Treatment Effect in Trials and Implications for Surveillance of Clinical Effectiveness Using Regression Discontinuity Designs. Am J Respir Crit Care Med. 2015 Dec 1;192(11):1399. doi: 10.1164/rccm.201508-1533LE. No abstract available.

Study record dates

Study Major Dates

• Study Start: December 1, 2002
• Primary Completion (Actual): December 1, 2005

Study Registration Dates

• First Submitted: July 15, 2016
• First Submitted That Met QC Criteria: July 21, 2016
• First Posted (Estimate): July 26, 2016

Study Record Updates

• Last Update Posted (Estimate): July 26, 2016
• Last Update Submitted That Met QC Criteria: July 21, 2016
• Last Verified: July 1, 2016

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Anti-Infective Agents; Drotrecogin alfa activated
• Other Study ID Numbers: H-35010

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO
• IPD Plan Description: Individual participant data (IPD) is only available through consultation with the original PROGRESS registry study sponsor through https://www.clinicalstudydatarequest.com/