The impact of analytic method on interpretation of outcomes in longitudinal clinical trials

A Prakash, R C Risser, C H Mallinckrodt, A Prakash, R C Risser, C H Mallinckrodt

Abstract

Aims: Various analytical strategies for addressing missing data in clinical trials are utilised in reporting study results. The most commonly used analytical methods include the last observation carried forward (LOCF), observed case (OC) and the mixed model for repeated measures (MMRM). Each method requires certain assumptions regarding the characteristics of the missing data. If the assumptions for any particular method are not valid, results from that method can be biased. Results based on these different analytical methods can, therefore, be inconsistent, thereby making interpretation of clinical study results confusing. In this investigation, we compare results from MMRM, LOCF and OC in order to illustrate the potential biases and problems in interpretation.

Methods: Data from an 8-month, double-blind, randomised, placebo-controlled (placebo; n = 137), outpatient depression clinical trial comparing a serotonin-noradrenalin reuptake inhibitor (SNRI; n = 273) with a selective serotonin reuptake inhibitor (SSRI; n = 274) were used. The study visit schedule included efficacy and safety assessments weekly to week 4, bi-weekly to week 8, and then monthly. Visitwise mean changes for the 17-item Hamilton Depression Rating Scale (HAMD(17)) Maier subscale (primary efficacy outcome), blood pressure, and body weight were analysed using LOCF, MMRM and OC.

Results: Last observation carried forward consistently underestimated within-group mean changes in efficacy (benefit) and safety (risk) for both drugs compared with MMRM, whereas OC tended to overestimate within-group changes.

Conclusions: Inferences are based on between-group comparisons. Therefore, whether or not underestimating (overestimating) within-group changes was conservative or anticonservative depended on the relative magnitude of the bias in each treatment and on whether within-group changes represented improvement or worsening. Preference should be given in analytic plans to methods whose assumptions are more likely to be valid rather than relying on a method based on the hope that its results, if biased, will be conservative.

Trial registration: ClinicalTrials.gov NCT00073411.

Figures

Figure 1
Figure 1
Percentage of patients remaining at each time point during the 8-month study. Flexible dosing and rescue from placebo were available after week 8. Rescue from placebo to active drug was based on investigator decision and lack of response to placebo. Data from patients rescued from placebo to active drug were analysed separately and are not presented here. SNRI, serotonin-noradrenalin reuptake inhibitor; SSRI, selective serotonin reuptake inhibitor
Figure 2
Figure 2
Time-course of improvement on the HAMD17 Maier subscale by all the three analytical methods. Double-blind placebo rescue was available after week 8. *p ≤ 0.05 vs. placebo; §p ≤ 0.05 SSRI vs. SNRI. LOCF, last observation carried forward; OC, observed case; MMRM, mixed model for repeated measures; HAMD17, 17-item Hamilton Depression Rating Scale
Figure 3
Figure 3
Time-course of change in systolic blood pressure by all the three analytical methods. Double-blind placebo rescue was available after week 8. *p ≤ 0.05 vs. placebo; §p ≤ 0.05 SSRI vs. SNRI
Figure 4
Figure 4
Time-course of change in diastolic blood pressure by all the three analytical methods. Double-blind placebo rescue was available after week 8. *p ≤ 0.05 vs. placebo; §p ≤ 0.05 SSRI vs. SNRI
Figure 5
Figure 5
Time-course of change in body weight by all the three analytical methods. Double-blind placebo rescue was available after Week 8. *p ≤ 0.05 vs. placebo; §p ≤ 0.05 SSRI vs. SNRI

References

    1. Milliken GA, Johnson DE. The Analysis of Messy Data: Designed Experiments. Vol. 1. New York: Chapman & Hall; 1993.
    1. Gibbons RD, Hedeker D, Elkin I, et al. Some conceptual and statistical issues in analysis of longitudinal psychiatric data. Arch Gen Psychiatry. 1993;50:739–50.
    1. Laird NM. Missing data in longitudinal studies. Stat Med. 1988;7:305–15.
    1. Little R, Rubin D. Statistical Analysis with Missing Data. New York: John Wiley and Sons; 1987.
    1. Lavori PW. Clinical trials in psychiatry: should protocol deviation censor patient data? Neuropsychopharmacology. 1992;6:39–48.
    1. Verbeke G, Molenberghs G. Linear Mixed Models for Longitudinal Data. New York: Springer; 2000.
    1. Littell RC, Milliken GA, Stroup WW, Wolfinger RD. The SAS System for Mixed Models. Cary, NC: SAS Institute Inc.; 1996. pp. 1–10. ch.
    1. Cnaan A, Laird NM, Slasor P. Using the general linear mixed model to analyse unbalanced repeated measures and longitudinal data. Stat Med. 1997;16:2349–80.
    1. Lavori PW, Dawson R, Shera D. A multiple imputation strategy for clinical trials with truncation of patient data. Stat Med. 1995;14:1913–25.
    1. Siddiqui O, Ali MW. A comparison of the random-effects pattern mixture model with last-observation-carried-forward (LOCF) analysis in longitudinal clinical trials with dropouts. J Biopharm Stat. 1998;8:545–63.
    1. Heyting A, Tolboom JT, Essers JG. Statistical handling of drop-outs in longitudinal clinical trials. Stat Med. 1992;11:2043–61.
    1. Mallinckrodt CH, Clark WS, David SR. Accounting for dropout bias using mixed-effects models. J Biopharm Stat. 2001;11:9–21.
    1. Mallinckrodt CH, Clark WS, David SR. Type I error rates from mixed effects model repeated measures versus with fixed effects ANOVA with missing values imputed via last observation carried forward. Drug Inf J. 2001;35:1215–25.
    1. Rubin DB, Shenker N. Multiple imputation in health-care databases: an overview and some applications. Stat Med. 1991;10:585–98.
    1. Little R, Yau L. Intent-to-treat analysis for longitudinal studies with drop-outs. Biometrics. 1996;52:1324–33.
    1. Rubin DB, Stern HS, Vehovar V. Handling “don't know” survey responses: the case of the Slovenian plebiscite. J Am Stat Assoc. 1995;90:822–8.
    1. Diggle PJ, Liang K–Y, Zeger SL. Analysis of Longitudinal Data. Oxford: Clarendon Press; 1994.
    1. Diggle PJ, Kenward MG. Informative dropout in longitudinal data analysis (with discussion) Appl Stat. 1994;43:49–93.
    1. Molenberghs G, Thijs H, Jansen I, et al. Analyzing incomplete longitudinal clinical trial data. Biostatistics. 2004;5:445–64.
    1. Robins JM, Rotnitsky A, Scharfstein DO. Semiparametric regression for repeated outcomes with non-ignorable non-response. J Am Stat Assoc. 1998;93:1321–39.
    1. Mallinckrodt CH, Kaiser CJ, Watkin JG, Detke MJ, Molenberghs G, Carroll RJ. Type I error rates from likelihood-based repeated measures analyses of incomplete longitudinal data. Pharm Stat. 2004;3:171–86.
    1. Mallinckrodt CH, Kaiser CJ, Watkin JG, Molenberghs G, Carroll RJ. The effect of correlation structure on treatment contrasts estimated from incomplete clinical trial data with likelihood-based repeated measures compared with last observation carried forward ANOVA. Clin Trials. 2004;1:477–89.
    1. Laird NM, Ware JH. Random-effects models for longitudinal data. Biometrics. 1982;38:963–74.
    1. Mallinckrodt CH, Clark WS, Carroll RJ, Molenbergh G. Assessing response profiles from incomplete longitudinal clinical trial data under regulatory considerations. J Biopharm Stat. 2003;13:179–90.
    1. Liu G, Gould AL. Comparison of alternative strategies for analysis of longitudinal trials with dropouts. J Biopharm Stat. 2002;12:207–2.
    1. Gadbury GL, Coffey CS, Allison DB. Modern statistical methods for handling missing repeated measurements in obesity trials: beyond LOCF. Obes Rev. 2003;4:175–84.
    1. Shao J, Zhong B. Last observation carry-forward and last observation analysis. Stat Med. 2003;22:2429–41.
    1. Mallinckrodt CH, Raskin J, Wohlreich MM, Watkin JG, Detke MJ. The efficacy of duloxetine: a comprehensive summary of results from MMRM and LOCF_ANCOVA in eight clinical trials. BMC Psychiatry. 2004;4:26.
    1. Barnes SA, Carter K, Lindborg S, Mallinckrodt C. The impact of missing data and how it is handled on the rate of false positive results in drug development. Pharm Stat. 2007 in press.
    1. Lane PW. Handling drop-out in longitudinal clinical trials: a comparison of the LOCF and MMRM approaches. Pharm Stat. 2008;7:93–106.
    1. Gueorguieva R, Krystal JH. Move over ANOVA: progress in analyzing repeated measures data and its reflection in papers published in the Archives of General Psychiatry. Arch Gen Psychiatry. 2004;61:310–7.
    1. Mallinckrodt CH, Lane PW, Schnell D, Peng Y, Mancuso JP. Recommendations for the primary analysis of continuous endpoints in longitudinal clinical trials. Drug Inf J. 2008 in press.
    1. Mallinckrodt CH, Watkin JG, Molenberghs G, Carroll RJ. Choice of the primary analysis in longitudinal clinical trials. Pharm Stat. 2004;3:161–9.
    1. Lieberman JA, Greenhouse J, Hamer RM, et al. Comparing the effects of antidepressants: consensus guidelines for evaluating quantitative reviews of antidepressant efficacy. Neuropsychopharmacology. 2005;30:445–60.
    1. Leon AC, Mallinckrodt CH, Chuang-Stein C, Archibald DG, Archer GE, Chartier K. Attrition in randomized controlled clinical trials: methodological issues in psychopharmacology. Biol Psychiatry. 2006;59:1001–5.
    1. Molenberghs G, Kenward MG. Missing Data in Clinical Studies. Chichester: John Wiley & Sons; 2007.
    1. Nierenberg AA, Greist JH, Mallinckrodt CH, et al. Duloxetine versus escitalopram and placebo in the treatment of patients with major depressive disorder: onset of antidepressant action, a noninferiority study. Curr Med Res Opin. 2007;23:401–16.
    1. Pigott TA, Prakash A, Arnold LM, Aaronson ST, Mallinckrodt CH, Wohlreich MM. Duloxetine versus escitalopram and placebo: an 8-month, double-blind trial in patients with major depressive disorder. Curr Med Res Opin. 2007;23:1303–18.
    1. American Psychiatric Association . Diagnostic and Statistical Manual of Mental Disorders. 4th edn. Washington: APA; 1994.
    1. Montgomery SA, Asberg M. A new depression scale designed to be sensitive to change. Br J Psychiatry. 1979;134:382–9.
    1. Guy W. ECDEU Assessment Manual for Psychopharmacology. Rockville: National Institutes of Mental Health; 1976. revised (1976)
    1. Maier W, Philipp M. Improving the assessment of severity of depressive states: a reduction of the Hamilton Depression Scale. Pharmacopsychiatry. 1985;18:114–5.
    1. Khan A, Schwartz K, Redding N, Kolts RL, Brown WA. Psychiatric diagnosis and clinical trial completion rates: analysis of the FDA SBA reports. Neuropsychopharmacology. 2007;32:2422–30.
    1. Mallinckrodt CH, Lane PW, Schnell D, Peng Y, Mancuso JP. Recommendations for the primary analysis of continuous endpoints in longitudinal clinical trials. Drug Inf J. 2008 in press.
    1. Mallinckrodt C, Chuang-Stein C, McSorley P, et al. A case study comparing a randomized withdrawal trial and a double-blind long-term trial for assessing the long-term efficacy of an antidepressant. Pharm Stat. 2007;6:9–22.
    1. ICH Guidelines General Considerations for Clinical Trials E8. .
    1. Wade A, Gembert K, Florea I. A comparative study of the efficacy of acute and continuation treatment with escitalopram versus duloxetine in patients with major depressive disorder. Curr Med Res Opin. 2007;23:1605–14.
    1. Khan A, Bose A, Alexopoulos GS, Gommoll C, Li D, Gandhi C. Double-blind comparison of escitalopram and duloxetine in the acute treatment of major depressive disorder. Clin Drug Investig. 2007;27:481–92.

Source: PubMed

Sponsorer og samarbeidspartnere

Medisinsk tilstand

Legemiddelintervensjoner

3
Abonnere