Use of personalized Dynamic Treatment Regimes (DTRs) and Sequential Multiple Assignment Randomized Trials (SMARTs) in mental health studies

Ying Liu, Donglin Zeng, Yuanjia Wang, Ying Liu, Donglin Zeng, Yuanjia Wang

Abstract

Dynamic treatment regimens (DTRs) are sequential decision rules tailored at each point where a clinical decision is made based on each patient's time-varying characteristics and intermediate outcomes observed at earlier points in time. The complexity, patient heterogeneity, and chronicity of mental disorders call for learning optimal DTRs to dynamically adapt treatment to an individual's response over time. The Sequential Multiple Assignment Randomized Trial (SMARTs) design allows for estimating causal effects of DTRs. Modern statistical tools have been developed to optimize DTRs based on personalized variables and intermediate outcomes using rich data collected from SMARTs; these statistical methods can also be used to recommend tailoring variables for designing future SMART studies. This paper introduces DTRs and SMARTs using two examples in mental health studies, discusses two machine learning methods for estimating optimal DTR from SMARTs data, and demonstrates the performance of the statistical methods using simulated data.

Keywords: O-learning; Q-learning; SMART; double robust estimation; dynamic treatment regimes; personalized medicine.

Figures

Figure 1.. Design of adaptive reinforcement-based treatment…
Figure 1.. Design of adaptive reinforcement-based treatment for pregnant drug abusers
Figure 2.. Design of trial on adaptive…
Figure 2.. Design of trial on adaptive pharmacological and behavioral treatments for children with Attention Deficit/Hyperactivity Disorder (ADHD)

References

    1. Lavori PW, Dawson R. A design for testing clinical strategies: biased adaptive within-subject randomization. J R Stat Soc Ser A Stat Soc. 2000;163(1):29–38. doi: 10.1111/1467-985X.00154.
    1. Thall PF, Sung HG, Estey EH. Selecting therapeutic strategies based on efficacy and death in multicourse clinical trials. J Am Stat Assoc. 2002;97(457):29–39.
    1. Thall PF, Wathen JK. Covariate-adjusted adaptive randomization in a sarcoma trial with multi-stage treatments. Stat Med. 2005;24(13):1947–1964. doi: 10.1002/sim.2077.
    1. Lunceford JK, Davidian M, Tsiatis AA. Estimation of survival distributions of treatment policies in two-stage randomization designs in clinical trials. Biometrics. 2002;58(1):48–57. doi: 10.1111/j.0006-341X.2002.00048.x.
    1. Wahed AS, Tsiatis A A. Optimal estimator for the survival distribution and related quantities for treatment policies in two-stage randomization designs in clinical trials. Biometrics. 2004;60(1):124–133. doi: 10.1111/j.0006-341X.2004.00160.x.
    1. Wahed AS, Tsiatis AA. Semiparametric efficient estimation of survival distributions in two-stage randomisation designs in clinical trials with censored data. Biometrika. 2006;93(1):163–177.
    1. Rush AJ, Fava M, Wisniewski SR, Lavori PW, Trivedi MH, Sackeim HA, et al. Sequenced treatment alternatives to relieve depression (STAR*D): rationale and design. Controlled clin trials. 2004;25(1):119–142.
    1. Rush A, Trivedi M, Wisniewski S, Nierenberg A, Stewart J, Warden D, et al. Acute and longer-term outcomes in depressed outpatients requiring one or several treatment steps: a STAR*D report. Am J Psychiatry. 2006;163(11):1905–1917.
    1. Schneider LS, Ismail MS, Dagerman K, Davis S, Olin J, McManus D, et al. Clinical antipsychotic trials of intervention effectiveness (CATIE): Alzheimer’s disease trial. Schizophr bull. 2003;29(1):57. doi: 10.1093/schbul/sbj026.
    1. McKay J, Horn DV, Oslin D, Lynch K, Ivey M. A randomized trial of extended telephone-based continuing care for alcohol dependence: within-treatment substance use outcomes. J Consult Clin Psychol. 2010;78:912–923. doi: 10.1037/a0020700.
    1. Jones H, O’Grady K, Tuten M. Reinforcement-based treatment improves the maternal treatment and neonatal outcomes of pregnant patients enrolled in comprehensive care treatment. Am J Addict. 2011;20:196–204. doi: 10.1111/j.1521-0391.2011.00119.x.
    1. Pelham WE, Hoza B, Pillow DR, Gnagy EM, Kipp HL, Greiner AR, et al. Effects of methylphenidate and expectancy on children with ADHD: Behavior academic performance and attributions in a summer treatment program and regular classroom setting. J Consult Clin Psychol. 2002;70:320–335.
    1. Pelham W, Fabiano G A. Evidence-based psychosocial treatments for attention-deficit/hyperactivity disorder. J Clin Child Adolesc Psychol. 2008;37(1):184–214. doi: 10.1080/15374410701818681.
    1. Kasari C, Freeman S, Paparella T. Joint attention and symbolic play in young children with autism:a randomized controlled intervention study. J Child Psychol Psychiatry. 2006;47:611–620. doi: 10.1111/j.1469-7610.2005.01567.x.
    1. Lei H, Nahum-Shani I, Lynch K, Oslin D, Murphy S. (2012). A “SMART” design for building individualized treatment sequences. Annu Rev Clin Psychol. 2012;8(1):21–48. doi: 10.1146/annurev-clinpsy-032511-143152. Epub 2011 Dec 12.
    1. Murphy SA, Van Der Laan MJ, Robins JM. Marginal mean models for dynamic regimes. J Am Stat Assoc. 2001;96(456):1410–1423.
    1. Moodie EE, Richardson TS, Stephens DA. Demystifying optimal dynamic treatment regimes. Biometrics. 2007;63(2):447–455. doi: 10.1111/j.1541-0420.2006.00686.x.
    1. Rubin DB. Bayesian inference for causal effects: The role of randomization. Ann Stat. 1978;6:34–58.
    1. Berkane M. Latent variable modeling and applications to causality. New York: Springer New York. 1997
    1. Fabiano G, Pelham E, Waschbusch D, Gnagy M, Lahey B. A practical measure of impairment: psychometric properties of the impairment rating scale in samples of children withattention deficit hyperactivity disorder and two school-based samples. J Clin Child Adolesc Psychol. 2006;35:369–385.
    1. Pelham WE Jr, Gnagy EM, Greenslade KE, Milich R. Teacher ratings of DSM-III-R symptoms for the disruptive behavior disorders. J Am Acad Child Adolesc Psychiatry. 1992;31(2):210–218. doi: 10.1097/00004583-199203000-00006.
    1. Hernán Má, Brumback B, Robins JM. Marginal structural models to estimate the causal effect of zidovudine on the survival of HIV-positive men. Epidemiology. 2000;11(5):561–570.
    1. Nahum-Shani IM, Qian M, Almirall D. Experimental design and primary data analysis methods for comparing adaptive interventions. Psychological methods. 2012;17(4):457. doi: 10.1037/a0029372.
    1. Oetting AI, Levy JA, Weiss RD, Murphy SA. Statistical methodology for a SMART design in the development of adaptive treatment strategies. In: Causality and Psychopathology: Finding the Determinants of Disorders and their Cures Arlington. VA: American Psychiatric Publishing Inc. 2007
    1. Chakraborty B, Moodie EE. Statistical Methods for Dynamic Treatment Regimes. New York: Springer. 2013
    1. Watkins CJ. Learning from delayed rewards (Ph.D. dissertation) UK: University of Cambridge. 1989
    1. Murphy SA, Collins LM, Rush AJ. Customizing treatment to the patient: Adaptive treatment strategies. Drug Alcohol Depend. 2007;88(Suppl 2):S1–S3. doi: 10.1016/j.drugalcdep.2007.02.001.
    1. Zhao Y, Kosorok MR, Zeng D. Reinforcement learning design for cancer clinical trials. Stat Med. 2009;28(26):3294–3315. doi: 10.1002/sim.3720.
    1. Ertefaie A, Almirall DA, Huang L, Dziak JJ, Wagner AT, Murphy SA. SAS PROCQLEARN users’ guide (Version 1.0) University Park: The Methodology Center Penn State. 2012 Retrieved from: .
    1. Qian M, Murphy SA. Performance guarantees for individualized treatment rules. Ann Stat. 2011;39(2):1180.
    1. Zhao Y, Zeng D, Rush AJ, Kosorok MR. Estimating individualized treatment rules using outcome weighted learning. J Am Stat Assoc. 2012;107 (499):1106–1118. doi: 10.1080/01621459.2012.695674.
    1. Zhao Y, Zeng D, Laber E, Kosorok MR. New statistical learning methods for estimating optimal dynamic treatment regimes. J Am Stat Assoc. 2014 doi: 10.1080/01621459.2014.937488. in press.
    1. Zeng D, Wang Y, Liu Y, Kosorok M. Improved Outcome Weighted Learning for Dynamic Treatment Regimes. Massachusettes Boston: Joint Statistical Meetings. 2014
    1. Robins JM, Rotnitzky A, Zhao LP. Estimation of regression coefficients when some regressors are not always observed. J Am Stat Assoc. 1994;89:846–866.
    1. Shortreed SM, Laber E, Stroup TS, Pineau J. A multiple imputation strategy for sequential multiple assignment randomized trials. Stat Med. 2014;33(24):4202–4214. doi: 10.1002/sim.6223. Epub 11 Jun 2014.

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe