A case study of SMART attributes: a qualitative assessment of generalizability, retention rate, and trial quality

Erica E M Moodie, James C Karran, Susan M Shortreed, Erica E M Moodie, James C Karran, Susan M Shortreed

Abstract

Background: Personalizing medical care is becoming increasingly popular, particularly mental health care. There is growing interest in formalizing medical decision making based on evolving patient symptoms in an evidence-based manner. To determine optimal sequencing of treatments, the sequences themselves must be studied; this may be accomplished by using a sequential multiple assignment randomized trial (SMART). It has been hypothesized that SMART studies may improve participant retention and generalizability.

Methods: We examine the hypotheses that SMART studies are more generalizable and have better retention than traditional randomized clinical trials via a case study of a SMART study of antipsychotic medications. We considered the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) schizophrenia study, comparing the trial participant characteristics and overall retention to those of comparable trials found via a review of all related trials conducted from 2000 onwards.

Results: A MEDLINE search returned 6435 results for primary screening; ultimately, 48 distinct trials were retained for analysis. The study population in CATIE was similar to, although perhaps less symptomatic than, the study populations of traditional randomized clinical trials (RCTs), suggesting no large gains in generalizability despite the pragmatic nature of the trial. However, CATIE did see good month-by-month retention.

Conclusions: SMARTs offer the possibility of studying treatment sequences in a way that a series of traditional RCTs cannot. SMARTs may offer improved retention; however, this case study did not find evidence to suggest greater generalizability using this trial design.

Trial registration: ClinicalTrials.gov NCT00014001 . Registered on 6 April 2001.

Keywords: Antipsychotic medication; Generalizability; Randomized trial; Retention; Schizophrenia; Sequential randomization.

Figures

Fig. 1
Fig. 1
A simplified schematic of the CATIE study design. Circled Rs represent randomization
Fig. 2
Fig. 2
Retention rate in CATIE and traditional RCTs of patients with schizophrenia by the planned length of follow-up of the trial. Circle radius is a function of the total number enrolled in the trial (for reference the CATIE trial enrolled 1460 participants). The black line indicates the follow-up rate by month in CATIE, with retention rate at the 18-month CATIE follow-up indicated by the black circle
Fig. 3
Fig. 3
Retention rate in CATIE and traditional RCTs of patients with schizophrenia versus mean PANSS score at study enrollment. Location of circle on horizontal axis indicates mean PANSS score at baseline, and circle radius is a function of the total number enrolled in the trial; circle color indicates the planned length of follow-up. The horizontal bars indicate +/- one standard deviation of PANSS scores at study enrollment. The CATIE study is represented by the black circle

References

    1. Almirall D, Lizotte D, Murphy SA. SMART design issues and the consideration of opposing outcomes: discussion of “Evaluation of Viable Dynamic Treatment Regimes in a Sequentially Randomized Trial of Advanced Prostate Cancer” by Wang et al. J Am Stat Assoc. 2012;107:509–12. doi: 10.1080/01621459.2012.665615.
    1. Barnes TR. A rating scale for drug-induced akathisia. Br J Psychiatry. 1989;154:672–6. doi: 10.1192/bjp.154.5.672.
    1. Collins L, Nahum-Shani I, Almirall D. Optimization of behavioral dynamic treatment regimens based on sequential, multiple assignment, randomized trial (SMART) Clin Trials. 2011;11:426–34. doi: 10.1177/1740774514536795.
    1. Grill JD, Raman R, Ernstrom K, Aisen P, Dowsett SA, Chen Y-F, et al. Comparing recruitment, retention, and safety reporting among geographic regions in multinational Alzheimer’s disease clinical trials. Alzheimers Res Ther. 2015;7:39. doi: 10.1186/s13195-015-0122-5.
    1. Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions. Chichester (West Sussex), England: Wiley; 2008.
    1. Lavori P, Dawson R. A design for testing clinical strategies: biased adaptive within-subject randomization. J R Stat Soc Ser A. 2000;163:29–38. doi: 10.1111/1467-985X.00154.
    1. Lei H, Nahum-Shani I, Lynch K, et al. A SMART design for building individualized treatment sequences. Annu Rev Clin Psychol. 2012;8:21–48. doi: 10.1146/annurev-clinpsy-032511-143152.
    1. MediciGlobal. Clinical trial retention meta-analysis. How patient recruitment methods directly relate to the retention of subjects (White paper). 2015.
    1. Messias E, Chen C-Y, Eaton WM. Epidemiology of schizophrenia: review of findings and myths. Psychiatr Clin North Am. 2007;30(3):323–38. doi: 10.1016/j.psc.2007.04.007.
    1. Müller MJ, Mller KM, Fellgiebel A. Detection of depression in acute schizophrenia: sensitivity and specificity of 2 standard observer rating scales. Canad J Psych Revue canadienne de psychiatrie. 2006;51(6):387–92.
    1. Murphy SA. Optimal dynamic treatment regimes (with discussions) J R Stat Soc Ser B. 2003;65:331–66. doi: 10.1111/1467-9868.00389.
    1. Murphy SA. A generalization error for Q-learning. J Mach Learn Res. 2005;6:1073–97.
    1. Oetting AI, Levy JA, Weiss RD, et al. Statistical methodology for a SMART design in the development of adaptive treatment strategies. In: Shrout P, Shrout P, Keyes K, et al., editors. Causality and psychopathology: finding the determinants of disorders and their cures. Arlington: American Psychiatric Publishing, Inc.; 2011. pp. 179–205.
    1. Robins JM. Optimal structural nested models for optimal sequential decisions. In: Lin DY, Heagerty PJ, editors. Proceedings of the Second Seattle Symposium on Biostatistics. New York: Springer; 2004. pp. 189–326.
    1. Shortreed SM, Moodie EEM. Estimating the optimal dynamic antipsychotic treatment regime: evidence from the sequential-multiple assignment randomized CATIE Schizophrenia Study. J R Stat Soc Ser B. 2012;61:577–99. doi: 10.1111/j.1467-9876.2012.01041.x.
    1. Stroup TS, McEvoy JP, Swartz MS, et al. The National Institute of Mental Health clinical antipsychotic trials of intervention effectiveness (CATIE) project: schizophrenia trial design and protocol development. Schizophr Bull. 2003;29(1):15–31. doi: 10.1093/oxfordjournals.schbul.a006986.
    1. Swartz MS, Perkins DO, Stroup TS, et al. Assessing clinical and functional outcomes in the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) schizophrenia trial. Schizophr Bull. 2003;29(1):33–43. doi: 10.1093/oxfordjournals.schbul.a006989.
    1. Zhang B, Tsiatis AA, Laber EB, et al. Robust method for estimating optimal treatment regimes. Biometrics. 2012;168:1010–8. doi: 10.1111/j.1541-0420.2012.01763.x.

Source: PubMed

スポンサーと協力者

医学的状態

薬物療法

3
購読する