A web application for the design of multi-arm clinical trials

Michael J Grayling, James Ms Wason, Michael J Grayling, James Ms Wason

Abstract

Background: Multi-arm designs provide an effective means of evaluating several treatments within the same clinical trial. Given the large number of treatments now available for testing in many disease areas, it has been argued that their utilisation should increase. However, for any given clinical trial there are numerous possible multi-arm designs that could be used, and choosing between them can be a difficult task. This task is complicated further by a lack of available easy-to-use software for designing multi-arm trials.

Results: To aid the wider implementation of multi-arm clinical trial designs, we have developed a web application for sample size calculation when using a variety of popular multiple comparison corrections. Furthermore, the application supports sample size calculation to control several varieties of power, as well as the determination of optimised arm-wise allocation ratios. It is built using the Shiny package in the R programming language, is free to access on any device with an internet browser, and requires no programming knowledge to use. It incorporates a variety of features to make it easier to use, including help boxes and warning messages. Using design parameters motivated by a recently completed phase II oncology trial, we demonstrate that the application can effectively determine and evaluate complex multi-arm trial designs.

Conclusions: The application provides the core information required by statisticians and clinicians to review the operating characteristics of a chosen multi-arm clinical trial design. The range of designs supported by the application is broader than other currently available software solutions. Its primary limitation, particularly from a regulatory agency point of view, is its lack of validation. However, we present an approach to efficiently confirming its results via simulation.

Keywords: False discovery rate; Familywise error-rate; Multiple comparisons; Optimal design; Power; Sample size.

Conflict of interest statement

The authors declare that they have no competing interests.

Figures

Fig. 1
Fig. 1
Design parameters box. The box in which input parameters are specified is shown. The specific values that can be seen are those that correspond to the trial described in [37]
Fig. 2
Fig. 2
Design summary box. The box in which a summary of the input parameters and of the identified design is rendered is shown. The specific output that can be seen corresponds to the inputs from Fig. 1
Fig. 3
Fig. 3
Operating characteristics summary. The boxes in which a summary of the identified designs operating characteristics is produced is shown. The specific output that can be seen corresponds to the inputs from Fig. 1
Fig. 4
Fig. 4
Operating characteristics plots. The boxes in which plots of the identified designs operating characteristics are produced is shown. The specific output that can be seen corresponds to the inputs from Fig. 1

References

    1. DiMasi JA, Grabowski HG, Hansen RW. Innovation in the pharmaceutical industry: new estimates of R&D costs. J Health Econ. 2016;47:20–33. doi: 10.1016/j.jhealeco.2016.01.012.
    1. Biotechnology Innovation Organization (BIO), Biomedtracker, AMPLION, Clinical development success rates 2006-2015 (2016).
    1. Parmar MKB, Carpenter J, Sydes MR. More multiarm randomised trials of superiority are needed. Lancet. 2014;384(9940):283–4. doi: 10.1016/S0140-6736(14)61122-3.
    1. Jaki T, Wason JMS. Multi-arm multi-stage trials can improve the efficiency of finding effective treatments for stroke: a case study. BMC Cardiovasc Disord. 2018;18(1):215. doi: 10.1186/s12872-018-0956-4.
    1. Wason JMS, Stecher L, Mander AP. Correcting for multiple-testing in multi-arm trials: is it necessary and is it done? Trials. 2014;15:364. doi: 10.1186/1745-6215-15-364.
    1. Baron G, Perrodeau E, Boutron I, Ravaud P. Reporting of analyses from randomized controlled trials with multiple arms: a systematic review. BMC Med. 2013;11:84. doi: 10.1186/1741-7015-11-84.
    1. Juszczak E, Altman DG, Hopewell S, Schulz K. Reporting of multi-arm parallel-group randomized trials: extension of the CONSORT 2010 statement. JAMA. 2019;321(16):1610–20. doi: 10.1001/jama.2019.3087.
    1. Rothman KJ. No adjustments are needed for multiple comparisons. Epidemiology. 1990;1(1):43–6. doi: 10.1097/00001648-199001000-00010.
    1. Cook RJ, Farewell VT. Multiplicity considerations in the design and analysis of clinical trials. J R Stat Soc Ser A. 1996;159(1):93–110. doi: 10.2307/2983471.
    1. Proschan MA, Waclawiw MA. Practical guidelines for multiplicity adjustment in clinical trials. Control Clin Trials. 2000;21(6):527–39. doi: 10.1016/S0197-2456(00)00106-9.
    1. Bender R, Lange S. Adjusting for multiple testing - when and how? J Clin Epidemiol. 2001;54(4):343–349. doi: 10.1016/S0895-4356(00)00314-0.
    1. Feise RJ. Do multiple outcome measures require p-value adjustment?, BMC Med Res Methodol. 2002;2:8. doi: 10.1186/1471-2288-2-8.
    1. Hughes MD. Multiplicity in clinical trials. Encycl Biostat. 2005;5:3446–51.
    1. Freidlin B, Korn EL, Gray R, Martin A. Multi-arm clinical trials of new agents: some design considerations. Clin Cancer Res. 2008;14:4368–4371. doi: 10.1158/1078-0432.CCR-08-0325.
    1. Li G, Taljaard M, Van den Heuvel ER, Levine MAH, Cook DJ, Wells GA, Devereaux PJ, Thabane L. An introduction to multiplicity issues in clinical trials: the what, why, when and how. Int J Epidemiol. 2016;46(2):746–55.
    1. EM Agency, Guideline on Multiplicity Issues in Clinical Trials (2017). . Accessed 17 Jan 2020.
    1. U. F. D. Administration, Multiple Endpoints in Clinical Trials Guidance for Industry (2017). . Accessed 17 Jan 2020.
    1. Howard DR, Brown JM, Todd S, Gregory WM. Recommendations on multiple testing adjustment in multi-arm trials with a shared control group. Stat Methods Med Res. 2018;27(5):1513–30. doi: 10.1177/0962280216664759.
    1. Hochberg Y, Tamhane AC. Multiple Comparison Procedures. New York: Wiley; 1987.
    1. Hsu JC. Multiple Comparisons. London: Chapman & Hall; 1996.
    1. Bretz F, Hothorn T, Westfall P. Multiple Comparisons using R. Boca Raton: CRC Press; 2010.
    1. Sankoh AJ, D’Agostino RBS, Huque MF. Efficacy endpoint selection and multiplicity adjustment methods in clinical trials with inherent multiple endpoint issues. Stat Med. 2003;22(20):3133–50. doi: 10.1002/sim.1557.
    1. Atkinson A, Donev A, Tobias R. Optimum Experimental Designs, with SAS. Oxford: Oxford University Press; 2007.
    1. W Chang, J Cheng, JJ Allaire, Y Xie, J McPherson, shiny: Web Application Framework for R (2019). . Accessed 17 Jan 2020.
    1. R Core Team . R: a Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing; 2018.
    1. MJ Grayling, multiarm: Design and analysis of fixed-sample multi-arm clinical trials (2019). . Accessed 17 Jan 2020.
    1. Bonferroni CE. Teoria statistica delle classi e calcolo delle probabilità. Pubblicazioni del R Istituto Superiore di Scienze Economiche e Commerciali di Firenze. 1936;8:3–62.
    1. Šidák Z. Rectangular confidence regions for the means of multivariate normal distributions. J Am Stat Assoc. 1967;62(318):626–33.
    1. Dunnett CW. A multiple comparison procedure for comparing several treatments with a control. J Am Stat Assoc. 1955;50(272):1096–121. doi: 10.1080/01621459.1955.10501294.
    1. Holm S. A simple sequentially rejective multiple test procedure. Scand J Stat. 1979;6(2):65–70.
    1. Hochberg Y. A sharper bonferroni procedure for multiple tests of significance. Biometrika. 1988;75(4):800–2. doi: 10.1093/biomet/75.4.800.
    1. Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Ser B. 1995;57(1):289–300.
    1. Benjamini Y, Yekutieli D. The control of the false discovery rate in multiple testing under dependency. Annals Stat. 1995;29(4):1165–88.
    1. Wason J, Magirr D, Law M, Jaki T. Some recommendations for multi-arm multi-stage trials. Stat Methods Med Res. 2016;25(2):716–27. doi: 10.1177/0962280212465498.
    1. Sverdlov O, Rosenberger WF. On recent advances in optimal allocation designs in clinical trials. J Stat Theory Pract. 2013;7(4):753–73. doi: 10.1080/15598608.2013.783726.
    1. A Genz, F Bretz, T Miwa, M X, L F, S F, H T, mvtnorm: Multivariate normal and t distributions. R package version 1.0-10 (2019). . Accessed 17 Jan 2020.
    1. Jacob L, M U, Boulet S, Begaj I, Chevret S. Evaluation of a multi-arm multi-stage Bayesian design for phase II drug selection trials - an example in hemato-oncology. BMC Med Res Methodol. 2016;16:67. doi: 10.1186/s12874-016-0166-7.
    1. CRAN Task View: Clinical Trial Design, Monitoring, and Analysis. . Accessed: 16 Oct 2019.
    1. Stata. . Accessed: 16 Oct 2019.
    1. SAS. . Accessed: 16 Oct 2019.
    1. nQuery. . Accessed: 16 Oct 2019.
    1. East. . Accessed: 04 May 2019.
    1. FACTS. . Accessed: 16 Oct 2019.
    1. PASS. . Accessed: 16 Oct 2019.
    1. Chow S, Wang H, Shao J. Sample Size Calculations in Clinical Research. Boca Raton: Chapman & Hall; 2008.
    1. Wheeler GM, Sweeting MJ, Mander AP. AplusB: A Web Application for Investigating A + B Designs for Phase I Cancer Clinical Trials. PLoS ONE. 2016;11(7):0159026. doi: 10.1371/journal.pone.0159026.
    1. Wages NA, Petroni GR. A web tool for designing and conducting phase I trials using the continual reassessment method. BMC Cancer. 2018;18:133. doi: 10.1186/s12885-018-4038-x.
    1. Magirr D, Jaki T, Whitehead J. A generalized Dunnett test for multi-arm multi-stage clinical studies with treatment selection. Biometrika. 2012;99(2):494–501. doi: 10.1093/biomet/ass002.
    1. Wason J, Stallard N, Bowden J, Jennison C. A multi-stage drop-the-losers design for multi-arm clinical trials. Stat Methods Med Res. 2017;26(1):508–24. doi: 10.1177/0962280214550759.
    1. Barthel FMS, Royston P, Parmar MKB. A menu-driven facility for sample-size calculation in novel multiarm, multistage randomized controlled trials with a time-to-event outcome. Stata J. 2009;9(4):505–23. doi: 10.1177/1536867X0900900401.
    1. Jaki T, Pallmann P, Magirr D. The R package MAMS for designing multi-arm multi-stage clinical trials. J Stat Softw. 2019;88(4):1–25. doi: 10.18637/jss.v088.i04.
    1. Dimairo M, Coates E, Pallmann P, Todd S, Julious SA, Jaki T, Wason J, Mander AP, Weir CJ, Koenig F, Walton MK, Biggs K, Nicholl J, Hamasaki T, Proschan MA, Scott JA, Ando Y, Hind D, Altman DG. Development process of a consensus-driven CONSORT extension for randomised trials using an adaptive design. BMC Med. 2018;16:210. doi: 10.1186/s12916-018-1196-2.

Source: PubMed

Sponsorit ja yhteistyökumppanit

Sairaudet

Huumeiden interventiot

3
Tilaa