Remote, real-time expert elicitation to determine the prior probability distribution for Bayesian sample size determination in international randomised controlled trials: Bronchiolitis in Infants Placebo Versus Epinephrine and Dexamethasone (BIPED) study

Jingxian Lan, Amy C Plint, Stuart R Dalziel, Terry P Klassen, Martin Offringa, Anna Heath, Pediatric Emergency Research Canada (PERC) KIDSCAN/PREDICT BIPED Study Group, Jingxian Lan, Amy C Plint, Stuart R Dalziel, Terry P Klassen, Martin Offringa, Anna Heath, Pediatric Emergency Research Canada (PERC) KIDSCAN/PREDICT BIPED Study Group

Abstract

Background: Bayesian methods are increasing in popularity in clinical research. The design of Bayesian clinical trials requires a prior distribution, which can be elicited from experts. In diseases with international differences in management, the elicitation exercise should recruit internationally, making a face-to-face elicitation session expensive and more logistically challenging. Thus, we used a remote, real-time elicitation exercise to construct prior distributions. These elicited distributions were then used to determine the sample size of the Bronchiolitis in Infants with Placebo Versus Epinephrine and Dexamethasone (BIPED) study, an international randomised controlled trial in the Pediatric Emergency Research Network (PERN). The BIPED study aims to determine whether the combination of epinephrine and dexamethasone, compared to placebo, is effective in reducing hospital admission for infants presenting with bronchiolitis to the emergency department.

Methods: We developed a Web-based tool to support the elicitation of the probability of hospitalisation for infants with bronchiolitis. Experts participated in online workshops to specify their individual prior distributions, which were aggregated using the equal-weighted linear pooling method. Experts were then invited to provide their comments on the aggregated distribution. The average length criterion determined the BIPED sample size.

Results: Fifteen paediatric emergency medicine clinicians from Canada, the USA, Australia and New Zealand participated in three workshops to provide their elicited prior distributions. The mean elicited probability of admission for infants with bronchiolitis was slightly lower for those receiving epinephrine and dexamethasone compared to supportive care in the aggregate distribution. There were substantial differences in the individual beliefs but limited differences between North America and Australasia. From this aggregate distribution, a sample size of 410 patients per arm results in an average 95% credible interval length of less than 9% and a relative predictive power of 90%.

Conclusion: Remote, real-time expert elicitation is a feasible, useful and practical tool to determine a prior distribution for international randomised controlled trials. Bayesian methods can then determine the trial sample size using these elicited prior distributions. The ease and low cost of remote expert elicitation mean that this approach is suitable for future international randomised controlled trials.

Trial registration: ClinicalTrials.gov NCT03567473.

Keywords: Bayesian statistics; Expert elicitation; Prior probability distribution; Randomised controlled trials; Sample size determination; Trial design.

Conflict of interest statement

The authors declare that they have no competing interests. The authors alone are responsible for the writing and content of this article.

© 2022. The Author(s).

Figures

Fig. 1
Fig. 1
Individual-level elicited prior distributions for hospitalisation probability under (a) supportive care (left) or (b) treatment with the combination of epinephrine and dexamethasone (EpiDex, right). Each line depicts the distribution scored by an individual participant (n = 15). Distributions for first elicitation round on top; second round at bottom
Fig. 2
Fig. 2
Pooled elicited prior distributions for hospitalisation probability under (a) supportive care (supportive, solid black line) or (b) treatment with the combination of epinephrine and dexamethasone (EpiDex, dashed red line). Distributions for first elicitation round top; second round bottom
Fig. 3
Fig. 3
The width of the average 95% posterior credible interval length for “admission probability difference” between placebo and the combination of epinephrine and dexamethasone EpiDex plotted across the BIPED clinical trial sample sizes increasing between 400 and 630 in increments of 5 (solid black line). Average length criterion (ALC) thresholds of 0.09 and 0.08 are plotted as dashed black lines (see text)

References

    1. van de Schoot R, Depaoli S, King R, Kramer B, Märtens K, Tadesse M, et al. Bayesian statistics and modelling. Nat Rev Methods Primers. 2021;1(1). 10.1038/s43586-020-00001-2.
    1. Berry D. Bayesian clinical trials. Nat Rev Drug Discov. 2006;5(1):27–36.
    1. Spiegelhalter D, Freedman L, Parmar M. Bayesian approaches to randomized trials. J R Stat Soc A (Statistics in Society) 1994;157(3):357–387.
    1. Jack Lee J, Chu C. Bayesian clinical trials in action. Stat Med. 2012;31(25):2955–2972.
    1. Berry S, Carlin B, Lee J, Muller P. Bayesian adaptive methods for clinical trials. Boca Raton: CRC Press; 2010.
    1. Adcock C. Sample size determination: a review. J R Stat Soc D (The Statistician). 46(2):261–283, 2997.
    1. Cook J, Julious S, Sones W, Hampson L, Hewitt C, et al. DELTA2 guidance on choosing the target difference and undertaking and reporting the sample size calculation for a randomised controlled trial. BMJ. 2018;363:k3750.
    1. Joseph L, Du Berger R, Bélisle P. Bayesian and mixed Bayesian/likelihood criteria for sample size determination. Stat Med. 1997;16(7):769–781.
    1. Chaloner K, Rhame F. Quantifying and documenting prior beliefs in clinical trials. Stat Med. 2001;20(4):581–600.
    1. Hampson L, Whitehead J, Eleftheriou D, Tudur-Smith C, Jones R, Jayne D, Hickey H, Beresford M, Bracaglia C, Caldas A, Cimaz R. Elicitation of expert prior opinion: application to the MYPAN trial in childhood polyarteritis nodosa. PLoS One. 2015;10(3):e0120981.
    1. Choy S, O'Leary R, Mengersen K. Elicitation by design in ecology: using expert opinion to inform priors for Bayesian statistical model. Ecology. 2009;90(1):265–277.
    1. Cao J, Lee J, Alber S. Comparison of Bayesian sample size criteria: ACC, ALC, and WOC. J Stat Plan Inference. 2009;139(12):4111–4122.
    1. De Santis F. Using historical data for Bayesian sample size determination. J R Stat Soc A (Statistics in Society) 2007;170(1):95–113.
    1. Dias L, Morton A, Quigley J. Elicitation: the science and art of structuring judgement. Cham: Springer; 2018.
    1. Estévez R, Mardones F, Álamos F, Arriagada G, Carey J, Correa C, Escobar-Dodero J, Gaete Á, Gallardo A, Ibarra R, Ortiz C. Eliciting expert judgements to estimate risk and protective factors for Piscirickettsiosis in Chilean salmon farming. Aquaculture. 2019;507:402–410.
    1. Azzolina D, Berchialla P, Gregori D, Baldi I. Prior elicitation for use in clinical trial design and analysis: a literature review. Int J Environ Res Public Health. 2021;18(4):1833.
    1. Jansen J, Wang H, Holcomb J, Harvin J, Richman J, Avritscher E, Stephens S, Truong V, Marques M, DeSantis S, Yamal J. Elicitation of prior probability distributions for a proposed Bayesian randomized clinical trial of whole blood for trauma resuscitation. Transfusion. 2020;60(3):498–506.
    1. See C, Srinivasan M, Saravanan S, Oldenburg C, Esterberg E, Ray K, Glaser T, Tu E, Zegans M, McLeod S, Acharya N. Prior elicitation and Bayesian analysis of the steroids for corneal ulcers trial. Ophthalmic Epidemiol. 2012;19(6):407–413.
    1. Ramanan A, Hampson L, Lythgoe H, Jones A, Hardwick B, Hind H, Jacobs B, Vasileiou D, Wadsworth I, Ambrose N, Davidson J. Defining consensus opinion to develop randomised controlled trials in rare diseases using Bayesian design: an example of a proposed trial of adalimumab versus pamidronate for children with CNO/CRMO. PLoS One. 2019;14(6):e0215739.
    1. Browne E, Rathinam S, Kanakath A, Thundikandy R, Babu M, Lietman T, Acharya N. A Bayesian analysis of a randomized clinical trial comparing antimetabolite therapies for non-infectious uveitis. Ophthalmic Epidemiol. 2017;24(1):63–70.
    1. McBride M, Garnett S, Szabo J, Burbidge A, Butchart S, Christidis L, Dutson G, Ford H, Loyn R, Watson D, Burgman M. Structured elicitation of expert judgments for threatened species assessment: a case study. Methods Ecol Evol. 2012;3(5):906–920.
    1. Grigore B, Peters J, Hyde C, Stein K. EXPLICIT: a feasibility study of remote expert elicitation in health technology assessment. BMC Med Inform Decis Making. 2017;17(1):1–10.
    1. Aspinall W, Cooke R, Havelaar A, Hoffmann S, Hald T. Evaluation of a performance-based expert elicitation: WHO global attribution of foodborne diseases. PLoS One. 2016;11(3):e0149817.
    1. Gosling J. SHELF: the Sheffield elicitation framework, in Elicitation. Cham: Springer; 2021. pp. 61–93.
    1. A. O'Hagan and J. Oakley, SHELF: the Sheffield elicitation framework. Sheffield, URL: . Accessed 7 Jul 2021.
    1. Schuh S, Babl F, Dalziel S, Freedman S, Macias C, Stephens D, Steele D, Fernandes R, Zemek R, Plint A, Florin T. Practice variation in acute bronchiolitis: a pediatric emergency research networks study. Pediatrics. 2017;140(6):e20170842.
    1. Babl F, Borland M, Ngo P, Acworth J, Krieser D, Pandit S, Pitt W, Cotterell E, Jamison S. Paediatric Research in Emergency Departments International Collaborative (PREDICT): first steps towards the development of an Australian and New Zealand research network. Emerg Med Australas. 2006;18(2):143–147.
    1. Ricci V, Nunes V, Murphy M, Cunningham S. Bronchiolitis in children: summary of NICE guidance. BMJ. 2015;350:h2305.
    1. Ralston S, Lieberthal A, Meissner H, Alverson B, Baley J, Gadomski A, Johnson D, Light M, Maraqa N, Mendonca E, Phelan K. Clinical practice guideline: the diagnosis, management, and prevention of bronchiolitis. Pediatrics. 2014;134(5):e1474–e1502.
    1. Friedman J, Rieder, J M. Walton and Canadian Paediatric Society, Acute Care Committee, Bronchiolitis: recommendations for diagnosis, monitoring and management of children one to 24 months of age. Paediatr Child Health. 2014;19(9):485–491.
    1. Baumer J. SIGN guideline on bronchiolitis in infants. Arch Dis Child Educ Pract. 2007;92(5):ep149–ep151.
    1. Quinonez R, Schroeder A. Safely doing less and the new AAP bronchiolitis guideline. Pediatrics. 2015;135(5):793–795.
    1. Plint A, Johnson D, Patel H, Wiebe N, Correll R, Brant R, Mitton C, Gouin S, Bhatt M, Joubert G, Black K. Epinephrine and dexamethasone in children with bronchiolitis. N Engl J Med. 2009;360(20):2079–2089.
    1. Klassen T, Dalziel S, Babl F, Benito J, Bressan S, Chamberlain J, et al. The Pediatric Emergency Research Network (PERN): a decade of global research cooperation in paediatric emergency care. Emerg Med Australas. 2021. 10.1111/1742-6723.13801.
    1. Hartling L, Fernandes R, Bialy L, Milne A, Johnson D, Plint A, Klassen T, Vandermeer B. Steroids and bronchodilators for acute bronchiolitis in the first two years of life: systematic review and meta-analysis. BMJ. 2011;342:d1714.
    1. Wainwright C. Acute viral bronchiolitis in children-a very common condition with few therapeutic options. Paediatr Respir Rev. 2010;11(1):39–45.
    1. Bolstad W, Curran J. Introduction to Bayesian statistics. Hoboken: Wiley; 2016.
    1. Garthwaite P, Kadane J, O'Hagan A. Statistical methods for eliciting probability distributions. J Am Stat Assoc. 2005;100(470):680–701.
    1. Hemming V, Burgman M, Hanea A, McBride M, Wintle B. A practical guide to structured expert elicitation using the IDEA protocol. Methods Ecol Evol. 2018;9(1):169–180.
    1. Morris D, Oakley J, Crowe J. A web-based tool for eliciting probability distributions from experts. Environ Model Softw. 2014;52:1–4.
    1. Mason A, Gomes M, Grieve R, Ulug P, Powell J, Carpenter J. Development of a practical approach to expert elicitation for randomised controlled trials with missing health outcomes: application to the IMPROVE trial. Clin Trials. 2017;14(4):357–367.
    1. W. Chang, J. Cheng, J. Allaire, S. C., B. Schloerke, Y. Xie, J. Allen, J. McPherson, A. Dipert and B. Barbara Borges, shiny: Web application framework for R. R package version 1.6.0., , Vienna, Austria, 2021.
    1. R Core Team . R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2020.
    1. Devleesschauwer B, Torgerson P, Charlier J, Levecke B, Praet N, Roelandt S, Smit S, Dorny P, Berkvens D, Speybroeck N. prevalence: tools for prevalence assessment studies. 2014.
    1. Harris P, Taylor R, Minor B, Elliott V, Fernandez M, O'Neal L, McLeod L, Delacqua G, Delacqua F, Kirby J, Duda S. The REDCap consortium: building an international community of software platform partners. J Biomed Inform. 2019;95:103208.
    1. Harris P, Taylor R, Thielke R, Payne J, Gonzalez N, Conde J. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–381.
    1. European Food Safety Authority Guidance on expert knowledge elicitation in food and feed safety risk assessment. EFSA J. 2014;12(6):3734.
    1. Dietrich F, List C. The Oxford handbook of probability and philosophy. Oxford: Oxford University Press; 2016. Probabilistic opinion pooling; pp. 1–27.
    1. Johnson S, Tomlinson G, Hawker G, Granton J, Feldman B. Methods to elicit beliefs for Bayesian priors: a systematic review. J Clin Epidemiol. 2010;63(4):355–369.
    1. Williams C, Wilson K, Wilson N. A comparison of prior elicitation aggregation using the classical method and SHELF. J R Stat Soc A (Statistics in Society) 2021;184:920–940.
    1. Clemen R. Comment on Cooke’s classical method. Reliab Eng Syst Saf. 2008;93(5):760–765.
    1. Corneli H, Zorc J, Mahajan P, Shaw K, Holubkov R, Reeves S, Ruddy R, Malik B, Nelson K, Bregstein J, Brown K. A multicenter, randomized, controlled trial of dexamethasone for bronchiolitis. N Engl J Med. 2007;357(4):331–339.
    1. Florin T, Plint A, Zorc J. Viral bronchiolitis. Lancet. 2017;389(10065):211–224.
    1. Kirolos A, Manti S, Blacow R, Tse G, Wilson T, Lister M, Cunningham S, Campbell A, Nair H, Reeves R, Fernandes R. A systematic review of clinical practice guidelines for the diagnosis and management of bronchiolitis. J Infect Dis. 2020;222(7):S672–S679.
    1. Zoom Video Communications, Zoom Cloud Meetings, URL: . Accessed 9 Aug 2021, 2021.
    1. Joseph L, M'Lan C, Wolfson D. Bayesian sample size determination for binomial proportions. Bayesian Anal. 2008;3(2):269–296.
    1. Gilks W, Richardson S, Spiegelhalter D. Markov chain Monte Carlo in practice. Boca Raton, FL: CRC press; 1995.
    1. Plummer M. Proceedings of the 3rd international workshop on distributed statistical computing. 2003. JAGS: a program for analysis of Bayesian graphical models using Gibbs sampling; pp. 1–10.
    1. Cooke R, Mendel M, Thijs W. Calibration and information in expert resolution; a classical approach. Automatica. 1988;24(1):87–93.
    1. Werner C, Hanea A, Morales-Nápoles O. Eliciting multivariate uncertainty from experts: considerations and approaches along the expert judgement process, in Elicitation. Cham: Springer; 2018. pp. 171–210.

Source: PubMed

Patrocinadores e Colaboradores

Condições médicas

Intervenções de drogas

3
Se inscrever