Sample size estimation in clinical trials using ventilator-free days as the primary outcome: a systematic review

Laurent Renard Triché, Emmanuel Futier, Manuela De Carvalho, Nathalie Piñol-Domenech, Laëtitia Bodet-Contentin, Matthieu Jabaudon, Bruno Pereira, Laurent Renard Triché, Emmanuel Futier, Manuela De Carvalho, Nathalie Piñol-Domenech, Laëtitia Bodet-Contentin, Matthieu Jabaudon, Bruno Pereira

Abstract

Background: Ventilator-free days (VFDs) are a composite endpoint increasingly used as the primary outcome in critical care trials. However, because of the skewed distribution and competitive risk between components, sample size estimation remains challenging. This systematic review was conducted to systematically assess whether the sample size was congruent, as calculated to evaluate VFDs in trials, with VFDs' distribution and the impact of alternative methods on sample size estimation.

Methods: A systematic literature search was conducted within the PubMed and Embase databases for randomized clinical trials in adults with VFDs as the primary outcome until December 2021. We focused on peer-reviewed journals with 2021 impact factors greater than five. After reviewing definitions of VFDs, we extracted the sample size and methods used for its estimation. The data were collected by two independent investigators and recorded in a standardized, pilot-tested forms tool. Sample sizes were calculated using alternative statistical approaches, and risks of bias were assessed with the Cochrane risk-of-bias tool.

Results: Of the 26 clinical trials included, 19 (73%) raised "some concerns" when assessing risks of bias. Twenty-four (92%) trials were two-arm superiority trials, and 23 (89%) were conducted at multiple sites. Almost all the trials (96%) were unable to consider the unique distribution of VFDs and death as a competitive risk. Moreover, significant heterogeneity was found in the definitions of VFDs, especially regarding varying start time and type of respiratory support. Methods for sample size estimation were also heterogeneous, and simple models, such as the Mann-Whitney-Wilcoxon rank-sum test, were used in 14 (54%) trials. Finally, the sample sizes calculated varied by a factor of 1.6 to 17.4.

Conclusions: A standardized definition and methodology for VFDs, including the use of a core outcome set, seems to be required. Indeed, this could facilitate the interpretation of findings in clinical trials, as well as their construction, especially the sample size estimation which is a trade-off between cost, ethics, and statistical power. Systematic review registration PROSPERO ID: CRD42021282304. Registered 15 December 2021 ( https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021282304 ).

Keywords: Respiration, artificial; Respiratory insufficiency; Time factors; Treatment outcome; Ventilators, mechanical.

Conflict of interest statement

The authors declare that they have no competing interests.

© 2023. The Author(s).

Figures

Fig. 1
Fig. 1
Sample size estimation as reported in each trial and computed according to different alternative tests. For each study, sample size estimation is plotted (in blue) against the highest value among the sample size estimated in the study and five different tests: the Student t-test, the Mann–Whitney–Wilcoxon rank-sum test, the Mann–Whitney–Wilcoxon rank-sum test using the Noether formula, Cox regression and zero-inflated negative binomial (ZINB) regression. When an estimation is missing, the whole length of the line is gray. The estimation was only possible for the following studies: Mackle [25]; Villar [26]; Zhou [27]; Trouillet [28]; Simonis [29]; Algera [30]; Tomazini [31]; Grieco [32]; Spragg [34]; Welte [35]; Rice_1 [36]; Chung [38]; Rice_2 [39]; Bein [40]; Kacmarek [42]; Liu [44]; Rice_3 [45]; Matthay [46]; Bennett [47]; and McAuley [48]
Fig. 2
Fig. 2
Distribution of ventilator-free days (VFDs) in selected trials. Histograms representing a a Gaussian distribution (mean of 11.7 days, standard deviation of 10.5) used in some studies for sample size estimation and b the empirical distribution of VFDs (mean of 11.7 days, standard deviation of 10.71 and median of 12.23, interquartile range 0.00–22.00) found in Jabaudon et al.’s meta-analysis [50]. The red bars correspond to the theoretical data that should be seen if the distribution were normal

References

    1. Marshall DC, Hatch RA, Gerry S, Young JD, Watkinson P. Conditional survival with increasing duration of ICU admission: an observational study of three intensive care databases. Crit Care Med. 2020;48:91–97. doi: 10.1097/CCM.0000000000004082.
    1. Hill AD, Fowler RA, Burns KEA, Rose L, Pinto RL, Scales DC. Long-term outcomes and health care utilization after prolonged mechanical ventilation. Ann Am Thorac Soc. 2017;14:355–362. doi: 10.1513/AnnalsATS.201610-792OC.
    1. Herridge MS, Chu LM, Matte A, Tomlinson G, Chan L, Thomas C, et al. The RECOVER Program: disability risk groups and 1-year outcome after 7 or more days of mechanical ventilation. Am J Respir Crit Care Med. 2016;194:831–844. doi: 10.1164/rccm.201512-2343OC.
    1. Veldhoen RA, Howes D, Maslove DM. Is mortality a useful primary end point for critical care trials? Chest. 2020;158:206–211. doi: 10.1016/j.chest.2019.11.019.
    1. Spragg RG, Bernard GR, Checkley W, Curtis JR, Gajic O, Guyatt G, et al. Beyond mortality: future clinical research in acute lung injury. Am J Respir Crit Care Med. 2010;181:1121–1127. doi: 10.1164/rccm.201001-0024WS.
    1. Harhay MO, Wagner J, Ratcliffe SJ, Bronheim RS, Gopal A, Green S, et al. Outcomes and statistical power in adult critical care randomized trials. Am J Respir Crit Care Med. 2014;189:1469–1478. doi: 10.1164/rccm.201401-0056CP.
    1. Novack V, Beitler JR, Yitshak-Sade M, Thompson BT, Schoenfeld DA, Rubenfeld G, et al. Alive and ventilator-free: a hierarchical, composite outcome for clinical trials in the acute respiratory distress syndrome. Crit Care Med. 2020;48:158–166. doi: 10.1097/CCM.0000000000004104.
    1. Yehya N, Harhay MO, Curley MAQ, Schoenfeld DA, Reeder RW. Reappraisal of ventilator-free days in critical care research. Am J Respir Crit Care Med. 2019;200:828–836. doi: 10.1164/rccm.201810-2050CP.
    1. Bernard GR, Artigas A, Brigham KL, Carlet J, Falke K, Hudson L, et al. The American-European Consensus Conference on ARDS Definitions, mechanisms, relevant outcomes, and clinical trial coordination. Am J Respir Crit Care Med. 1994;149:818–824. doi: 10.1164/ajrccm.149.3.7509706.
    1. Schoenfeld DA, Bernard GR. Statistical evaluation of ventilator-free days as an efficacy measure in clinical trials of treatments for acute respiratory distress syndrome. Crit Care Med. 2002;30:1772–1777. doi: 10.1097/00003246-200208000-00016.
    1. Bolarinwa O. Sample size estimation for health and social science researchers: the principles and considerations for different study designs. Niger Postgrad Med J. 2020;27:67–75. doi: 10.4103/npmj.npmj_19_20.
    1. Lenth RV. Some practical guidelines for effective sample size determination. Am Stat. 2001;55:187–193. doi: 10.1198/000313001317098149.
    1. Chow S-C, Shao J, Wang H, Lokhnygina Y. Sample size calculations in clinical research. 3. New York: Chapman and Hall/CRC; 2017.
    1. Béduneau G, Pham T, Schortgen F, Piquilloud L, Zogheib E, Jonas M, et al. Epidemiology of weaning outcome according to a new definition. The WIND study. Am J Respir Crit Care Med. 2017;195:772–783. doi: 10.1164/rccm.201602-0320OC.
    1. Fine JP, Gray RJ. A proportional hazards model for the subdistribution of a competing risk. J Am Stat Assoc. 1999;94:496–509. doi: 10.1080/01621459.1999.10474144.
    1. Contentin L, Ehrmann S, Giraudeau B. Heterogeneity in the definition of mechanical ventilation duration and ventilator-free days. Am J Respir Crit Care Med. 2014;189:998–1002. doi: 10.1164/rccm.201308-1499LE.
    1. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. doi: 10.1136/bmj.n71.
    1. Renard Triché L, Jabaudon M, De Carvalho M, Piñol-Domenech N, Pereira B. Sample size estimation in randomized controlled trials using ventilator-free days as an outcome: protocol for a systematic review. PROSPERO CRD42021282304. 2021 [cited 2022 Nov 6].
    1. Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O’Neal L, et al. The REDCap consortium: building an international community of software platform partners. J Biomed Inform. 2019;95:103208. doi: 10.1016/j.jbi.2019.103208.
    1. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377–381. doi: 10.1016/j.jbi.2008.08.010.
    1. Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898. doi: 10.1136/bmj.l4898.
    1. R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. [cited 2022 Feb 13].
    1. Morris PE, Papadakos P, Russell JA, Wunderink R, Schuster DP, Truwit JD, et al. A double-blind placebo-controlled study to evaluate the safety and efficacy of L-2-oxothiazolidine-4-carboxylic acid in the treatment of patients with acute respiratory distress syndrome. Crit Care Med. 2008;36:782–788. doi: 10.1097/CCM.0B013E318164E7E4.
    1. Paine R, Standiford TJ, Dechert RE, Moss M, Martin GS, Rosenberg AL, et al. A randomized trial of recombinant human GM-CSF for patients with acute lung injury. Crit Care Med. 2012;40:90–97. doi: 10.1097/CCM.0b013e31822d7bf0.
    1. Mackle D, Bellomo R, Bailey M, Beasley R, Deane A, Eastwood G, et al. Conservative oxygen therapy during mechanical ventilation in the ICU. N Engl J Med. 2020;382:989–998. doi: 10.1056/NEJMoa1903297.
    1. Villar J, Ferrando C, Martínez D, Ambrós A, Muñoz T, Soler JA, et al. Dexamethasone treatment for the acute respiratory distress syndrome: a multicentre, randomised controlled trial. Lancet Respir Med. 2020;8:267–276. doi: 10.1016/S2213-2600(19)30417-5.
    1. Zhou Y, Jin X, Lv Y, Wang P, Yang Y, Liang G, et al. Early application of airway pressure release ventilation may reduce the duration of mechanical ventilation in acute respiratory distress syndrome. Intensive Care Med. 2017;43:1648–1659. doi: 10.1007/s00134-017-4912-z.
    1. Trouillet J-L, Luyt C-E, Guiguet M, Ouattara A, Vaissier E, Makri R, et al. Early percutaneous tracheotomy versus prolonged intubation of mechanically ventilated patients after cardiac surgery. Ann Intern Med. 2011;154:373–383. doi: 10.7326/0003-4819-154-6-201103150-00002.
    1. Simonis FD, Neto AS, Binnekade JM, Braber A, Bruin KCM, Determann RM, et al. Effect of a low vs intermediate tidal volume strategy on ventilator-free days in intensive care unit patients without ARDS: a randomized clinical trial. JAMA. 2018;320:1872–1880. doi: 10.1001/jama.2018.14280.
    1. Algera AG, Pisani L, Neto AS, Gama de Abreu M, Pelosi P, Schultz MJ, et al. Effect of a lower vs higher positive end-expiratory pressure strategy on ventilator-free days in ICU patients without ARDS: a randomized clinical trial. JAMA. 2020;324:2509–2520. doi: 10.1001/jama.2020.23517.
    1. Tomazini BM, Maia IS, Cavalcanti AB, Berwanger O, Rosa RG, Veiga VC, et al. Effect of dexamethasone on days alive and ventilator-free in patients with moderate or severe acute respiratory distress syndrome and COVID-19: the CoDEX randomized clinical trial. JAMA. 2020;324:1307–1316. doi: 10.1001/jama.2020.17021.
    1. Grieco DL, Menga LS, Cesarano M, Rosà T, Spadaro S, Bitondo MM, et al. Effect of helmet noninvasive ventilation vs high-flow nasal oxygen on days free of respiratory support in patients with COVID-19 and moderate to severe hypoxemic respiratory failure: the HENIVOT randomized clinical trial. JAMA. 2021;325:1731–1743. doi: 10.1001/jama.2021.4682.
    1. van Meenen DMP, van der Hoeven SM, Binnekade JM, de Borgie CAJM, Merkus MP, Bosch FH, et al. Effect of on-demand vs routine nebulization of acetylcysteine with salbutamol on ventilator-free days in intensive care unit patients receiving invasive ventilation: a randomized clinical trial. JAMA. 2018;319:993–1001. doi: 10.1001/jama.2018.0949.
    1. Spragg RG, Lewis JF, Walmrath H-D, Johannigman J, Bellingan G, Laterre P-F, et al. Effect of recombinant surfactant protein C-based surfactant on the acute respiratory distress syndrome. N Engl J Med. 2004;351:884–892. doi: 10.1056/NEJMoa033181.
    1. Welte T, Dellinger RP, Ebelt H, Ferrer M, Opal SM, Singer M, et al. Efficacy and safety of trimodulin, a novel polyclonal antibody preparation, in patients with severe community-acquired pneumonia: a randomized, placebo-controlled, double-blind, multicenter, phase II trial (CIGMA study) Intensive Care Med. 2018;44:438–448. doi: 10.1007/s00134-018-5143-7.
    1. Rice TW, Wheeler AP, Thompson BT, deBoisblanc BP, Steingrub J, Rock P, et al. Enteral omega-3 fatty acid, γ-linolenic acid, and antioxidant supplementation in acute lung injury. JAMA. 2011;306:1574–1581. doi: 10.1001/jama.2011.1435.
    1. Bernard GR, Francois B, Mira J-P, Vincent J-L, Dellinger RP, Russell JA, et al. Evaluating the efficacy and safety of two doses of the polyclonal anti-tumor necrosis factor-α fragment antibody AZD9773 in adult patients with severe sepsis and/or septic shock: randomized, double-blind, placebo-controlled phase IIb study*. Crit Care Med. 2014;42:504–511. doi: 10.1097/CCM.0000000000000043.
    1. Chung KK, Wolf SE, Renz EM, Allan PF, Aden JK, Merrill GA, et al. High-frequency percussive ventilation and low tidal volume ventilation in burns: a randomized controlled trial. Crit Care Med. 2010;38:1970–1977. doi: 10.1097/CCM.0b013e3181eb9d0b.
    1. Rice TW, Wheeler AP, Thompson BT, Steingrub J, Hite RD, Moss M, et al. Initial trophic vs full enteral feeding in patients with acute lung injury: the EDEN randomized trial. JAMA. 2012;307:795–803. doi: 10.1001/jama.2012.137.
    1. Bein T, Weber-Carstens S, Goldmann A, Müller T, Staudinger T, Brederlau J, et al. Lower tidal volume strategy (≈3 ml/kg) combined with extracorporeal CO2 removal versus ‘conventional’ protective ventilation (6 ml/kg) in severe ARDS: the prospective randomized Xtravent-study. Intensive Care Med. 2013;39:847–856. doi: 10.1007/s00134-012-2787-6.
    1. Hodgson CL, Cooper DJ, Arabi Y, King V, Bersten A, Bihari S, et al. Maximal recruitment open lung ventilation in acute respiratory distress syndrome (PHARLAP). A phase II, multicenter randomized controlled clinical trial. Am J Respir Crit Care Med. 2019;200:1363–1372. doi: 10.1164/rccm.201901-0109OC.
    1. Kacmarek RM, Villar J, Parrilla D, Alba F, Solano R, Liu S, et al. Neurally adjusted ventilatory assist in acute respiratory failure: a randomized controlled trial. Intensive Care Med. 2020;46:2327–2337. doi: 10.1007/s00134-020-06181-5.
    1. Kacmarek RM, Wiedemann HP, Lavin PT, Wedel MK, Tütüncü AS, Slutsky AS. Partial liquid ventilation in adult patients with acute respiratory distress syndrome. Am J Respir Crit Care Med. 2006;173:882–889. doi: 10.1164/rccm.200508-1196OC.
    1. Liu KD, Levitt J, Zhuo H, Kallet RH, Brady S, Steingrub J, et al. Randomized clinical trial of activated protein C for the treatment of acute lung injury. Am J Respir Crit Care Med. 2008;178:618–623. doi: 10.1164/rccm.200803-419OC.
    1. Rice TW, Mogan S, Hays MA, Bernard GR, Jensen GL, Wheeler AP. Randomized trial of initial trophic versus full-energy enteral nutrition in mechanically ventilated patients with acute respiratory failure. Crit Care Med. 2011;39:967–974. doi: 10.1097/CCM.0b013e31820a905a.
    1. Matthay MA, Brower RG, Carson S, Douglas IS, Eisner M, Hite D, et al. Randomized, placebo-controlled clinical trial of an aerosolized β2-agonist for treatment of acute lung injury. Am J Respir Crit Care Med. 2011;184:561–568. doi: 10.1164/rccm.201012-2090OC.
    1. Bennett-Guerrero E, Romeiser JL, Talbot LR, Ahmed T, Mamone LJ, Singh SM, et al. Severe acute respiratory syndrome coronavirus 2 convalescent plasma versus standard plasma in coronavirus disease 2019 infected hospitalized patients in New York: a double-blind randomized trial. Crit Care Med. 2021;49:1015–1025. doi: 10.1097/CCM.0000000000005066.
    1. McAuley DF, Laffey JG, O’Kane CM, Perkins GD, Mullan B, Trinder TJ, et al. Simvastatin in the acute respiratory distress syndrome. N Engl J Med. 2014;371:1695–1703. doi: 10.1056/NEJMoa1403285.
    1. Bodet-Contentin L, Frasca D, Tavernier E, Feuillet F, Foucher Y, Giraudeau B. Ventilator-free day outcomes can be misleading. Crit Care Med. 2018;46:425–429. doi: 10.1097/CCM.0000000000002890.
    1. Jabaudon M, Blondonnet R, Pereira B, Cartin-Ceba R, Lichtenstern C, Mauri T, et al. Plasma sRAGE is independently associated with increased mortality in ARDS: a meta-analysis of individual patient data. Intensive Care Med. 2018;44:1388–1399. doi: 10.1007/s00134-018-5327-1.
    1. Blackwood B, Ringrow S, Clarke M, Marshall JC, Connolly B, Rose L, et al. A core outcome set for critical care ventilation trials. Crit Care Med. 2019;47:1324–1331. doi: 10.1097/CCM.0000000000003904.
    1. Beitler JR, Sarge T, Banner-Goodspeed VM, Gong MN, Cook D, Novack V, et al. Effect of titrating positive end-expiratory pressure (PEEP) with an esophageal pressure-guided strategy vs an empirical high PEEP-Fio2 strategy on death and days free from mechanical ventilation among patients with acute respiratory distress syndrome: a randomized clinical trial. JAMA. 2019;321:846–857. doi: 10.1001/jama.2019.0555.
    1. Lazarus SC, Krishnan JA, King TS, Lang JE, Blake KV, Covar R, et al. Mometasone or tiotropium in mild asthma with a low sputum eosinophil level. N Engl J Med. 2019;380:2009–2019. doi: 10.1056/NEJMoa1814917.
    1. Bartunek J, Terzic A, Davison BA, Behfar A, Sanz-Ruiz R, Wojakowski W, et al. Cardiopoietic stem cell therapy in ischaemic heart failure: long-term clinical outcomes. ESC Heart Fail. 2020;7:3345–3354. doi: 10.1002/ehf2.13031.
    1. Ludbrook J. Should we use one-sided or two-sided P values in tests of significance? Clin Exp Pharmacol Physiol. 2013;40:357–361. doi: 10.1111/1440-1681.12086.
    1. Zeileis A, Kleiber C, Jackman S. Regression models for count data in R. J Stat Softw. 2008;27:1–25. doi: 10.18637/jss.v027.i08.
    1. Steingrimsson JA, Hanley DF, Rosenblum M. Improving precision by adjusting for prognostic baseline variables in randomized trials with binary outcomes, without regression model assumptions. Contemp Clin Trials. 2017;54:18–24. doi: 10.1016/j.cct.2016.12.026.
    1. Tai B, Chen Z, Machin D. Estimating sample size in the presence of competing risks—cause-specific hazard or cumulative incidence approach? Stat Methods Med Res. 2018;27:114–125. doi: 10.1177/0962280215623107.
    1. Granholm A, Kaas-Hansen BS, Lange T, Munch MW, Harhay MO, Zampieri FG, et al. Use of days alive without life support and similar count outcomes in randomised clinical trials—an overview and comparison of methodological choices and analysis methods. BMC Med Res Methodol. 2023;23:139. doi: 10.1186/s12874-023-01963-z.
    1. Kumle L, Võ ML-H, Draschkow D. Estimating power in (generalized) linear mixed models: an open introduction and tutorial in R. Behav Res Methods. 2021;53:2528–2543. doi: 10.3758/s13428-021-01546-0.
    1. Gagniuc PA. Markov chains: from theory to implementation and experimentation. Hoboken: Wiley; 2017.
    1. Renard Triché L. Sample size estimation according to different tests. Zenodo; 2023 Jun.

Source: PubMed

Sponsorer og samarbejdspartnere

Medicinske tilstande

Narkotikainterventioner

3
Abonner