Exploring threats to generalisability in a large international rehabilitation trial (AVERT)

Julie Bernhardt, Audrey Raffelt, Leonid Churilov, Richard I Lindley, Sally Speare, Jacqueline Ancliffe, Md Ali Katijjahbe, Shahul Hameed, Sheila Lennon, Anna McRae, Dawn Tan, Jan Quiney, Hannah C Williamson, Janice Collier, Helen M Dewey, Geoffrey A Donnan, Peter Langhorne, Amanda G Thrift, AVERT Trialists’ Collaboration

Abstract

Objective: The purpose of this paper is to examine potential threats to generalisability of the results of a multicentre randomised controlled trial using data from A Very Early Rehabilitation Trial (AVERT).

Design: AVERT is a prospective, parallel group, assessor-blinded randomised clinical trial. This paper presents data assessing the generalisability of AVERT.

Setting: Acute stroke units at 44 hospitals in 8 countries.

Participants: The first 20,000 patients screened for AVERT, of whom 1158 were recruited and randomised.

Model: We use the Proximal Similarity Model, which considers the person, place, and setting and practice, as a framework for considering generalisability. As well as comparing the recruited patients with the target population, we also performed an exploratory analysis of the demographic, clinical, site and process factors associated with recruitment.

Results: The demographics and stroke characteristics of the included patients in the trial were broadly similar to population-based norms, with the exception that AVERT had a greater proportion of men. The most common reason for non-recruitment was late arrival to hospital (ie, >24 h). Overall, being older and female reduced the odds of recruitment to the trial. More women than men were excluded for most of the reasons, including refusal. The odds of exclusion due to early deterioration were particularly high for those with severe stroke (OR=10.4, p<0.001, 95% CI 9.27 to 11.65).

Conclusions: A model which explores person, place, and setting and practice factors can provide important information about the external validity of a trial, and could be applied to other clinical trials.

Trial registration number: Australian New Zealand Clinical Trials Registry (ACTRN12606000185561) and Clinicaltrials.gov (NCT01846247).

Keywords: Generalisability; Proximal Similarity Model; Randomised Control Trial; Rehabilitation.

Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Figures

Figure 1
Figure 1
Proximal similarity framework applied to the AVERT trial: a model for conceptualising the dimensions along which the sample of patients may be similar to the target population. Each dimension (person, place and setting and practice) is affected by specific factors which may threaten external validity (AVERT, A Very Early Rehabilitation Trial; ICU, intensive care unit).
Figure 2
Figure 2
Relationship between trial inclusion/exclusion criteria and screening log categories (AVERT, A Very Early Rehabilitation Trial).
Figure 3
Figure 3
Methods of explorative analysis using the first reason for non-recruitment (arrived after 24 h) as an example. Bold boxes indicate data grouping. Analyses were repeated for all 10 reasons for non-recruitment, with four patient demographic and clinical factors. Trial site and month of trial were controlled for in each analysis (ICU, intensive care unit; mRS, modified Rankin Scale).

References

    1. Higgins JP, Green S. Cochrane handbook for systematic reviews of interventions. West Sussex, England: The Cochrane Collaboration, 2008.
    1. Rothwell PM. External validity of randomised controlled trials: “To whom do the results of this trial apply?” Lancet 2005;365:82–93. 10.1016/S0140-6736(04)17670-8
    1. Moher D, Hopewell S, Schulz KF et al. . CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol 2010;63:e1–e37. 10.1016/j.jclinepi.2010.03.004
    1. Glasgow RE, Green LW, Klesges LM et al. . External validity: we need to do more. Ann Behav Med 2006;31:105–8. 10.1207/s15324796abm3102_1
    1. Rothwell PM. Commentary: external validity of results of randomized trials: disentangling a complex concept. Int J Epidemiol 2010;39:94–6. 10.1093/ije/dyp305
    1. Rothwell PM. Factors that can affect the external validity of randomised controlled trials. PLoS Clin Trials 2006;1:e9 10.1371/journal.pctr.0010009
    1. Glasgow R, Klesges L, Dzewaltowski D et al. . The future of health behavior change research: what is needed to improve translation of research into health promotion practice? Ann Behav Med 2004;27:3–12. 10.1207/s15324796abm2701_2
    1. Glasgow RE, Emmons KM. How can we increase translation of research into practice? Types of evidence needed. Annu Rev Public Health 2007;28:413–33. 10.1146/annurev.publhealth.28.021406.144145
    1. Dekkers OM, von Elm E, Algra A et al. . How to assess the external validity of therapeutic trials: a conceptual approach. Int J Epidemiol 2010;39:89–94. 10.1093/ije/dyp174
    1. Jüni P, Altman DG, Egger M. Assessing the quality of controlled clinical trials. BMJ 2001;323:42–6. 10.1136/bmj.323.7303.42
    1. Bernhardt J, Churilov L, Dewey H et al. . Statistical analysis plan (SAP) for A Very Early Rehabilitation Trial (AVERT): an international trial to determine the efficacy and safety of commencing out of bed standing and walking training (very early mobilisation) within 24 h of stroke onset vs usual stroke unit care. Int J Stroke 2015;10:23–4. 10.1111/ijs.12423
    1. Trochim WMK. The research methods knowledge base. 2nd edn William M.K. Trochim, 2006. (cited 18 February 2015).
    1. National Stroke Foundation. National Stroke Audit—Acute Services Organisational Survey report 2013. Melbourne, Australia: National Stroke Foundation.
    1. Veerbeek JM, Kwakkel G, van Wegen EEH et al. . Early prediction of outcome of activities of daily living after stroke: a systematic review. Stroke 2011;42:1482–8. 10.1161/STROKEAHA.110.604090
    1. van Almenkerk S, Smalbrugge M, Depla MFIA et al. . What predicts a poor outcome in older stroke survivors? A systematic review of the literature. Dis Rehabil 2013;35:1774–82. 10.3109/09638288.2012.756941
    1. Briggs D, Felberg R, Malkoff M et al. . Should mild or moderate strokes be admitted to an intensive care unit? Stroke 2001;32:871–6. 10.1161/01.STR.32.4.871
    1. Brott T, Adams H, Olinger C et al. . Measurements of acute cerebral infarction: a clinical examination scale. Stroke 1989;20:864–70. 10.1161/01.STR.20.7.864
    1. Ali M, Bath PM, Curram J et al. . The virtual international stroke trials archive. Stroke 2007;38:1905–10. 10.1161/STROKEAHA.106.473579
    1. Feigin VL, Lawes CMM, Bennett DA et al. . Worldwide stroke incidence and early case fatality reported in 56 population-based studies: a systematic review. Lancet Neurol 2009;8:355–69. 10.1016/S1474-4422(09)70025-0
    1. Feigin VL, Lawes CMM, Bennettt DA et al. . Stroke epidemiology: a review of population-based studies of incidence, prevalence, and case-fatality in the late 20th century. Lancet Neurol 2003;2:43–53. 10.1016/S1474-4422(03)00266-7
    1. The VISTA Steering Committees. Development, expansion, and use of a stroke clinical trials resource for novel exploratory analyses. Int J Stroke 2012;7:133–8.
    1. Schulz KF, Altman DG, Moher D; for the CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials. Ann Intern Med 2010;152:1–7. 10.7326/0003-4819-152-11-201006010-00232
    1. Busija L, Tao LW, Liew D et al. . Do patients who take part in stroke research differ from non-participants? Implications for generalisability of results. Cerebrovasc Dis 2013;35:483–91. 10.1159/000350724
    1. Haast RA, Gustafson DR, Kiliaan AJ. Sex differences in stroke. J Cereb Blood Flow Metab 2012;32:2100–7. 10.1038/jcbfm.2012.141
    1. Reeves MJ, Bushnell CD, Howard G et al. . Sex differences in stroke: epidemiology, clinical presentation, medical care, and outcomes. Lancet Neurol 2008;7:915–26. 10.1016/S1474-4422(08)70193-5
    1. Di Carlo A, Lamassa M, Baldereschi M et al. . Sex differences in the clinical presentation, resource use, and 3-month outcome of acute stroke in Europe: data from a multicenter multinational hospital-based registry. Stroke 2003;34:1114–19. 10.1161/01.STR.0000068410.07397.D7
    1. Gall S, Donnan G, Dewey H et al. . Sex differences in presentation, severity, and management of stroke in a population-based study. Neurology 2010;74:975–81. 10.1212/WNL.0b013e3181d5a48f
    1. Barr J, McKinley S, O'Brien E et al. . Patient recognition of and response to symptoms of TIA or stroke. Neuroepidemiology 2006;26:168–75. 10.1159/000091659
    1. Mishra NK, Diener HC, Lyden PD et al. . the VISTA Collaborators. Influence of age on outcome from thrombolysis in acute stroke: a controlled comparison in patients from the Virtual International Stroke Trials Archive (VISTA). Stroke 2010;41:2840–8. 10.1161/STROKEAHA.110.586206
    1. Morgenstern LB, Hemphill JC, Anderson C et al. . Guidelines for the management of spontaneous intracerebral hemorrhage: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 2010;41:2108–29. 10.1161/STR.0b013e3181ec611b
    1. O'Donnell MJ, Xavier D, Liu L et al. . Risk factors for ischaemic and intracerebral haemorrhagic stroke in 22 countries (the INTERSTROKE study): a case-control study. Lancet 2010;376:112–23. 10.1016/S0140-6736(10)60834-3
    1. Lacy CR, Suh DC, Bueno M et al. . Delay in presentation and evaluation for acute stroke: Stroke Time Registry for Outcomes Knowledge and Epidemiology (STROKE). Stroke 2001;32:63–9. 10.1161/01.STR.32.1.63
    1. Sankoh AJ, Huque MF, Dubey SD. Some comments on frequently used multiple endpoint adjustment methods in clinical trials. Stat Med 1997;16:2529–42. 10.1002/(SICI)1097-0258(19971130)16:22<2529::AID-SIM692>;2-J

Source: PubMed

Sponsors et collaborateurs

Les conditions médicales

Interventions en matière de drogue

3
S'abonner