Opportunities and challenges of delivering digital clinical trials: lessons learned from a randomised controlled trial of an online behavioural intervention for children and young people

Charlotte L Hall, Charlotte Sanderson, Beverly J Brown, Per Andrén, Sophie Bennett, Liam R Chamberlain, E Bethan Davies, Kareem Khan, Natalie Kouzoupi, David Mataix-Cols, Caitlin McKenzie, Tara Murphy, Mark Townsend, Chris Hollis, Elizabeth Murray, Charlotte L Hall, Charlotte Sanderson, Beverly J Brown, Per Andrén, Sophie Bennett, Liam R Chamberlain, E Bethan Davies, Kareem Khan, Natalie Kouzoupi, David Mataix-Cols, Caitlin McKenzie, Tara Murphy, Mark Townsend, Chris Hollis, Elizabeth Murray

Abstract

Background: Despite being the gold standard of research to determine effectiveness, randomised controlled trials (RCTs) often struggle with participant recruitment, engagement and retention. These issues may be exacerbated when recruiting vulnerable populations, such as participants with mental health issues. We aimed to update understanding of the scope of these problems in trials of health technology and identify possible solutions through reflecting on experiences from an exemplar trial (Online Remote Behavioural Intervention for Tics; ORBIT).

Method: We extracted anonymised data on recruitment, retention and requests for more funding and time from trials funded by the largest funder of health technology trials in the UK (the National Institute of Health Research Health Technology Assessment) between 2010 and 2020, and compared these with data from a recent, successful trial (ORBIT). ORBIT aimed to assess the clinical- and cost-effectiveness of blended online and human behavioural therapy for tics in young people. Many of the trial procedures, including recruitment, the intervention and data collection, were undertaken online.

Results: Data were extracted on 51 trials conducted between 2010 and 2020. Sixty per cent of trials failed to reach their original recruitment target and only 44% achieved their follow-up in the specified time frame. In contrast, ORBIT recruited to target and achieved 90% follow-up. We posit that these achievements are related to (a) judicious use of digital technology for trial procedures and (b) adequate numbers of highly trained and motivated trial staff. We provide details of both these to help other research teams plan and cost for successful trials.

Conclusion: An approach combining human and online methods may be advantageous in facilitating trial delivery, particularly in paediatric mental health services. Given the importance of successful clinical trials in advancing healthcare delivery and the waste of human and economic resources associated with unsuccessfully delivered trials, it is imperative that trials are appropriately costed and future research focusses on improving trial design and delivery.

Trial registration: The ORBIT trial is registered with ISRTCN ( ISRCTN70758207 ) Registered on March 20, 2018. and ClinicalTrials.gov ( NCT03483493 ). Registered on March 30, 2018.

Keywords: Chronic tic disorder; Internet; Online; Randomised controlled trials; Recruitment; Research design; Retention; Tourette syndrome.

Conflict of interest statement

MT is employed by the NIHR HTA. All other co-authors are co-applicants or employed on the NIHR HTA funded ORBIT trial. DMC receives royalties for contributing articles to UpToDate, Wolters Kluwer Health, and for editorial work from Elsevier, all unrelated to the submitted work.

Figures

Fig. 1
Fig. 1
ORBIT study flow. DAWBA, Development And Well-Being Assessment given at screening to determine eligibility; PIC, patient identification centre
Fig. 2
Fig. 2
ORBIT process for obtaining follow-up measures. The online and researcher based measures were completed as a simultaneous process. Researchers checked several times a week to check meausure completion

References

    1. Hariton E, Locascio JJ. Randomised controlled trials—the gold standard for effectiveness research. BJOG. 2018;125(13):1716.
    1. Harbour R, Lowe G, Twaddle S. Scottish intercollegiate guidelines network: the first 15 years (1993-2008) J R Coll Physicians Edinb. 2011;41(2):163–168.
    1. National Institute for Health and Care Excellence. . Accessed 20 Apr 2020.
    1. Agency for Healthcare Research and Quality. . Accessed 17 Apr 2020.
    1. Donovan JL, Rooshenas L, Jepson M, Elliott D, Wade J, Avery K, Mills N, Wilson C, Paramasivan S, Blazeby JM. Optimising recruitment and informed consent in randomised controlled trials: the development and implementation of the Quintet Recruitment Intervention (QRI) Trials. 2016;17(1):283.
    1. Kasenda B, Von Elm E, You J, Blümle A, Tomonaga Y, Saccilotto R, Amstutz A, Bengough T, Meerpohl JJ, Stegert M. Prevalence, characteristics, and publication of discontinued randomized trials. JAMA. 2014;311(10):1045–1052.
    1. Briel M, Speich B, von Elm E, Gloy V. Comparison of randomized controlled trials discontinued or revised for poor recruitment and completed trials with the same research question: a matched qualitative study. Trials. 2019;20(1):800.
    1. Williams RJ, Tse T, DiPiazza K, Zarin DA. Terminated trials in the ClinicalTrials. gov results database: evaluation of availability of primary outcome data and reasons for termination. PLoS One. 2015;10(5):e0127242.
    1. Campbell MK, Snowdon C, Francis D, Elbourne DR, McDonald AM, Knight RC, Entwistle V, Garcia J, Roberts I, Grant AM. Recruitment to randomised trials: strategies for trial enrolment and participation study. The STEPS study. Health Tech Assessment. 2007;11(iii):ix–105.
    1. Huang GD, Bull J, McKee KJ, Mahon E, Harper B, Roberts JN. Clinical trials recruitment planning: a proposed framework from the clinical trials transformation initiative. Contemp Clin Trials. 2018;66:74–79.
    1. Treweek S, Lockhart P, Pitkethly M, Cook JA, Kjeldstrøm M, Johansen M, Taskila TK, Sullivan FM, Wilson S, Jackson C. Methods to improve recruitment to randomised controlled trials: Cochrane systematic review and meta-analysis. BMJ Open. 2013;3(2):e002360.
    1. Gkioni E, Rius R, Dodd S, Gamble C. A systematic review describes models for recruitment prediction at the design stage of a clinical trial. J Clin Epidemiol. 2019;115:141–149.
    1. Walters SJ, dos Anjos Henriques-Cadby IB, Bortolami O, Flight L, Hind D, Jacques RM, Knox C, Nadin B, Rothwell J, Surtees M. Recruitment and retention of participants in randomised controlled trials: a review of trials funded and published by the United Kingdom Health Technology Assessment Programme. BMJ Open. 2017;7(3):e015276.
    1. Watson NL, Mull KE, Heffner JL, McClure JB, Bricker JB. Participant recruitment and retention in remote eHealth intervention trials: methods and lessons learned from a large randomized controlled trial of two web-based smoking interventions. JMIR. 2018;20(8):e10351.
    1. Metse AP, Stockings E, Bailey J, Regan T, Bartlem K, Wolfenden L, Taylor G, Wiggers J, Bowman J. Rates of retention of persons with a mental health disorder in outpatient smoking cessation and reduction trials, and associated factors: protocol for a systematic review and meta-analysis. BMJ Open. 2019;9(9):e030646.
    1. Van der Meer RM, Willemsen MC, Smit F, Cuijpers P. Smoking cessation interventions for smokers with current or past depression. Cochrane Database Syst Rev. 2013;(8). 10.1002/14651858.CD006102.pub2.
    1. Apollonio D, Philipps R, Bero L. Interventions for tobacco use cessation in people in treatment for or recovery from substance use disorders. Cochrane Database Syst Rev. 2016;(11). 10.1002/14651858.CD010274.pub2.
    1. Naidoo N, Ravaud P, Young B, Amiel P, Schanté D, Clarke M, Boutron I. The research burden of randomized controlled trial participation: a systematic thematic synthesis of qualitative evidence. BMC Med. 2020;18(1):6.
    1. Fisher L, Hessler D, Naranjo D, Polonsky W. AASAP: a program to increase recruitment and retention in clinical trials. Patient Educ Couns. 2012;86(3):372–377.
    1. Abshire M, Dinglas VD, Cajita MIA, Eakin MN, Needham DM, Himmelfarb CD. Participant retention practices in longitudinal clinical research studies with high retention rates. BMC Med Res Methodol. 2017;17(1):30.
    1. Barak A. Psychological applications on the Internet: a discipline on the threshold of a new millennium. Appl Prev Psychol. 1999;8(4):231–245.
    1. Koneska E, Appelbe D, Williamson PR, Dodd S. Usage metrics of web-based interventions evaluated in randomized controlled trials: systematic review. J Med Internet Res. 2020;22(4):e15474.
    1. Murray E, Hekler EB, Andersson G, Collins LM, Doherty A, Hollis C, Rivera DE, West R, Wyatt JC. Evaluating digital health interventions: key questions and approaches. Am J Prev Med. 2016;51(5):843–851.
    1. Murray E, Khadjesari Z, White I, Kalaitzaki E, Godfrey C, McCambridge J, Thompson S, Wallace P. Methodological challenges in online trials. J Med Int Res. 2009;11(2):e9.
    1. Aspvall K, Lenhard F, Melin K, Krebs G, Norlin L, Näsström K, Jassi A, Turner C, Knoetze E, Serlachius E. Implementation of internet-delivered cognitive behaviour therapy for pediatric obsessive-compulsive disorder: lessons from clinics in Sweden, United Kingdom and Australia. Internet Interv. 2020;20:100308.
    1. O’Connor S, Hanlon P, O’Donnell CA, Garcia S, Glanville J, Mair FS. Understanding factors affecting patient and public engagement and recruitment to digital health interventions: a systematic review of qualitative studies. BMC Med Inform Decis Mak. 2016;16(1):120.
    1. Lillevoll KR, Vangberg HCB, Griffiths KM, Eisemann MR. Uptake and adherence of a self-directed internet-based mental health intervention with tailored e-mail reminders in senior high schools in Norway. BMC Psychiatry. 2014;14(1):14.
    1. Owen JE, Jaworski BK, Kuhn E, Makin-Byrd KN, Ramsey KM, Hoffman JE. mHealth in the wild: using novel data to examine the reach, use, and impact of PTSD coach. JMIR Ment Health. 2015;2(1):e7.
    1. Zanaboni P, Ngangue P, Mbemba GIC, Schopf TR, Bergmo TS, Gagnon M-P. Methods to evaluate the effects of internet-based digital health interventions for citizens: systematic review of reviews. J Med Internet Res. 2018;20(6):e10202.
    1. Hall C, Davies E, Andrén P, Murphy T, Bennett S, Brown B, Brown S, Chamberlain L, Craven M, Evans A, et al. Investigating a therapist-guided, parent-assisted remote digital behavioural intervention for tics in children and adolescents—‘Online Remote Behavioural Intervention for Tics’ (ORBIT) trial: protocol of an internal pilot study and single-blind randomised controlled trial. BMJ Open. 2019;9(1):e027583.
    1. ISRTCN: . Accssed 18 Apr 2020.
    1. National Institute of Health Reseach: . Accessed 17 Apr 2020.
    1. Zivetz L. The ICD-10 Classification of Mental and Behavioural Disorders: Clinical Descriptions and Diagnostic Guidelines, vol. 1: World Health Organization 1992; 67(30): 227.
    1. Cuenca J, Glazebrook C, Kendall T, Hedderly T, Heyman I, Jackson G, Murphy T, Rickards H, Robertson M, Stern J, Trayner P, Hollis C. Perceptions of treatment for tics among young people with Tourette syndrome and their parents: a mixed methods study. BMC Psychiatry. 2015;15(1):46.
    1. Khadjesari Z, Murray E, Kalaitzaki E, White IR, McCambridge J, Thompson SG, Wallace P, Godfrey C. Impact and costs of incentives to reduce attrition in online trials: two randomized controlled trials. J Med Internet Res. 2011;13(1):e26.
    1. Andrén P, Aspvall K, de la Cruz LF, Wiktor P, Romano S, Andersson E, Murphy T, Isomura K, Serlachius E, Mataix-Cols D. Therapist-guided and parent-guided internet-delivered behaviour therapy for paediatric Tourette’s disorder: a pilot randomised controlled trial with long-term follow-up. BMJ Open. 2019;9(2):e024685.
    1. Piacentini J, Woods DW, Scahill L, Wilhelm S, Peterson AL, Chang S, Ginsburg GS, Deckersbach T, Dziura J, Levi-Pearl S. Behavior therapy for children with Tourette disorder: a randomized controlled trial. JAMA. 2010;303(19):1929–1937.
    1. MindTech MedTech Co-operative. . Accssed 21 Apr 2020.
    1. Khan K, Hollis C, Hall CL, Davies EB, Mataix-Cols D, Andrén P, Murphy T, Brown BJ, Murray E, Glazebrook C. Protocol for the Process Evaluation of the Online Remote Behavioural Intervention for Tics (ORBIT) randomized controlled trial for children and young people. Trials. 2020;21(1):1–10.
    1. Cosgrove V, Gliddon E, Berk L, Grimm D, Lauder S, Dodd S, Berk M, Suppes T. Online ethics. where will the interface of mental health and the internet lead us? Int J Bipolar Disord. 2017;5(1):26.
    1. Sanderson C, Kouzoupi N, Hall CL. Technology matters: the human touch in a digital age–a blended approach in mental healthcare delivery with children and young people. Child Adolesc Mental Health. 2020;26(2):120–122.
    1. Jackson M, Berman N, Huber M, Snetselaar L, Granek I, Boe K, Milas C, Spivak J, Chlebowski RT. Research staff turnover and participant adherence in the Women’s Health Initiative. Control Clin Trials. 2003;24(4):422–435.
    1. Beatty L, Binnion C. A systematic review of predictors of, and reasons for, adherence to online psychological interventions. Int J Behav Med. 2016;23:776–794.
    1. Vernmark K, Hesser H, Topooco N, Berger T, Riper H, Luuk L, Backlund L, Carlbring P, Andersson G. Working alliance as a predictor of change in depression during blended cognitive behaviour therapy. Cogn Behav Ther. 2019;48(4):285–299.

Source: PubMed

Szponzorok és közreműködők

Egészségi állapot

Kábítószer-beavatkozások

3
Iratkozz fel