Maximizing Participant Engagement, Participation, and Retention in Cohort Studies Using Digital Methods: Rapid Review to Inform the Next Generation of Very Large Birth Cohorts

Joanna Nkyekyer, Susan A Clifford, Fiona K Mensah, Yichao Wang, Lauren Chiu, Melissa Wake, Joanna Nkyekyer, Susan A Clifford, Fiona K Mensah, Yichao Wang, Lauren Chiu, Melissa Wake

Abstract

Background: Many current research needs can only be addressed using very large cohorts. In such studies, traditional one-on-one phone, face-to-face, or paper-based engagement may not be feasible. The only realistic mechanism for maintaining engagement and participation at this scale is via digital methods. Given the substantial investment being made into very large birth cohort studies, evidence for optimal methods of participant engagement, participation, and retention over sustained periods without in-person contact from researchers is paramount.

Objective: This study aims to provide an overview of systematic reviews and meta-analyses evaluating alternative strategies for maximizing participant engagement and retention rates in large-scale studies using digital methods.

Methods: We used a rapid review method by searching PubMed and Ovid MEDLINE databases from January 2012 to December 2019. Studies evaluating at least 1 e-engagement, participation, or retention strategy were eligible. Articles were screened for relevance based on preset inclusion and exclusion criteria. The methodological quality of the included reviews was assessed using the AMSTAR-2 (Assessing the Methodological Quality of Systematic Reviews 2) measurement tool, and a narrative synthesis of the data was conducted.

Results: The literature search yielded 19 eligible reviews. Overall, 63% (n=12) of these reviews reported on the effectiveness of e-engagement or participation promotion strategies. These evaluations were generally not conducted within very large observational digital cohorts. Most of the contributing reviews included multipurpose cohort studies (with both observational and interventional elements) conducted in clinical and research settings. Email or SMS text message reminders, SMS text messages or voice notifications, and incentives were the most commonly used design features to engage and retain participants. For parental outcomes, engagement-facilitation interventions influenced uptake and behavior change, including video feedback, goal setting, and intensive human facilitation and support. Participant-stated preferences for content included new knowledge, reminders, solutions, and suggestions about health issues presented in a clear, short, and personalized way. Perinatal and postpartum women valued self-monitoring and personalized feedback. Digital reminders and multiple SMS text messages were specific strategies that were found to increase adherence to medication and clinic attendance, respectively.

Conclusions: This review adds to the growing literature evaluating methods to optimize engagement and participation that may apply to large-scale studies using digital methods; it is promising that most e-engagement and participation promotion strategies appear to be effective. However, these reviews canvassed relatively few strategies, suggesting that few alternative strategies have been experimentally evaluated. The reviews also revealed a dearth of experimental evidence generated within very large observational digital cohort studies, which may reflect the small number of such studies worldwide. Thus, very large studies may need to proactively build in experimental opportunities to test engagement and retention approaches to enhance the success of their own and other large digital contact studies.

Keywords: cohort studies; communication modes; digital study; mobile phone; participant engagement; research methodology; retention; systematic reviews.

Conflict of interest statement

Conflicts of Interest: None declared.

©Joanna Nkyekyer, Susan A Clifford, Fiona K Mensah, Yichao Wang, Lauren Chiu, Melissa Wake. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 14.05.2021.

Figures

Figure 1
Figure 1
Search and screening process.

References

    1. Fry A, Littlejohns TJ, Sudlow C, Doherty N, Adamska L, Sprosen T, Collins R, Allen NE. Comparison of sociodemographic and health-related characteristics of UK biobank participants with those of the general population. Am J Epidemiol. 2017 Nov 01;186(9):1026–34. doi: 10.1093/aje/kwx246.
    1. McCarthy M. US cancels plan to study 100,000 children from "womb" to age 21. Br Med J. 2014 Dec 19;349:g7775. doi: 10.1136/bmj.g7775.
    1. Pearson H. Massive UK baby study cancelled. Nature. 2015 Oct 29;526(7575):620–1. doi: 10.1038/526620a.
    1. Magnus P, Irgens LM, Haug K, Nystad W, Skjaerven R, Stoltenberg C, MoBa Study Group Cohort profile: the Norwegian Mother and Child Cohort Study (MoBa) Int J Epidemiol. 2006 Oct;35(5):1146–50. doi: 10.1093/ije/dyl170.
    1. O'Brien HL, Toms EG. What is user engagement? A conceptual framework for defining user engagement with technology. J Am Soc Inf Sci Technol. 2008 Apr;59(6):938–55. doi: 10.1002/asi.20801.
    1. Simblett S, Greer B, Matcham F, Curtis H, Polhemus A, Ferrão J, Gamble P, Wykes T. Barriers to and facilitators of engagement with remote measurement technology for managing health: systematic review and content analysis of findings. J Med Internet Res. 2018 Jul 12;20(7):e10480. doi: 10.2196/10480.
    1. Batterham PJ, Calear AL, Sunderland M, Kay-Lambkin F, Farrer LM, Gulliver A. A brief intervention to increase uptake and adherence of an online program for depression and anxiety: Protocol for the Enhancing Engagement with Psychosocial Interventions (EEPI) Randomized Controlled Trial. Contemp Clin Trials. 2019 Mar;78:107–15. doi: 10.1016/j.cct.2019.01.015.
    1. Ritterband LM, Thorndike FP, Cox DJ, Kovatchev BP, Gonder-Frederick LA. A behavior change model for internet interventions. Ann Behav Med. 2009 Aug;38(1):18–27. doi: 10.1007/s12160-009-9133-4.
    1. Teague S, Youssef GJ, Macdonald JA, Sciberras E, Shatte A, Fuller-Tyszkiewicz M, Greenwood C, McIntosh J, Olsson CA, Hutchinson D, SEED Lifecourse Sciences Theme Retention strategies in longitudinal cohort studies: a systematic review and meta-analysis. BMC Med Res Methodol. 2018 Nov 26;18(1):151. doi: 10.1186/s12874-018-0586-7.
    1. Gustavson K, von Soest T, Karevold E, Røysamb E. Attrition and generalizability in longitudinal studies: findings from a 15-year population-based study and a Monte Carlo simulation study. BMC Public Health. 2012 Oct 29;12:918. doi: 10.1186/1471-2458-12-918.
    1. Szklo M. Population-based cohort studies. Epidemiol Rev. 1998;20(1):81–90. doi: 10.1093/oxfordjournals.epirev.a017974.
    1. Booker CL, Harding S, Benzeval M. A systematic review of the effect of retention methods in population-based cohort studies. BMC Public Health. 2011 Apr 19;11:249. doi: 10.1186/1471-2458-11-249.
    1. Fewtrell MS, Kennedy K, Singhal A, Martin RM, Ness A, Hadders-Algra M, Koletzko B, Lucas A. How much loss to follow-up is acceptable in long-term randomised trials and prospective studies? Arch Dis Child. 2008 Jun;93(6):458–61. doi: 10.1136/adc.2007.127316.
    1. Siddiqi A, Sikorskii A, Given CW, Given B. Early participant attrition from clinical trials: role of trial design and logistics. Clin Trials. 2008;5(4):328–35. doi: 10.1177/1740774508094406.
    1. Gupta A, Calfas KJ, Marshall SJ, Robinson TN, Rock CL, Huang JS, Epstein-Corbin M, Servetas C, Donohue MC, Norman GJ, Raab F, Merchant G, Fowler JH, Griswold WG, Fogg BJ, Patrick K. Clinical trial management of participant recruitment, enrollment, engagement, and retention in the SMART study using a Marketing and Information Technology (MARKIT) model. Contemp Clin Trials. 2015 May;42:185–95. doi: 10.1016/j.cct.2015.04.002.
    1. Watson JM, Torgerson DJ. Increasing recruitment to randomised trials: a review of randomised controlled trials. BMC Med Res Methodol. 2006 Jul 19;6:34. doi: 10.1186/1471-2288-6-34.
    1. Robinson KA, Dennison CR, Wayman DM, Pronovost PJ, Needham DM. Systematic review identifies number of strategies important for retaining study participants. J Clin Epidemiol. 2007 Aug;60(8):757–65. doi: 10.1016/j.jclinepi.2006.11.023.
    1. Robinson KA, Dinglas VD, Sukrithan V, Yalamanchilli R, Mendez-Tellez PA, Dennison-Himmelfarb C, Needham DM. Updated systematic review identifies substantial number of retention strategies: using more strategies retains more study participants. J Clin Epidemiol. 2015 Dec;68(12):1481–7. doi: 10.1016/j.jclinepi.2015.04.013.
    1. Wake M, Hu YJ, Warren H, Danchin M, Fahey M, Orsini F, Pacilli M, Perrett KP, Saffery R, Davidson A. Integrating trials into a whole-population cohort of children and parents: statement of intent (trials) for the Generation Victoria (GenV) cohort. BMC Med Res Methodol. 2020 Sep 24;20(1):238. doi: 10.1186/s12874-020-01111-x.
    1. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA, PRISMA-P Group Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015 Jan 01;4:1. doi: 10.1186/2046-4053-4-1.
    1. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, Moher D, Tugwell P, Welch V, Kristjansson E, Henry DA. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. Br Med J. 2017 Sep 21;358:j4008. doi: 10.1136/bmj.j4008.
    1. Adams J, Bateman B, Becker F, Cresswell T, Flynn D, McNaughton R, Oluboyede Y, Robalino S, Ternent L, Sood BG, Michie S, Shucksmith J, Sniehotta FF, Wigham S. Effectiveness and acceptability of parental financial incentives and quasi-mandatory schemes for increasing uptake of vaccinations in preschool children: systematic review, qualitative study and discrete choice experiment. Health Technol Assess. 2015 Nov;19(94):1–176. doi: 10.3310/hta19940. doi: 10.3310/hta19940.
    1. Alkhaldi G, Hamilton FL, Lau R, Webster R, Michie S, Murray E. The effectiveness of prompts to promote engagement with digital interventions: a systematic review. J Med Internet Res. 2016 Jan 08;18(1):e6. doi: 10.2196/jmir.4790.
    1. Ames HM, Glenton C, Lewin S, Tamrat T, Akama E, Leon N. Clients' perceptions and experiences of targeted digital communication accessible via mobile devices for reproductive, maternal, newborn, child, and adolescent health: a qualitative evidence synthesis. Cochrane Database Syst Rev. 2019 Oct 14;10:CD013447. doi: 10.1002/14651858.CD013447.
    1. Atkinson KM, Wilson K, Murphy MS, El-Halabi S, Kahale LA, Laflamme LL, El-Khatib Z. Effectiveness of digital technologies at improving vaccine uptake and series completion - a systematic review and meta-analysis of randomized controlled trials. Vaccine. 2019 May 21;37(23):3050–60. doi: 10.1016/j.vaccine.2019.03.063.
    1. Baumeister H, Reichler L, Munzinger M, Lin J. The impact of guidance on Internet-based mental health interventions — a systematic review. Internet Interv. 2014 Oct;1(4):205–15. doi: 10.1016/j.invent.2014.08.003.
    1. Belisario JS, Jamsek J, Huckvale K, O'Donoghue J, Morrison CP, Car J. Comparison of self-administered survey questionnaire responses collected using mobile apps versus other methods. Cochrane Database Syst Rev. 2015 Jul 27;(7):MR000042. doi: 10.1002/14651858.MR000042.pub2.
    1. Dol J, Delahunty-Pike A, Anwar Siani S, Campbell-Yeo M. eHealth interventions for parents in neonatal intensive care units: a systematic review. JBI Database System Rev Implement Rep. 2017 Dec;15(12):2981–3005. doi: 10.11124/JBISRIR-2017-003439.
    1. Dubad M, Winsper C, Meyer C, Livanou M, Marwaha S. A systematic review of the psychometric properties, usability and clinical impacts of mobile mood-monitoring applications in young people. Psychol Med. 2018 Jan;48(2):208–28. doi: 10.1017/S0033291717001659.
    1. Garrido S, Millington C, Cheers D, Boydell K, Schubert E, Meade T, Nguyen QV. What works and what doesn't work? A systematic review of digital mental health interventions for depression and anxiety in young people. Front Psychiatry. 2019;10:759. doi: 10.3389/fpsyt.2019.00759. doi: 10.3389/fpsyt.2019.00759.
    1. Kang HS, De Gagne JC, Son YD, Chae S. Completeness of human papilloma virus vaccination: a systematic review. J Pediatr Nurs. 2018;39:7–14. doi: 10.1016/j.pedn.2017.12.003.
    1. Lattie EG, Adkins EC, Winquist N, Stiles-Shields C, Wafford QE, Graham AK. Digital mental health interventions for depression, anxiety, and enhancement of psychological well-being among college students: systematic review. J Med Internet Res. 2019 Jul 22;21(7):e12869. doi: 10.2196/12869.
    1. Lim NL, Shorey S. Effectiveness of technology-based educational interventions on the empowerment related outcomes of children and young adults with cancer: a quantitative systematic review. J Adv Nurs. 2019 Oct;75(10):2072–84. doi: 10.1111/jan.13974.
    1. Mertens L, Braeken MA, Bogaerts A. Effect of lifestyle coaching including telemonitoring and telecoaching on gestational weight gain and postnatal weight loss: a systematic review. Telemed J E Health. 2019 Oct;25(10):889–901. doi: 10.1089/tmj.2018.0139.
    1. Parsons D, Cordier R, Vaz S, Lee HC. Parent-mediated intervention training delivered remotely for children with autism spectrum disorder living outside of urban areas: systematic review. J Med Internet Res. 2017 Aug 14;19(8):e198. doi: 10.2196/jmir.6651.
    1. Robotham D, Satkunanathan S, Reynolds J, Stahl D, Wykes T. Using digital notifications to improve attendance in clinic: systematic review and meta-analysis. BMJ Open. 2016 Oct 24;6(10):e012116. doi: 10.1136/bmjopen-2016-012116.
    1. Thakkar J, Kurup R, Laba T, Santo K, Thiagalingam A, Rodgers A, Woodward M, Redfern J, Chow CK. Mobile telephone text messaging for medication adherence in chronic disease: a meta-analysis. JAMA Intern Med. 2016 Mar;176(3):340–9. doi: 10.1001/jamainternmed.2015.7667.
    1. Tromp K, Zwaan CM, van de Vathorst S. Motivations of children and their parents to participate in drug research: a systematic review. Eur J Pediatr. 2016 May;175(5):599–612. doi: 10.1007/s00431-016-2715-9.
    1. Välimäki M, Anttila K, Anttila M, Lahti M. Web-based interventions supporting adolescents and young people with depressive symptoms: systematic review and meta-analysis. JMIR Mhealth Uhealth. 2017 Dec 08;5(12):e180. doi: 10.2196/mhealth.8624.
    1. Whitaker C, Stevelink S, Fear N. The use of Facebook in recruiting participants for health research purposes: a systematic review. J Med Internet Res. 2017 Aug 28;19(8):e290. doi: 10.2196/jmir.7071.
    1. Safran C, Pompilio-Weitzner G, Emery KD, Hampers L. Collaborative approaches to e-Health: valuable for users and non-users. Stud Health Technol Inform. 2005;116:879–84.
    1. Brown MJ, Sinclair M, Liddle D, Hill AJ, Madden E, Stockdale J. A systematic review investigating healthy lifestyle interventions incorporating goal setting strategies for preventing excess gestational weight gain. PLoS One. 2012;7(7):e39503. doi: 10.1371/journal.pone.0039503.
    1. Gardner B, Wardle J, Poston L, Croker H. Changing diet and physical activity to reduce gestational weight gain: a meta-analysis. Obes Rev. 2011 Jul;12(7):602–20. doi: 10.1111/j.1467-789X.2011.00884.x.
    1. Fernandez ID, Groth SW, Reschke JE, Graham ML, Strawderman M, Olson CM. eMoms: electronically-mediated weight interventions for pregnant and postpartum women. Study design and baseline characteristics. Contemp Clin Trials. 2015 Jul;43:63–74. doi: 10.1016/j.cct.2015.04.013.
    1. Herring SJ, Cruice JF, Bennett GG, Davey A, Foster GD. Using technology to promote postpartum weight loss in urban, low-income mothers: a pilot randomized controlled trial. J Nutr Educ Behav. 2014;46(6):610–5. doi: 10.1016/j.jneb.2014.06.002.
    1. Gilmore LA, Klempel MC, Martin CK, Myers CA, Burton JH, Sutton EF, Redman LM. Personalized mobile health intervention for health and weight loss in postpartum women receiving women, infants, and children benefit: a randomized controlled pilot study. J Womens Health (Larchmt) 2017 Jul;26(7):719–27. doi: 10.1089/jwh.2016.5947.
    1. Kernot J, Olds T, Lewis LK, Maher C. Effectiveness of a facebook-delivered physical activity intervention for post-partum women: a randomized controlled trial protocol. BMC Public Health. 2013 May 29;13:518. doi: 10.1186/1471-2458-13-518.
    1. Phelan S, Brannen A, Erickson K, Diamond M, Schaffner A, Muñoz-Christian K, Stewart A, Sanchez T, Rodriguez VC, Ramos DI, McClure L, Stinson C, Tate DF. 'Fit Moms/Mamás Activas' internet-based weight control program with group support to reduce postpartum weight retention in low-income women: study protocol for a randomized controlled trial. Trials. 2015 Mar 25;16:59. doi: 10.1186/s13063-015-0573-9.
    1. Pollak KI, Alexander SC, Bennett G, Lyna P, Coffman CJ, Bilheimer A, Farrell D, Bodner ME, Swamy GK, Østbye T. Weight-related SMS texts promoting appropriate pregnancy weight gain: a pilot study. Patient Educ Couns. 2014 Nov;97(2):256–60. doi: 10.1016/j.pec.2014.07.030.
    1. Willcox J, Wilkinson S, Lappas M, Ball K, Crawford D, McCarthy E, Fjeldsoe B, Whittaker R, Maddison R, Campbell K. A mobile health intervention promoting healthy gestational weight gain for women entering pregnancy at a high body mass index: the txt4two pilot randomised controlled trial. Brit J Obstet Gynec. 2017 Oct 20;124(11):1718–28. doi: 10.1111/1471-0528.14552.
    1. Davies EB, Morriss R, Glazebrook C. Computer-delivered and web-based interventions to improve depression, anxiety, and psychological well-being of university students: a systematic review and meta-analysis. J Med Internet Res. 2014 May 16;16(5):e130. doi: 10.2196/jmir.3142.
    1. Harrer M, Adam SH, Baumeister H, Cuijpers P, Karyotaki E, Auerbach RP, Kessler RC, Bruffaerts R, Berking M, Ebert DD. Internet interventions for mental health in university students: a systematic review and meta-analysis. Int J Methods Psychiatr Res. 2019 Jun;28(2):e1759. doi: 10.1002/mpr.1759.

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe