How one small text change in a study document can impact recruitment rates and follow-up completions

Alexandra Godinho, Christina Schell, John A Cunningham, Alexandra Godinho, Christina Schell, John A Cunningham

Abstract

Background: The validity and reliability of longitudinal research is highly dependent on the recruitment and retention of representative samples. Various strategies have been developed and tested for improving recruitment and follow-up rates into health-behavioural research, but few have examined the role of linguistic choices and study document readability on participation rates. This study examined the impact of one small text change, assigning an inappropriate or grade-8 reading level password for intervention access, on participation rates and attrition in an online alcohol intervention trial.

Methods: Participants were recruited into an online alcohol intervention study using Amazon's Mechanical Turk via a multi-step recruitment process which required participants to log into a study portal using a pre-assigned password. Passwords were qualitatively coded as grade-8 and/or inappropriate for use within a professional setting. Separate logistic regressions examined which demographic, clinical characteristics, and password categorizations were most strongly associated with recruitment rates and follow-up completions.

Results: Inappropriate passwords were a barrier for recruitment among participants with post-secondary education as compared to those with less education (p = 0.044), while grade-8 passwords appeared to significantly facilitate the completion of 6-month follow-ups (p = 0.005).

Conclusions: Altogether, these findings suggest that some linguistic choices may play an important role in recruitment, while others, such as readability, may have longer-term effects on follow-up rates and attrition. Possible explanations for the findings, as well as, sample selection biases during recruitment and follow-up are discussed. Limitations of the study are stated and recommendations for researchers are provided.

Trial registration: ClinicalTrials.gov NCT02977026. Registered 27 Nov 2016.

Keywords: Attrition; Follow-up rates; Recruitment; Research methodology; Research participation.

Conflict of interest statement

The authors declare that they have no competing interests.

© 2019 The Authors.

Figures

Fig. 1
Fig. 1
Trial consort diagram.

References

    1. Barry A.E. How attrition impacts the internal and external validity of longitudinal research. J. Sch. Health. 2005;75(7):267–270.
    1. Bonevski B., Randell M., Paul C., Chapman K., Twyman L., Bryant J. Reaching the hard-to-reach: a systematic review of strategies for improving health and medical research with socially disadvantaged groups. BMC Med. Res. Methodol. 2014;14(42):1–29.
    1. Booker C.L., Harding S., Benzeval M. A systematic review of the effect of retention methods in population-based cohort studies. BMC Public Health. 2011;11(249):1–12.
    1. Capaldi D., Patterson G.R. An approach to the problem of recruitment and retention rates for longitudinal research. Behav. Assess. 1987;9(2):19–177.
    1. Cunningham J.A., Godinho A., Kushnir V. Using Mechanical Turk to recruit participants for Internet intervention research: experience from recruitment for four trials. BMC Med. Res. Methodol. 2017;17(1):156.
    1. Cunningham J.A., Godinho A., Kushnir V. Can Amazon's mechanical Turk be used to recruit participants for internet intervention trials? A pilot study involving a randomized control trial of a brief online intervention for hazardous alcohol use. Internet Interv. 2017;10:12–16.
    1. Cunningham JA, Godinho A, Bertholet N. Outcomes of two randomized controlled trials, employing participants recruited through Mechanical Turk, of Internet interventions targeting unhealthy alcohol use. under review. 2019.
    1. David M.C., Alati R., Ware R.S., Kinner S.A. Attrition in a longitudinal study with hard-to-reach participants was reduced by ongoing contact. J. Clin. Epidemiol. 2013;66:575–581.
    1. Davis T.C., Jackson R.H., George R.B., Long S.W., Talley D., Murphy P.W. Reading ability in patients in substance misuse treatment centers. International Journal of Addictions. 1993;28:571–582.
    1. Ennis L., Wykes T. Sense and readability: participant information sheets for research studies. Br. J. Psychiatry. 2016;208(2):189–194.
    1. Foe G., Larson E.L. Reading level and comprehension of research consent forms: an integrative review. Journal of Empirical Research on Human Research Ethics. 2016;11(1):31–46.
    1. Friedman D.B., Kim S., Tanner A., Bergeron C.D., Foster C., General K. How are we communicating about clinical trials? An assessment of the content and readability of recruitment resources. Contemporary Clinical Trials. 2014;38:275–283.
    1. Hall A.E., Sanson-Fisher R.W., Lynagh M.C., Threlfall T., D'Este C.A. Format and readability of an enhanced invitation letter did not affect participation rates in a cancer registry-based study: a randomized controlled trial. J. Clin. Epidemiol. 2013;66:85–94.
    1. Hamnes B., van Hustings-Eijk Y., Primdahl J. Readability of patient information and consent documents in rheumatological studies. BMC Medical Ethics. 2016;17(42):1–9.
    1. Hosmer D., Lemeshow S. Wiley; New York, NY: 1989. Applied Logistic Regression.
    1. Hunter J., Corcoran K., Leeder S., Phelps K. Appealing to altruism is not enough: motivators for participating in health services research. Journal of Empirical Research on Human Research Ethics. 2012;7(3):84–90.
    1. Johnson M.E., Fisher D.G., Davis D.C., Cagle H.H. Reading abilities of drug users in Anchorage. Alaska. Journal of Drug Education. 1995;25:73–80.
    1. Kim H., Hodgins D. Reliability and validity of data obtained from alcohol, cannabis, and gambling populations on Amazon's Mechanical Turk. Psychol. Addict. Behav. 2017;31(1):85–94.
    1. Kristman V., Manno M., Cote P. Loss to follow-up in cohort studies: how much is too much? Eur. J. Epidemiol. 2004;19:751–760.
    1. Latkin C., Edwards C., Davey-Rothwell M., Tobin K. The relationship between social desirability bias and self-reports of health, substance use, and social network factors among urban substance users in Baltimore. Maryland. Addictive Behaviors. 2017;73:133–136.
    1. Man M.-S., Rick J., Bower P. Improving recruitment to a study of telehealth management for long-term condition in primary care: two embedded, randomised controlled trials of optimised patient information materials. Trials. 2015;16:309.
    1. Perinelli E., Gremigni P. Use of social desirability scales in clinical psychology: a systematic review. J. Clin. Psychol. 2016;72(6):534–551.
    1. Scott C.K. A replicable model for achieving over 90% follow-up rates in longitudinal studies of substance abusers. Drug Alcohol Depend. 2004;74(1):21–36.
    1. Smith L.J., McNamara P.J., King A.C. Optimizing follow-up and study retention in the 21st century: advances from the front line in alcohol and tobacco research. Drug Alcohol Depend. 2017;175:171–178.
    1. Strekalova Y.A. Defining research: the effect of linguistic choices on the intentions to participate in clinical research. Clin. Nurs. Res. 2018;27(7):790–799.
    1. Strickland J.C., Stoops W.W. Feasibility, acceptability, and validity of crowdsourcing for collecting longitudinal alcohol use data. J. Exp. Anal. Behav. 2018;110(1):136–153.
    1. Subbaraman M.S., Lauder A.B., Ritter L.A., Stunz A., Kaskutas L.A. Multisource recruitment strategies for advancing addiction recovery research beyond treated samples. Journal of Community Psychology. 2015;43(5):560–575.
    1. Thomson C.L., Thomson C.L., Morley K.C., Teesson M., Sannibale C., Haber P.S. Issues with recruitment to randomised controlled trials in the drug and alcohol field: a literature review and Australian case study. Drug and Alcohol Review. 2008;27(2):115–122.
    1. Thornton L.K., Harris K., Baker A.L., Johnson M., Kay-Lambkin F.J. Recruiting for addictions research via Facebook. Drug and Alcohol Review. 2016;35:494–502.
    1. Trauth J.M., Musa D., Siminoff K., Jewell I.K., Ricci E. Public attitudes regarding willingness to participate in medical research studies. Journal of Health and Social Policy. 2000;12(2):23–43.
    1. van Gelder M.M.H.J., Vlenterie R., IntHout J., Engelen L.J.L.P.G., Vrieling A., van de Belt T.H. Most response-inducing strategies do not increase participation in observational studies: a systematic review and meta-analysis. J. Clin. Epidemiol. 2018;99:1–13.
    1. von Allmen R.S., Weiss S., Tevaearai H.T., Kuemmerli C., Tinner C., Carrel T.P. Completeness of follow-up determines validity of study findings: results of a prospective repeated measures cohort study. PLoS One. 2015;10(10)
    1. Zoran B., Gauss C.H., Williams D.K., Hosmer D.W. Purposeful selection of variables in logistic regression. Source Code for Biology and Medicine. 2008;3

Source: PubMed

스폰서 및 공동 작업자

건강 상태

약물 개입

3
구독하다