Generalizing findings from a randomized controlled trial to a real-world study of the iLookOut, an online education program to improve early childhood care and education providers' knowledge and attitudes about reporting child maltreatment

Chengwu Yang, Carlo Panlilio, Nicole Verdiglione, Erik B Lehman, Robert M Hamm, Richard Fiene, Sarah Dore, David E Bard, Breanna Grable, Benjamin Levi, Chengwu Yang, Carlo Panlilio, Nicole Verdiglione, Erik B Lehman, Robert M Hamm, Richard Fiene, Sarah Dore, David E Bard, Breanna Grable, Benjamin Levi

Abstract

In recent years, real-world studies (RWS) are gaining increasing interests, because they can generate more realistic and generalizable results than randomized controlled clinical trials (RCT). In 2017, we published a RCT in 741 early childhood care and education providers (CCPs). It is the Phase I of our iLookOut for Child Abuse project (iLookOut), an online, interactive learning module about reporting suspected child maltreatment. That study demonstrated that in a RCT setting, the iLookOut is efficient at improving CCPs' knowledge of and attitudes towards child maltreatment reporting. However, the generalizability of that RCT's results in a RWS setting remains unknown. To address this question, we design and conduct this large RWS in 11,065 CCPs, which is the Phase II of the iLookOut. We hypothesize replication of the earlier RCT findings, i.e., the iLookOut can improve CCPs' knowledge of and attitudes toward child maltreatment reporting in a real world setting. In addition, this RWS also explores whether demographic factors affect CCPs' performance. Results of this RWS confirmed the generalizability of the previous RCT's results in a real world setting. It yielded similar effect sizes for knowledge and attitudes as were found in the earlier RCT. Cohen's d for knowledge improvement was 0.95 in that RCT, 0.96 in this RWS; Cohen's d for attitude improvement was 0.98 in that RCT, 0.80 in this RWS. Also, we found several significant differences in knowledge and attitude improvement with regard to age, race, education, and employment status. In conclusion, iLookOut improves knowledge and attitudes of CCPs about child maltreatment prevention and reporting in a real-world setting. The generalizability of the initial RCT findings to this RWS provides strong evidence that the iLookout will be effective in other real world settings. It can be a useful model for other interventions aimed at preventing child maltreatment. Clinical trial registration for the original RCT: NCT02225301 (ClinicalTrials.gov Identifier).

Conflict of interest statement

The authors have declared that no competing interests exist.

References

    1. Kaptchuk TJ. The double-blind, randomized, placebo-controlled trial: gold standard or golden calf? J Clin Epidemiol. 2001;54(6):541–9. 10.1016/s0895-4356(00)00347-4
    1. Garrison LP Jr, Neumann PJ, Erickson P, Marshall D, Mullins CD. Using real‐world data for coverage and payment decisions: The ISPOR real‐world data task force report. Value in Health. 2007;10(5):326–35. 10.1111/j.1524-4733.2007.00186.x
    1. Sherman RE, Anderson SA, Dal Pan GJ, Gray GW, Gross T, Hunter NL, et al. Real-World Evidence—What Is It and What Can It Tell Us? N Engl J Med. 2016;375(23):2293–7. 10.1056/NEJMsb1609216
    1. Administration USFD. Use of real-world evidence to support regulatory decision-making for medical devices. Guidance for industry and food and drug administration staff. Docket Number: FDA-2016-D-2153. 2017.
    1. Sun X, Tan J, Tang L, Guo JJ, Li X. Real world evidence: experience and lessons from China. BMJ. 2018;360:j5262 10.1136/bmj.j5262
    1. Corrigan-Curay J, Sacks L, Woodcock J. Real-world evidence and real-world data for evaluating drug safety and effectiveness. JAMA. 2018;320(9):867–8. 10.1001/jama.2018.10136
    1. Food U, Administration D. Framework for FDA’s real-world evidence program. 2018.
    1. Klonoff DC. The Expanding Role of Real-World Evidence Trials in Health Care Decision Making. Journal of Diabetes Science and Technology. 2019:1932296819832653.
    1. Evans K. Real World Evidence: Can We Really Expect It to Have Much Influence? Drugs—Real World Outcomes. 2019;6(2):43–5. 10.1007/s40801-019-0155-3
    1. Calvert MJ, O’Connor DJ, Basch EM. Harnessing the patient voice in real-world evidence: the essential role of patient-reported outcomes. Nature Reviews Drug Discovery. 2019;18(5).
    1. Mathews B, Yang C, Lehman EB, Mincemoyer C, Verdiglione N, Levi BH. Educating early childhood care and education providers to improve knowledge and attitudes about reporting child maltreatment: A randomized controlled trial. PLoS ONE. 2017;12(5):e0177777 10.1371/journal.pone.0177777
    1. Curry SJ, Krist AH, Owens DK, Barry MJ, Caughey AB, Davidson KW, et al. Interventions to prevent child maltreatment: US Preventive Services Task Force Recommendation Statement. JAMA. 2018;320(20):2122–8. 10.1001/jama.2018.17772
    1. Kohl PL, Jonson-Reid M, Drake B. Time to leave substantiation behind: Findings from a national probability study. Child Maltreatment. 2009;14(1):17–26. 10.1177/1077559508326030
    1. Stoltenborgh M, Bakermans‐Kranenburg MJ, van IJzendoorn MH, Alink LR. Cultural–geographical differences in the occurrence of child physical abuse? A meta‐analysis of global prevalence. International Journal of Psychology. 2013;48(2):81–94. 10.1080/00207594.2012.697165
    1. Alvarez KM, Kenny MC, Donohue B, Carpin KM. Why are professionals failing to initiate mandated reports of child maltreatment, and are there any empirically based training programs to assist professionals in the reporting process? Aggression and Violent Behavior. 2004;9(5):563–78.
    1. Dinehart L, Kenny MC. Knowledge of child abuse and reporting practices among early care and education providers. Journal of Research in Childhood Education. 2015;29(4):429–43.
    1. IOM, NRC. New directions in child abuse and neglect research. The National Academies Press; Washington, DC; 2014.
    1. McGrath P, Cappelli M, Wiseman D, Khalil N, Allan B. Teacher awareness program on child abuse: a randomized controlled trial. Child Abuse & Neglect. 1987;11(1):125–32.
    1. Khan A, Rubin D, Winnik G. Evaluation of the mandatory child abuse course for physicians: do we need to repeat it? Public Health. 2005;119(7):626–31. 10.1016/j.puhe.2004.10.010
    1. Kenny MC. Web-based training in child maltreatment for future mandated reporters. Child Abuse & Neglect. 2007;31(6):671–8.
    1. Kenny MC, Lopez-Griman AM, Donohue B. Development and initial evaluation of a cost-effective, internet-based program to assist professionals in reporting suspected child maltreatment. Journal of Child & Adolescent Trauma. 2017;10(4):385–93.
    1. Von Elm E, Altman D, Egger M, Pocock S. G? tzsche PC, Vandenbroucke JP: The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. PLoS Med. 2007;4(10):e296 10.1371/journal.pmed.0040296
    1. Walsh K, Rassafiani M, Mathews B, Farrell A, Butler D. Exploratory factor analysis and psychometric evaluation of the teacher reporting attitude scale for child sexual abuse. J Child Sex Abus. 2012;21(5):489–506. 10.1080/10538712.2012.689423
    1. Cohen J. A power primer. Psychological bulletin. 1992;112(1):155 10.1037//0033-2909.112.1.155
    1. Piantadosi S. Clinical trials: a methodologic perspective: John Wiley & Sons; 2017.
    1. Nuzzo R. Scientific method: statistical errors. Nature News. 2014;506(7487):150.
    1. Wasserstein RL, Lazar NA. The ASA's Statement onp-Values: Context, Process, and Purpose. The American Statistician. 2016;70(2):129–33.
    1. Yang C, Vrana KE. Rescuing Suboptimal Patient-Reported Outcome Instrument Data in Clinical Trials: A New Strategy. Healthcare (Basel). 2018;6(1).
    1. Rothwell PM. Subgroup analysis in randomised controlled trials: importance, indications, and interpretation. The Lancet. 2005;365(9454):176–86.

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe