Fidelity Assessment Checklist Development for Community Nursing Research in Early Childhood

Lubna Anis, Karen M Benzies, Carol Ewashen, Martha J Hart, Nicole Letourneau, Lubna Anis, Karen M Benzies, Carol Ewashen, Martha J Hart, Nicole Letourneau

Abstract

Nurses play an important role in promoting positive childhood development via early interventions intended to support parenting. Despite recognizing the need to deliver vital parenting programs, monitoring fidelity has largely been ignored. Fidelity refers to the degree to which healthcare programs follow a well-defined set of criteria specifically designed for a particular program model. With increasing demands for early intervention programs to be delivered by non-specialists, rigorous yet pragmatic strategies for maintaining fidelity are needed. This paper describes the step-by-step development and evaluation of a program fidelity measure, using the Attachment and Child Health (ATTACH™) parenting program as an exemplar. The overall quality index for program delivery varied between "very good" to "excellent," with a mean of 4.3/5. Development of checklists like the ATTACH™ fidelity assessment checklist enables the systematic evaluation of program delivery and identification of therapeutic components that enable targeted efforts at improvement. In future, research should examine links between program fidelity and targeted outcomes to ascertain if increased fidelity scores yield more favorable effects of parenting programs.

Keywords: ATTACH; checklist; early childhood; fidelity tools; intervention; measure; parenting; program.

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2021 Anis, Benzies, Ewashen, Hart and Letourneau.

References

    1. Knauer HA, Ozer EJ, Dow WH, Fernald LC. Parenting quality at two developmental periods in early childhood and their association with child development. Early Child Res Q. (2019) 47:396–404. 10.1016/j.ecresq.2018.08.009
    1. Kopala-Sibley DC, Cyr M, Finsaas MC, Orawe J, Huang A, Tottenham N, et al. . Early childhood parenting predicts late childhood brain functional connectivity during emotion perception and reward processing. Child Dev. (2020) 91:110–28. 10.1111/cdev.13126
    1. Kemp L, Bruce T, Elcombe EL, Anderson T, Vimpani G, Price A, et al. . Quality of delivery of “right@ home”: implementation evaluation of an Australian sustained nurse home visiting intervention to improve parenting and the home learning environment. PLoS ONE. (2019) 14:e0215371. 10.1371/journal.pone.0215371
    1. Reticena KDO, Yabuchi V. D. N. T., Gomes MFP, Siqueira LDE, Abreu C. P. D., Fracolli LA, et al. . Role of nursing professionals for parenting development in early childhood: a systematic review of scope. Rev Lat Am Enfermagem. (2019) 27:e3213. 10.1590/1518-8345.3031.3213
    1. Schoenwald SK, Garland AF. A review of treatment adherence measurement methods. Psychol Assess. (2013) 25:146–156. 10.1037/a0029715
    1. Seay KD, Byers K, Feely M, Lanier P, Maguire-Jack K, McGill T. Scaling up: replicating promising interventions with fidelity. In: Daro D, Cohn Donnelly A, Huang L, Powell B, editors. Advances in Child Abuse Prevention Knowledge: The Perspective of New Leadership. New York, NY: Springer; (2015). p. 179–201.
    1. Biel CH, Buzhardt J, Brown JA, Romano MK, Lorio CM, Windsor KS, et al. . Language interventions taught to caregivers in homes and classrooms: a review of intervention and implementation fidelity. Early Child Res Q. (2019) 50:140–56. 10.1016/j.ecresq.2018.12.002
    1. Bond GR, Drake RE. Assessing the fidelity of evidence-based practices: history and current status of a standardized measurement methodology. Adm Policy Ment Health. (2019) 47:874–84. 10.1007/s10488-019-00991-6
    1. Dusenbury L, Brannigan R, Falco M, Hansen WB. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. (2003) 18:237–56. 10.1093/her/18.2.237
    1. Goense PB, Boendermaker L, van Yperen. T. Measuring treatment integrity: use of and experience with measurements in child and youth care organizations. J Behav Health Serv Res. (2018) 45:469–88. 10.1007/s11414-018-9600-4
    1. Lange AM, van der Rijken RE, Delsing MJ, Busschbach JJ, Scholte RH. Development of therapist adherence in relation to treatment outcomes of adolescents with behavioral problems. J Clin Child Adolesc Psychol. (2019) 48 (Supp. 1):S337–46. 10.1080/15374416.2018.1477049
    1. Breitenstein SM, Gross D, Garvey CA, Hill C, Fogg L, Resnick B. Implementation fidelity in community-based interventions. Res Nurs Health. (2010) 33:164–73. 10.1002/nur.20373
    1. Shonkoff JP, Richmond J, Levitt P, Bunge SA, Cameron JL, Duncan GJ, et al. . From Best Practices to Breakthrough Impacts a Science-Based Approach to Building a More Promising Future for Young Children and Families. Cambirdge, MA: Harvard University, Center on the Developing Child; (2016).
    1. Bettencourt AF, Gross D, Breitenstein S. Evaluating implementation fidelity of a school-based parenting program for low-income families. J School Nurs. (2018) 35:325–36. 10.1177/1059840518786995
    1. Wilson KR, Havighurst SS, Harley AE. Tuning in to kids: an effectiveness trial of a parenting program targeting emotion socialization of preschoolers. J Family Psychol. (2012) 26:56–65. 10.1037/a0026480
    1. Anis L, Letourneau NL, Benzies K, Ewashen C, Hart MJ. Effect of the attachment and child health parent training program on parent–child interaction quality and child development. Can J Nurs Res. (2020) 52:157–68. 10.1177/0844562119899004
    1. Letourneau N, Anis L, Ntanda H, Novick J, Steele M, Steele H, et al. . Attachment & child health (ATTACH) pilot trials: effect of parental reflective function intervention for families affected by toxic stress. Infant Ment Health J. (2020) 41:445–62. 10.1002/imhj.21833
    1. Borrelli B. The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. J Public Health Dent. (2011) 71:S52–63. 10.1111/j.1752-7325.2011.00233.x
    1. McLeod BD, Southam-Gerow MA, Jensen-Doss A, Hogue A, Kendall PC, Weisz JR. Benchmarking treatment adherence and therapist competence in individual cognitive-behavioral treatment for youth anxiety disorders. J Clin Child Adolesc Psychol. (2019) 48 (Supp. 1):S234–46. 10.1080/15374416.2017.1381914
    1. Carroll KM, Nich C, Sifry RL, Nuro KF, Frankforter TL, Ball SA, et al. . A general system for evaluating therapist adherence and competence in psychotherapy research in the addictions. Drug Alcohol Depend. (2000) 57:225–38. 10.1016/S0376-8716(99)00049-6
    1. Santacroce SJ, Maccarelli LM, Grey M. Intervention fidelity. Nurs Res. (2004) 53:63–6. 10.1097/00006199-200401000-00010
    1. Reed D, Titler MG, Dochterman JM, Shever LL, Kanak M, Picone DM. Measuring the dose of nursing intervention. Int J Nurs Terminol Classifications. (2007) 18:121–30. 10.1111/j.1744-618X.2007.00067.x
    1. Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. . Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. (2014) 348:g1687. 10.1136/bmj.g1687
    1. Dumas JE, Lynch AM, Laughlin JE, Smith EP, Prinz RJ. Promoting intervention fidelity: Conceptual issues, methods, and preliminary results from the EARLY ALLIANCE prevention trial. Am J Prev Med. (2001) 20:38–47. 10.1016/S0749-3797(00)00272-5
    1. Resnick B, Bellg AJ, Borrelli B, De Francesco C, Breger R, Hecht J, et al. . Examples of implementation and evaluation of treatment fidelity in the BCC studies: where we are and where we need to go. Ann Behav Med. (2005) 29:46. 10.1207/s15324796abm2902s_8
    1. Resnick B, Inguito P, Orwig D, Yahiro JY, Hawkes W, Werner M, et al. . Treatment fidelity in behavior change research: a case example. Nurs Res. (2005) 54:139–143. 10.1097/00006199-200503000-00010
    1. Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS, Ory M, et al. . Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH Behavior Change Consortium. Health Psychol. (2004) 23:443–51. 10.1037/0278-6133.23.5.443
    1. Radziewicz RM, Rose JH, Bowman KF, Berila RA, O'Toole EE, Given B. Establishing treatment fidelity in a coping and communication support telephone intervention for aging patients with advanced cancer and their family caregivers. Cancer Nurs. (2009) 32:193–202. 10.1097/NCC.0b013e31819b5abe
    1. Garbacz LL, Brown DM, Spee GA, Polo AJ, Budd KS. Establishing treatment fidelity in evidence-based parent training programs for externalizing disorders in children and adolescents. Clin Child Fam Psychol Rev. (2014) 17:230–47. 10.1007/s10567-014-0166-2
    1. Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual framework for implementation fidelity. Implement Sci. (2007) 2:40. 10.1186/1748-5908-2-40
    1. Dane AV, Schneider BH. Program integrity in primary and early secondary prevention: are implementation effects out of control? Clin Psychol Rev. (1998) 18:23–45. 10.1016/S0272-7358(97)00043-3
    1. Moncher FJ, Prinz RJ. Treatment fidelity in outcome studies. Clin Psychol Rev. (1991) 11:247–66. 10.1016/0272-7358(91)90103-2
    1. Perepletchikova F, Kazdin AE. Treatment integrity and therapeutic change: issues and research recommendations. Clin Psychol Sci Pract. (2005) 12:365–83. 10.1093/clipsy.bpi045
    1. Stein KF, Sargent JT, Rafaels N. Intervention research: establishing fidelity of the independent variable in nursing clinical trials. Nurs Res. (2007) 56:54–62. 10.1097/00006199-200701000-00007
    1. Feely M, Seay KD, Lanier P, Auslander W, Kohl PL. Measuring fidelity in research studies: a field guide to developing a comprehensive fidelity measurement system. Child Adolesc Soc Work J. (2018) 35:139–52. 10.1007/s10560-017-0512-6
    1. Faulkner MS. Intervention fidelity: ensuring application to practice for youth and families. J Special Pediatr Nurs. (2012) 17:33–40. 10.1111/j.1744-6155.2011.00305.x
    1. Chartier M, Enns JE, Nickel NC, Campbell R, Phillips-Beck W, Sarkar J, et al. . The association of a paraprofessional home visiting intervention with lower child maltreatment rates in First Nation families in Canada: a population-based retrospective cohort study. Child Youth Serv Rev. (2020) 108:104675. 10.1016/j.childyouth.2019.104675
    1. Lorber MF, Olds DL, Donelan-McCall N. The impact of a preventive intervention on persistent, cross-situational early onset externalizing problems. Prevent Sci. (2019) 20:684–94. 10.1007/s11121-018-0973-7
    1. Gawande A. The Checklist Manifesto: How to Get Things Right. New York, NY: Metropolitan Books; (2009).
    1. Perry AG, Potter PA, Ostendorf W. Skills Performance Checklists for Clinical Nursing Skills and Techniques. 8th ed. Maryland Heights, MO: Elsevier Mosby; (2014).
    1. Wilson C. Credible Checklists and Quality Questionnaires: A User-Centered Design Method. Waltham, MA: Morgan Kaufman; (2013).
    1. Becqué YN, Rietjens JA, van Driel AG, van der Heide A, Witkamp E. Nursing interventions to support family caregivers in end-of-life care at home: a systematic narrative review. Int J Nurs Stud. (2019) 97:28–39. 10.1016/j.ijnurstu.2019.04.011
    1. Eymard AS, Altmiller G. Teaching nursing students the importance of treatment fidelity in intervention research: students as interventionists. J Nurs Educ. (2016) 55:288–91. 10.3928/01484834-20160414-09
    1. Chiapa A, Smith JD, Kim H, Dishion TJ, Shaw DS, Wilson MN. The trajectory of fidelity in a multiyear trial of the family check-up predicts change in child problem behavior. J Consult Clin Psychol. (2015) 83:1006. 10.1037/ccp0000034
    1. Eames C, Daley D, Hutchings J, Hughes JC, Jones K, Martin P, et al. . The leader observation tool: a process skills treatment fidelity measure for the incredible years parenting programme. Child Care Health Dev. (2008) 34:391–400. 10.1111/j.1365-2214.2008.00828.x
    1. Rixon L, Baron J, McGale N, Lorencatto F, Francis J, Davies A. Methods used to address fidelity of receipt in health intervention research: a citation analysis and systematic review. BMC Health Serv Res. (2016) 16:663. 10.1186/s12913-016-1904-6
    1. Smith JD, Dishion TJ, Shaw DS, Wilson MN. Indirect effects of fidelity to the family check-up on changes in parenting and early childhood problem behaviors. J Consult Clin Psychol. (2013) 81:962. 10.1037/a0033950
    1. Alexander K. Integrative review of the relationship between mindfulness-based parenting interventions and depression symptoms in parents. J Obstetr Gynecol Neonatal Nurs. (2018) 47:184–90. 10.1016/j.jogn.2017.11.013
    1. Friedmutter R. The effectiveness of mindful parenting interventions: a meta analysis. In: Dissertation Abstracts International: Section B: The Sciences and Engineering. New York, NY, (2016). p. 76.
    1. Townshend K, Jordan Z, Stephenson M, Tsey K. The effectiveness of mindful parenting programs in promoting parents' and children's wellbeing: a systematic review. JBI Database Syst Rev Implement Rep. (2016) 14:139–80. 10.11124/JBISRIR-2016-2314
    1. Lombardo MV, Baron-Cohen S. The role of the self in mindblindness in autism. Conscious Cogn. (2011) 20:130–40. 10.1016/j.concog.2010.09.006
    1. Elliott R, Bohart AC, Watson JC, Greenberg LS. Empathy. Psychotherapy. (2011) 48:43–9. 10.1037/a0022187
    1. Oppenheim D, Koren-Karie N, Dolev S, Yirmiya N. Maternal insightfulness and resolution of the diagnosis are associated with secure attachment in preschoolers with autism spectrum disorders. Child Dev. (2009) 80:519–27. 10.1111/j.1467-8624.2009.01276.x
    1. Meins E, Fernyhough C, Wainwright R, Das Gupta M, Fradley E, Tuckey M. Maternal mind–mindedness and attachment security as predictors of theory of mind understanding. Child Dev. (2002) 73:1715–26. 10.1111/1467-8624.00501
    1. Bywater T, Gridley N, Berry V, Blower S, Tobin K. The parent programme implementation checklist (PPIC): the development and testing of an objective measure of skills and fidelity for the delivery of parent programmes. Child Care Pract. (2019) 25:281–309. 10.1080/13575279.2017.1414031
    1. Proctor KB, Brestan-Knight E. Evaluating the use of assessment paradigms for preventive interventions: a review of the triple p—positive parenting program. Child Youth Serv Rev. (2016) 62:72–82. 10.1016/j.childyouth.2016.01.018
    1. Forgatch MS, Patterson GR, DeGarmo DS. Evaluating fidelity: predictive validity for a measure of competent adherence to the Oregon model of parent management training. Behav Ther. (2005) 36:3–13. 10.1016/S0005-7894(05)80049-8
    1. Forgatch MS, DeGarmo DS, Beldavs ZG. An efficacious theory-based intervention for stepfamilies. Behav Ther. (2005) 36:357–65. 10.1016/S0005-7894(05)80117-0
    1. Thijssen J, Albrecht G, Muris P, de Ruiter C. Treatment fidelity during therapist initial training is related to subsequent effectiveness of parent management training—Oregon model. J Child Family Stud. (2017) 26:1991–9. 10.1007/s10826-017-0706-8
    1. Oats RG, Cross WF, Mason WA, Casey-Goldstein M, Thompson RW, Hanson K, et al. . Implementation assessment of widely used but understudied prevention programs: an illustration from the common sense parenting trial. Eval Program Plann. (2014) 44:89–97. 10.1016/j.evalprogplan.2014.02.002
    1. Knoche LL, Sheridan SM, Edwards CP, Osborn AQ. Implementation of a relationship-based school readiness intervention: a multidimensional approach to fidelity measurement for early childhood. Early Child Res Q. (2010) 25:299–313. 10.1016/j.ecresq.2009.05.003
    1. Caron E, Bernard K, Dozier M. In vivo feedback predicts parent behavior change in the attachment and biobehavioral catch-up intervention. J Clin Child Adolesc Psychol. (2018) 47 (Supp. 1):S35–46. 10.1080/15374416.2016.1141359
    1. Barnett ML, Niec LN, Acevedo-Polakovich ID. Assessing the key to effective coaching in parent–child interaction therapy: the therapist-parent interaction coding system. J Psychopathol Behav Assess. (2014) 36:211–23. 10.1007/s10862-013-9396-8
    1. Barnett ML, Niec LN, Peer SO, Jent JF, Weinstein A, Gisbert P, et al. . Successful therapist–parent coaching: how in vivo feedback relates to parent engagement in parent–child interaction therapy. J Clin Child Adolesc Psychol. (2017) 46:895–902. 10.1080/15374416.2015.1063428
    1. Forgatch MS, Patterson GR, Gewirtz AH. Looking forward: the promise of widespread implementation of parent training programs. Perspect Psychol Sci. (2013) 8:682–94. 10.1177/1745691613503478
    1. Song MK, Happ MB, Sandelowski M. Development of a tool to assess fidelity to a psycho-educational intervention. J Adv Nurs. (2010) 66:673–82. 10.1111/j.1365-2648.2009.05216.x
    1. Caron E, Weston-Lee P, Haggerty D, Dozier M. Community implementation outcomes of attachment and biobehavioral catch-up. Child Abuse Negl. (2016) 53:128–37. 10.1016/j.chiabu.2015.11.010
    1. Miller WR, Moyers TB, Ernst D, Amrhein P. Manual for the Motivational Interviewing Skill Code, Vol. 2007. Albuquerque, NM: University of New Mexico; (2000).
    1. Miller WR, Rollnick S. Motivational Interviewing: Preparing People to Change. New York, NY: The Guilford Press; (2002).
    1. Miller WR, Moyers TB, Arciniega L, Ernst D, Forcehimes A. Training, supervision and quality monitoring of the COMBINE Study behavioral interventions. J Stud Alcohol Drugs. (2005) 66(Suppl. 15):188–95. 10.15288/jsas.2005.s15.188
    1. Bragstad LK, Bronken BA, Sveen U, Hjelle EG, Kitzmüller G, Martinsen R, et al. . Implementation fidelity in a complex intervention promoting psychosocial well-being following stroke: an explanatory sequential mixed methods study. BMC Med Res Methodol. (2019) 19:59. 10.1186/s12874-019-0694-z
    1. Hurley JJ. Social validity assessment in social competence interventions for preschool children: a review. Topics Early Childhood Special Educ. (2012) 32:164–174. 10.1177/0271121412440186
    1. Leko MM. The value of qualitative methods in social validity research. Remed Special Educ. (2014) 35:275–86. 10.1177/0741932514524002
    1. Beckstead JW. Content validity is naught. Int J Nurs Stud. (2009) 46:1274–83. 10.1016/j.ijnurstu.2009.04.014
    1. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropract Med. (2016) 15:155–63. 10.1016/j.jcm.2016.02.012
    1. Miller WR, Moyers TB, Ernst D, Amrhein P. Manual for the Motivational Interviewing Skill Code (MISC). Unpublished Manuscript. Albuquerque: Center on Alcoholism, Substance Abuse and Addictions, University of New Mexico; (2003).
    1. Miller WR, Rollnick S. Motivational Interviewing: Helping People Change. New York, NY: Guilford Press; (2012).
    1. Madson MB, Campbell TC. Measures of fidelity in motivational enhancement: a systematic review. J Subst Abuse Treat. (2006) 31:67–73. 10.1016/j.jsat.2006.03.010
    1. Orwin RG. Assessing program fidelity in substance abuse health services research. Addiction. (2000) 95:309–27. 10.1080/09652140020004250
    1. Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to Their Development and Use. New York, NY: Oxford University Press; (2003).
    1. Tappin DM, McKay C, McIntyre D, Gilmour WH, Cowan S, Crawford F, et al. . A practical instrument to document the process of motivational interviewing. Behav Cogn Psychother. (2000) 28:17–32. 10.1017/S1352465800000035
    1. Al-Ubaydli O, List JA, Suskind D. The Science of Using Science: Towards an Understanding of the Threats to Scaling Experiments. Working Paper 25848. National Bureau of Economic Research (2019). Available online at: (accessed October 20, 2020).
    1. Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. Implementation Research: A Synthesis of the Literature. University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231) (2005).
    1. Hulleman CS, Cordray DS. Moving from the lab to the field: the role of fidelity and achieved relative intervention strength. J Res Edguc Effectiv. (2009) 2:88–110. 10.1080/19345740802539325

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe