Development and Testing of a Novel Measure to Assess Fidelity of Implementation: Example of the Mini-AFTERc Intervention

Nathalie Georgia Brandt, Calum Thomas McHale, Gerald Michael Humphris, Nathalie Georgia Brandt, Calum Thomas McHale, Gerald Michael Humphris

Abstract

Background: Fidelity of implementation (FOI) reflects whether an intervention was implemented in clinical practice according to the originally developed manual and is a key aspect in understanding intervention effectiveness. To illustrate this process of developing a fidelity measure, this study uses the Mini-AFTERc, a brief psychological intervention aimed at managing breast cancer patients' fear of cancer recurrence, as an example.

Objectives: To illustrate the development of an FOI measure through (1) applying this process to the Mini-AFTERc intervention, by including the design of a scoring system and rating criteria; (2) content validating the FOI measure using thematic framework analysis as a qualitative approach; (3) testing consistency of the FOI measure using interrater reliability.

Methods: The FOI measure was developed, its scoring system modified and the rating criteria defined. Thematic framework analysis was conducted to content validate the FOI measure using nine intervention discussions between four specialist cancer nurses and four breast cancer patients, and one simulated breast cancer patient. Intraclass-correlation was conducted to assess interrater reliability.

Results: The qualitative findings suggested that the Mini-AFTERc FOI measure has content validity as it was able to measure all five components of the Mini-AFTERc intervention. The interrater reliability suggested a moderate to excellent degree of reliability among three raters, r ICC = 0.84, 95% CI [0.51, 0.96].

Conclusion: The study has illustrated the steps that an FOI measure can be developed through a systematic approach applied to the Mini-AFTERc intervention. The FOI measure was found to have content validity and was consistently applied, independently, by three researchers familiar with the Mini-AFTERc intervention. Future studies should determine whether similar levels of interrater reliability can be obtained by distributing written and/or video instructions to researchers who are unfamiliar with the FOI measure, using a larger sample. Employing developed and validated FOI measures such as the one presented for the Mini-AFTERc would facilitate implementation of interventions in the FCR field in clinical practice as intended.

Clinical trial registration: www.ClinicalTrials.gov, identifier: NCT03763825.

Keywords: Mini-AFTERc intervention; breast cancer; fear of cancer recurrence; fidelity; implementation.

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2020 Brandt, McHale and Humphris.

Figures

FIGURE 1
FIGURE 1
The procedure of the Mini-AFTERc intervention.
FIGURE 2
FIGURE 2
Initial version of the Mini-AFTERc FOI measure.
FIGURE 3
FIGURE 3
Mini-AFTERc intervention interviews between SCNs and breast cancer patients.
FIGURE 4
FIGURE 4
Example of codes sorted into the relevant sub-theme and theme of the Mini-AFTERc intervention.

References

    1. Ackerman S. J., Hilsenroth M. J. (2003). A review of therapist characteristics and techniques positively impacting the therapeutic alliance. Clin. Psychol. Rev. 23 1–33. 10.1016/s0272-7358(02)00146-0
    1. Bordin E. S. (1994). “Theory and research on the therapeutic working alliance: new directions,” in The Working Alliance: Theory, Research, and Practice, eds Horvath A. O., Greenberg L. S. (Oxford: John Wiley & Sons; ), 13–37.
    1. Borrelli B. (2011). The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. J. Public Health Dent. 71 52–63.
    1. Braun V., Clarke V. (2006). Using thematic analysis in psychology. Qual. Res. Psychol. 3 77–101. 10.1191/1478088706qp063oa
    1. Braun V., Clarke V. (2012). “Thematic analysis,” in APA Handbook of Research Methods in Psychology, Vol. 2. Research Designs: Quantitative, Qualitative, Neuropsychological, and Biological, eds Cooper H., Camic P. M., Long D. L., Panter A. T., Rindskopf D., Sher K. J. (Washington, DC: American Psychological Association; ), 57–71.
    1. Breitenstein S. M., Gross D., Garvey C. A., Hill C., Fogg L., Resnick B. (2010). Implementation fidelity in community-based interventions. Res. Nurs. Health. 33 164–173. 10.1002/nur.20373
    1. Brownson R. C., Colditz G. A., Proctor E. K. (2017). Dissemination and Implementation Research in Health: Translating Science to Practice. New York, NY: Oxford University Press.
    1. Butow P. N., Turner J., Gilchrist J., Sharpe L., Smith A. B., Fardell J. E., et al. (2017). Randomized trial of ConquerFear: a novel, theoretically based psychosocial intervention for fear of cancer recurrence. J. Clin. Oncol. 35 4066–4077. 10.1200/jco.2017.73.1257
    1. Century J., Rudnick M., Freeman C. (2010). A framework for measuring fidelity of implementation: a foundation for shared language and accumulation of knowledge. Am. J. Eval. 31 199–218. 10.1177/1098214010366173
    1. Davidson J., Malloch M., Humphris G. (2018). A single-session intervention (the Mini-AFTERc) for fear of cancer recurrence: a feasibility study. Psychooncology 27 2668–2670. 10.1002/pon.4724
    1. Dieng M., Butow P. N., Costa D. S., Morton R. L., Menzies S. W., Mireskandari S., et al. (2016). Psychoeducational intervention to reduce fear of cancer recurrence in people at high risk of developing another primary melanoma: results of a randomized controlled trial. J. Clin. Oncol. 34 4405–4414. 10.1200/jco.2016.68.2278
    1. Dodds S. E., Pace T. W., Bell M. L., Fiero M., Negi L. T., Raison C. L., et al. (2015). Feasibility of Cognitively-Based Compassion Training (CBCT) for breast cancer survivors: a randomized, wait list controlled pilot study. Support. Care Cancer 23 3599–3608. 10.1007/s00520-015-2888-1
    1. Durlak J. A., DuPre E. P. (2008). Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am. J. Community Psychol. 41 327–350. 10.1007/s10464-008-9165-0
    1. Flückiger C., Del Re A., Wampold B. E., Horvath A. O. (2018). The alliance in adult psychotherapy: a meta-analytic synthesis. Psychotherapy 55 316–340. 10.1037/pst0000172
    1. Forgatch M. S., Patterson G. R., DeGarmo D. S. (2005). Evaluating fidelity: predictive validity for a measure of competent adherence to the oregon model of parent management training. Behav. Ther. 36 3–13. 10.1016/s0005-7894(05)80049-8
    1. Hepner K. A., Howard S., Paddock S. M., Hunter S. B., Osilla K. C., Watkins K. E. (2011). A Fidelity Coding Guide for a Group Cognitive Behavioral Therapy for Depression. Santa Monica, CA: RAND Corporation.
    1. Horvath A. O., Del Re A., Flückiger C., Symonds D. (2011). Alliance in individual psychotherapy. Psychotherapy 48 9–16. 10.1037/a0022186
    1. Humphris G., Ozakinci G. (2008). The AFTER intervention: a structured psychological approach to reduce fears of recurrence in patients with head and neck cancer. Br. J. Health Psychol. 13 223–230. 10.1348/135910708x283751
    1. Humphris G. M., Watson E., Sharpe M., Ozakinci G. (2018). Unidimensional scales for fears of cancer recurrence and their psychometric properties: the FCR4 and FCR7. Health Qual. Life Outcomes 16 30–41.
    1. Jacoby J., Matell M. S. (1971). Three point Likert scales are good enough. J. Mark. Res. 8 495–500. 10.2307/3150242
    1. Karver M. S., Handelsman J. B., Fields S., Bickman L. (2006). Meta-analysis of therapeutic relationship variables in youth and family therapy: the evidence for different relationship variables in the child and adolescent treatment outcome literature. Clin. Psychol. Rev. 26 50–65. 10.1016/j.cpr.2005.09.001
    1. Koo T. K., Li M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J. Chiropr. Med. 15 155–163. 10.1016/j.jcm.2016.02.012
    1. Lebel S., Maheu C., Lefebvre M., Secord S., Courbasson C., Singh M., et al. (2014). Addressing fear of cancer recurrence among women with cancer: a feasibility and preliminary outcome study. J. Cancer Surviv. 8 485–496. 10.1007/s11764-014-0357-3
    1. Lebel S., Ozakinci G., Humphris G., Mutsaers B., Thewes B., Prins J., et al. (2016). From normal response to clinical problem: definition and clinical features of fear of cancer recurrence. Support. Care Cancer 24 3265–3268. 10.1007/s00520-016-3272-5
    1. Lee-Jones C., Humphris G., Dixon R., Hatcher M. B. (1997). Fear of cancer recurrence—a literature review and proposed cognitive formulation to explain exacerbation of recurrence fears. Psychooncology 6 95–105. 10.1002/(sici)1099-1611(199706)6:2<95::aid-pon250>;2-b
    1. Leventhal H., Nerenz D., Steele D. (1984). “Illness representations and coping with health threats,” in Handbook of Psychology and Health, Volume IV: Social Psychology Aspects of Health, eds Baum A., Taylor S. E., Singer J. E. (Hillsdale, NJ: Erlbaum; ), 219–252. 10.4324/9781003044307-9
    1. Luborsky L. (1994). “Therapeutic alliances as predictors of psychotherapy outcomes: factors explaining the predictive success,” in The Working Alliance: Theory, Research, and Practice, eds Horvath A. O., Greenberg L. S. (Oxford: John Wiley & Sons; ), 38–50.
    1. Markle D. T., West R. E., Rich P. J. (2011). Beyond transcription: technology, change, and refinement of method. Forum Qual. Soc. Res. 12 1–21.
    1. McHale C. T., Cruickshank S., Torrens C., Armes J., Fenlon D., Banks E., et al. (2020). A controlled pilot trial of a nurse-led intervention (Mini-AFTERc) to manage fear of cancer recurrence in patients affected by breast cancer. Pilot Feasibil. Stud. 6 60–69.
    1. Mihalic S. (2004). The importance of implementation fidelity. Emo. Behav. Disorders Youth 4 83–105.
    1. Nelson M. C., Cordray D. S., Hulleman C. S., Darrow C. L., Sommer E. C. (2012). A procedure for assessing intervention fidelity in experiments testing educational and behavioral interventions. J. Behav. Health Serv. Res. 39 374–396. 10.1007/s11414-012-9295-x
    1. Orwin R. G. (2000). Assessing program fidelity in substance abuse health services research. Addiction 95(Suppl. 3), 309–327. 10.1080/09652140020004250
    1. Paul S., Charura D. (2014). An Introduction to the Therapeutic Relationship in Counselling and Psychotherapy. London: SAGE Publications Ltd.
    1. Perinetti G. (2018). StaTips Part IV: selection, interpretation and reporting of the intraclass correlation coefficient. South Eur. J. Orthod. Dentofac. Res. 5 3–5. 10.5937/sejodr5-17434
    1. Saini M. (2009). A meta-analysis of the psychological treatment of anger: developing guidelines for evidence-based practice. J. Am. Acad. Psychiatry Law 37 473–488.
    1. Shirk S. R., Karver M. (2003). Prediction of treatment outcome from relationship variables in child and adolescent therapy: a meta-analytic review. J. Consult. Clin. Psychol. 71 452–464. 10.1037/0022-006x.71.3.452
    1. Shirk S. R., Karver M. S., Brown R. (2011). The alliance in child and adolescent psychotherapy. Psychotherapy 48 17–24. 10.1037/a0022181
    1. Simard S., Thewes B., Humphris G., Dixon M., Hayden C., Mireskandari S., et al. (2013). Fear of cancer recurrence in adult cancer survivors: a systematic review of quantitative studies. J. Cancer Surviv. 7 300–322. 10.1007/s11764-013-0272-z
    1. Stains M., Vickrey T. (2017). Fidelity of implementation: an overlooked yet critical construct to establish effectiveness of evidence-based instructional practices. CBE Life Sci. Educ. 16 1–11.
    1. Terry G., Hayfield N., Clarke V., Braun V. (2017). “Thematic analysis,” in The SAGE Handbook of Qualitative Research in Psychology, eds Willig C., Stainton- Rogers W. (London: SAGE Publications Ltd; ), 17–37.
    1. Tomei C., Lebel S., Maheu C., Lefebvre M., Harris C. (2018). Examining the preliminary efficacy of an intervention for fear of cancer recurrence in female cancer survivors: a randomized controlled clinical trial pilot study. Support. Care Cancer 26 2751–2762. 10.1007/s00520-018-4097-1
    1. van de Wal M., Thewes B., Gielissen M., Speckens A., Prins J. (2017). Efficacy of blended cognitive behavior therapy for high fear of recurrence in breast, prostate, and colorectal cancer survivors: the SWORD study, a randomized controlled trial. J. Clin. Oncol. 35 2173–2183. 10.1200/jco.2016.70.5301

Source: PubMed

스폰서 및 공동 작업자

건강 상태

약물 개입

3
구독하다