Examining the sustainability and effectiveness of co-created physical activity interventions in vocational education and training: a multimethod evaluation

Eva Grüne, Johanna Popp, Johannes Carl, Jana Semrau, Klaus Pfeifer, Eva Grüne, Johanna Popp, Johannes Carl, Jana Semrau, Klaus Pfeifer

Abstract

Background: Co-creation approaches are increasingly used in physical activity promotion to develop interventions tailored to the target group and setting. The resulting complexity of such interventions raises challenges in evaluation. Accordingly, little is known about the effectiveness of co-created interventions and the underlying processes that impact their sustainable implementation. In this study, we attempt to fill this gap by evaluating co-created multi-component physical activity interventions in vocational education and training in nursing care and automotive mechatronics regarding (1) their sustainable implementation at the institutional level and (2) the effectiveness of single intervention components at the individual level.

Methods: Following a multimethod design, we conducted a questionnaire survey (n = 7) and semi-structured interviews (n = 4) to evaluate the sustainability of the interventions. Quantitative data were analyzed descriptively, and qualitative data were analyzed using qualitative content analysis. To examine the interventions' effectiveness, we conducted two non-randomized controlled trials (n = 111). Analysis of variance was used to examine differences between groups.

Results: At the institutional level, long-term implementation of single intervention components in nursing care was observed; in contrast, long-term implementation in automotive mechatronics was not observed. In this context, various factors at the outer contextual (e.g., COVID-19 pandemic), inner contextual (e.g., health-promoting leadership), intervention (e.g., acceptance), and personal levels (e.g., champion) influenced sustainability. At the individual level, no significant intervention effects were found for changes in physical activity behavior and physical activity-related health competence.

Conclusion: The role of co-creation on the effectiveness and sustainability of physical activity promotion in vocational education and training cannot be answered conclusively. Only in the nursing care sector, a co-creation approach appeared promising for long-term intervention implementation. Sustainable implementation depends on various influencing factors that should be considered from the outset. Demonstrating effectiveness at the individual level was challenging. To conclusively clarify both the role and impact of co-creation, methodologically complex and elaborate evaluation designs will be required in future research projects.

Trial registration: This study was retrospectively registered at clinicaltrials.gov on 24/08/2021 ( NCT05018559 ).

Keywords: Adolescents; Apprentices; Health promotion; Implementation; Maintenance; Participation; Participatory approach; Pragmatic evaluation approach.

Conflict of interest statement

The authors declare that they have no competing interests.

© 2022. The Author(s).

Figures

Fig. 1
Fig. 1
Overall study procedure of the PArC-AVE project, including the multimethod evaluation of sustainability and effectiveness. BuG = Ger. “Bewegt und Gesund”, Eng. physical activity and health); CG = control group; IG = intervention group; * quantitative data collection using standardized questionnaires; ** qualitative data collection using semi-structured interviews

References

    1. Hanvold TN, Kines P, Nykänen M, Thomée S, Holte KA, Vuori J, et al. Occupational Safety and Health Among Young Workers in the Nordic Countries: A Systematic Literature Review. Saf Health Work. 2019;10(1):3–20. doi: 10.1016/j.shaw.2018.12.003.
    1. Betz M, Graf-Weber G, Kapelke C, Wenchel K. Gesundheitsförderung in der überbetrieblichen Ausbildung am Beispiel des Kfz-Handwerks [Health promotion in inter-company apprenticeship using the automotive trade as an example] Dtsch med Wochenschr. 2012;137:03. doi: 10.1055/s-0032-1301803.
    1. Bomball J, Schwanke A, Stöver M, Görres S. Gesunde Pflege beginnt in der Pflegeausbildung [Healthy nursing care begins with nursing education] Die Schwester Der Pfleger. 2010;49:1048–54.
    1. Bonevski B, Guillaumier A, Paul C, Walsh R. The vocational education setting for health promotion: A survey of students’ health risk behaviours and preferences for help. Health Promotion J Aust. 2013;24(3):185–91. doi: 10.1071/HE13047.
    1. Lehmann F, von Lindeman K, Klewer J, Kugler J. BMI, physical inactivity, cigarette and alcohol consumption in female nursing students: a 5-year comparison. BMC Medical Education. 2014;14:82. doi: 10.1186/1472-6920-14-82.
    1. Warburton DER, Bredin SSD. Health benefits of physical activity: A systematic review of current systematic reviews. Curr Opin Cardiol. 2017;32(5):541–56. doi: 10.1097/HCO.0000000000000437.
    1. Lee I-M, Shiroma EJ, Lobelo F, Puska P, Blair SN, Katzmarzyk PT. Effect of physical inactivity on major non-communicable diseases worldwide: An analysis of burden of disease and life expectancy. The Lancet. 2012;380(9838):219–29. doi: 10.1016/S0140-6736(12)61031-9.
    1. Grüne E, Popp J, Carl J, Pfeifer K. What do we know about physical activity interventions in vocational education and training? A systematic review. BMC Public Health. 2020;20:978. doi: 10.1186/s12889-020-09093-7.
    1. Abu-Omar K, Rütten A, Burlacu I, Messing S, Pfeifer K, Ungerer-Röhrich U. Systematischer Review von Übersichtsarbeiten zu Interventionen der Bewegungsförderung: Methodologie und erste Ergebnisse. Gesundheitswesen. 2017;79(S 01):S45-S50.
    1. Heath GW, Parra DC, Sarmiento OL, Andersen LB, Owen N, Goenka S, et al. Evidence-based intervention in physical activity: lessons from around the world. The Lancet. 2012;380(9838):272–81. doi: 10.1016/S0140-6736(12)60816-2.
    1. Heath GW, Liguori G. Physical Activity and Health Promotion. In: Wright JD, editor. International encyclopedia of the social & behavioral sciences. 2. Amsterdam: Elsevier; 2015. pp. 91–9.
    1. Bauman AE, Reis RS, Sallis JF, Wells JC, Loos RJF, Martin BW. Correlates of physical activity: why are some people physically active and others not? The Lancet. 2012;380(9838):258–71. doi: 10.1016/S0140-6736(12)60735-1.
    1. Sallis JF, Owen N. Ecological Models of Health Behavior. In: Glanz K, Rimer BK, Viswanath K, editors. Health Behavior: Theory, Research, and Practice. 5. San Francisco: Jossey-Bass; 2015. pp. 43–64.
    1. Funk SC, Schaefer I, Kolip P. Was fördert die Verstetigung von Strukturen und Angeboten der Gesundheitsförderung? [Long-Term Implementation of Structures and Activities Needed for Health Promotion] Gesundheitswesen. 2019;81(1):38–42. doi: 10.1055/s-0042-116437.
    1. Wiltsey Stirman S, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7:17. doi: 10.1186/1748-5908-7-17.
    1. Leask CF, Sandlund M, Skelton DA, Altenburg TM, Cardon G, Chinapaw MJM, et al. Framework, principles and recommendations for utilising participatory methodologies in the co-creation and evaluation of public health interventions. Res Involv Engagem. 2019;5:2. doi: 10.1186/s40900-018-0136-9.
    1. McConnell T, Best P, Davidson G, McEneaney T, Cantrell C, Tully M. Coproduction for feasibility and pilot randomised controlled trials: learning outcomes for community partners, service users and the research team. Res Involv Engagem. 2018;4:32. doi: 10.1186/s40900-018-0116-0.
    1. Israel BA, Schulz AJ, Parker EA, Becker AB. REVIEW OF COMMUNITY-BASED RESEARCH: Assessing Partnership Approaches to Improve Public Health. Annu Rev Public Health. 1998;19:173–202. doi: 10.1146/annurev.publhealth.19.1.173.
    1. Anselma M, Chinapaw MJM, Altenburg TM. Determinants of Child Health Behaviors in a Disadvantaged Area from a Community Perspective: A Participatory Needs Assessment. Int J Environ Res Public Health. 2018;15(4):644. doi: 10.3390/ijerph15040644.
    1. Hayball FZ, Pawlowski CS. Using participatory approaches with children to better understand their physical activity behaviour. Health Educ J. 2018;77(5):542–54. doi: 10.1177/0017896918759567.
    1. van Sluijs EMF, Kriemler S. Reflections on physical activity intervention research in young people - dos, don’ts, and critical thoughts. Int J Behav Nutr Phys Act. 2016;13:25. doi: 10.1186/s12966-016-0348-z.
    1. Halvorsrud K, Kucharska J, Adlington K, Rüdell K, Brown Hajdukova E, Nazroo J, et al. Identifying evidence of effectiveness in the co-creation of research: a systematic review and meta-analysis of the international healthcare literature. J Public Health. 2021;43(1):197–208. doi: 10.1093/pubmed/fdz126.
    1. Wallerstein N, Duran B. Community-Based Participatory Research Contributions to Intervention Research: The Intersection of Science and Practice to Improve Health Equity. Am J Public Health. 2010;100(S1):S40-S46. doi: 10.2105/AJPH.2009.184036.
    1. Cargo M, Mercer SL. The value and challenges of participatory research: strengthening its practice. Annu Rev Public Health. 2008;29:325–50. doi: 10.1146/annurev.publhealth.29.091307.083824.
    1. Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Educ Res. 1998;13(1):87–108. doi: 10.1093/her/13.1.87.
    1. Greenhalgh T, Jackson C, Shaw S, Janamian T. Achieving Research Impact Through Co-creation in Community-Based Health Services: Literature Review and Case Study. Milbank Quart. 2016;94(2):392–429. doi: 10.1111/1468-0009.12197.
    1. Jagosh J, Macaulay AC, Pluye P, Salsberg J, Bush PL, Henderson J, et al. Uncovering the Benefits of Participatory Research: Implications of a Realist Review for Health Research and Practice. The Milbank Quarterly. 2012;90(2):311–46. doi: 10.1111/j.1468-0009.2012.00665.x.
    1. Bodkin A, Hakimi S. Sustainable by design: a systematic review of factors for health promotion program sustainability. BMC Public Health. 2020;20(1):964. doi: 10.1186/s12889-020-09091-9.
    1. Carl J, Sudeck G, Geidl W, Schultz K, Pfeifer K. Competencies for a Healthy Physically Active Lifestyle-Validation of an Integrative Model. Res Q Exerc Sport. 2020;92(3):514–28. doi: 10.1080/02701367.2020.1752885.
    1. Popp J, Carl J, Grüne E, Semrau J, Gelius P, Pfeifer K. Physical activity promotion in German vocational education: does capacity building work? Health Promot Int. 2020;35:1577–89. doi: 10.1093/heapro/daaa014.
    1. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655. doi: 10.1136/bmj.a1655.
    1. Treweek S. Complex interventions and the chamber of secrets: understanding why they work and why they do not. J Royal Soc Med. 2005;98:553. doi: 10.1177/014107680509801218.
    1. Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3-4):327–50. doi: 10.1007/s10464-008-9165-0.
    1. Crane M, Bauman A, Lloyd B, McGill B, Rissel C, Grunseit A. Applying pragmatic approaches to complex program evaluation: A case study of implementation of the New South Wales Get Healthy at Work program. Health Promot J Austr. 2019;30(3):422–32. doi: 10.1002/hpja.239.
    1. Malterud K, Aamland A, Iden KR. Small-scale implementation with pragmatic process evaluation: a model developed in primary health care. BMC Fam Pract. 2018;19(1):93. doi: 10.1186/s12875-018-0778-6.
    1. Popp J, Grüne E, Carl J, Semrau J, Pfeifer K. Co-creating physical activity interventions: a mixed methods evaluation approach. Health Res Policy Syst. 2021;19(1):37. doi: 10.1186/s12961-021-00699-w.
    1. Onwuegbuzie AJ, Hitchcock JH. A meta-framework for conducting mixed methods impact evaluations: Implications for altering practice and the teaching of evaluation. Stud Educ Eval. 2017;53:55–68. doi: 10.1016/j.stueduc.2017.02.001.
    1. Albright K, Gechter K, Kempe A. Importance of mixed methods in pragmatic trials and dissemination and implementation research. Acad Pediatr. 2013;13(5):400–7. doi: 10.1016/j.acap.2013.06.010.
    1. Rütten A. Kooperative Planung und Gesundheitsförderung: Ein Implementationsansatz [Cooperative Planning and health promotion: An implementation approach] Zeitschrift für Gesundheitswissenschaften. 1997;5(3):257–72.
    1. Rütten A, et al. Evaluating healthy public policies in community and regional contexts. In: Rootman I, Goodstadt M, Hyndman B, McQueen DV, Potvin L, Springett J, et al., editors. Evaluation in health promotion: Principles and perspectives. Geneva: World Health Organization Europe; 2001. pp. 341–64.
    1. Anguera MT, Blanco-Villaseñor A, Losada JL, Sánchez-Algarra P, Onwuegbuzie AJ. Revisiting the difference between mixed methods and multimethods: Is it all in the name? Quality & Quantity. 2018;52(6):2757–70. doi: 10.1007/s11135-018-0700-2.
    1. Morse JM. Principles of Mixed Methods and Multimethod Research Design. In: Tashakkori A, Teddlie C, editors. Handbook of Mixed Methods in Social & Behavioral Research. Thousand Oaks: Sage Publications; 2003. pp. 189–208.
    1. Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, et al. Standards for Reporting Implementation Studies (StaRI) Statement. BMJ. 2017;356:i6795. doi: 10.1136/bmj.i6795.
    1. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57. doi: 10.1093/intqhc/mzm042.
    1. Patton MQ. Qualitative research & evaluation methods: integrating theory and practice. 4. Los Angeles: SAGE; 2015.
    1. Shelton RC, Cooper BR, Wiltsey Stirman S. The Sustainability of Evidence-Based Interventions and Practices in Public Health and Health Care. Annu Rev Public Health. 2018;39:55–76. doi: 10.1146/annurev-publhealth-040617-014731.
    1. Kuckartz U. Qualitative Inhaltsanalyse. Methoden, Praxis, Computerunterstützung [Qualitative Text Analysis: A Guide to Methods, Practice and Using Software]. 4. Auflage. Weinheim, Basel: Beltz Juventa; 2018. (Grundlagentexte Methoden). Available from: URL: .
    1. Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The Behavior Change Technique Taxonomy (v1) of 93 Hierarchically Clustered Techniques: Building an International Consensus for the Reporting of Behavior Change Interventions. Ann Behav Med. 2013;46(1):81–95. doi: 10.1007/s12160-013-9486-6.
    1. Fuchs R, Klaperski S, Gerber M, Seelig H. Messung der Bewegungs- und Sportaktivität mit dem BSA-Fragebogen [Measurement of Physical Activity and Sport Activity With the BSA Questionnaire] Zeitschrift für Gesundheitspsychologie. 2015;23(2):60–76. doi: 10.1026/0943-8149/a000137.
    1. Dixon WJ, Tukey JW. Approximate Behavior of the Distribution of Winsorized t (Trimming/Winsorization 2) Technometrics. 1968;10(1):83–98.
    1. Zander-Jentsch B, Wagner F, Rzayeva N, Busse R. Germany. In: Rafferty AM, Busse R, Zander-Jentsch B, Sermeus W, Bruyneel L, editors. Strengthening health systems through nursing: Evidence from 14 European countries. Copenhagen: World Health Organization; 2019. pp. 43–52.
    1. Hailemariam M, Bustos T, Montgomery B, Barajas R, Evans LB, Drahota A. Evidence-based intervention sustainability strategies: a systematic review. Implement Sci. 2019;14(1):57. doi: 10.1186/s13012-019-0910-6.
    1. Nastasi BK, Varjas K, Schensul SL, Silva KT, Schensul JJ, Ratnayake P. The Participatory Intervention Model: A Framework for Conceptualizing and Promoting Intervention Acceptability. School Psychol Quart. 2000;15(2):207–32. doi: 10.1037/h0088785.
    1. Grüne E, Popp J, Carl J, Semrau J, Pfeifer K. Bewegungsbezogene Gesundheitskompetenz (BGK) als curricularer Inhalt in der Pflegeausbildung [Physical Activity-related Health Competence (PAHCO) as curricular content of nursing education] Bewegungstherapie und Gesundheitssport. 2020;36:266–268. doi: 10.1055/a-1292-6756.
    1. Green LW, Kreuter MW. Health program planning. An educational and ecological approach. 4. New York: McGraw-Hill Companies; 2005.
    1. Rowling L, Samdal O. Filling the black box of implementation for health-promoting schools. Health Education. 2011;111(5):347–62. doi: 10.1108/09654281111161202.
    1. Muellmann S, Steenbock B, de Cocker K, de Craemer M, Hayes C, O’Shea MP, et al. Views of policy makers and health promotion professionals on factors facilitating implementation and maintenance of interventions and policies promoting physical activity and healthy eating: results of the DEDIPAC project. BMC Public Health. 2017;17(1):932. doi: 10.1186/s12889-017-4929-9.
    1. Bunce AE, Gruß I, Davis JV, Cowburn S, Cohen D, Oakley J, et al. Lessons learned about the effective operationalization of champions as an implementation strategy: results from a qualitative process evaluation of a pragmatic trial. Implement Sci. 2020;15(1):87. doi: 10.1186/s13012-020-01048-1.
    1. Urquhart R, Kendell C, Geldenhuys L, Ross A, Rajaraman M, Folkes A, et al. The role of scientific evidence in decisions to adopt complex innovations in cancer care settings: a multiple case study in Nova Scotia, Canada. Implement Sci. 2019;14(1):14. doi: 10.1186/s13012-019-0859-5.
    1. Bonawitz K, Wetmore M, Heisler M, Dalton VK, Damschroder LJ, Forman J, et al. Champions in context: which attributes matter for change efforts in healthcare? Implement Sci. 2020;15(1):62. doi: 10.1186/s13012-020-01024-9.
    1. Skarholt K, Blix EH, Sandsund M, Andersen TK. Health promoting leadership practices in four Norwegian industries. Health Promot Int. 2016;31(4):936–45.
    1. Aarons GA, Ehrhart MG, Farahnak LR. The implementation leadership scale (ILS): development of a brief measure of unit level implementation leadership. Implementation Science. 2014;9:45. doi: 10.1186/1748-5908-9-45.
    1. Eriksson A, Orvik A, Strandmark M, Nordsteien A, Torp S. Management and Leadership Approaches to Health Promotion and Sustainable Workplaces: A Scoping Review. Societies. 2017;7(2):14. doi: 10.3390/soc7020014.
    1. Herbert-Maul A, Abu-Omar K, Frahsa A, Streber A, Reimers AK. Transferring a Community-Based Participatory Research Project to Promote Physical Activity Among Socially Disadvantaged Women-Experiences From 15 Years of BIG. Front Public Health. 2020;8:571413. doi: 10.3389/fpubh.2020.571413.
    1. Weiner BJ, Clary AS, Klaman SL, Turner K, Alishahi-Tabriz A. Organizational readiness for change: what we now, what we think we wnow, and what we need to know. In: Albers B, Shlonsky A, Mildon R, editors. Implementation Science 3.0. Cham: Springer International Publishing; 2020. pp. 101–44.
    1. Scaccia JP, Cook BS, Lamont A, Wandersman A, Castellow J, Katz J, et al. A practical implementation science heuristic for organizational readiness: R = MC2. J Community Psychol. 2015;43(4):484–501. doi: 10.1002/jcop.21698.
    1. Helfrich CD, Kohn MJ, Stapleton A, Allen CL, Hammerback KE, Chan KCG, et al. Readiness to Change Over Time: Change Commitment and Change Efficacy in a Workplace Health-Promotion Trial. Front Public Health. 2018;6:110. doi: 10.3389/fpubh.2018.00110.
    1. Zhang Y, Flum M, West C, Punnett L. Assessing Organizational Readiness for a Participatory Occupational Health/Health Promotion Intervention in Skilled Nursing Facilities. Health Promot Pract. 2015;16(5):724–32. doi: 10.1177/1524839915573945.
    1. Carl J, Grüne E, Popp J, Pfeifer K. Physical Activity Promotion for Apprentices in Nursing Care and Automotive Mechatronics-Competence Counts More than Volume. Int J Environ Res Public Health. 2020;17(3):793. doi: 10.3390/ijerph17030793.
    1. Jun SY, Kim J, Choi H, Kim JS, Lim SH, Sul B, et al. Physical Activity of Workers in a Hospital. Int J Environ Res Public Health. 2019;16(4):532. doi: 10.3390/ijerph16040532.
    1. Dadaczynski K, Schiemann S, Backhaus O. Promoting physical activity in worksite settings: results of a German pilot study of the online intervention Healingo fit. BMC Public Health. 2017;17(1):696. doi: 10.1186/s12889-017-4697-6.
    1. Eccles MP, Mittman BS. Welcome to Implementation Science. Implementation Science. 2006;1(1):1. doi: 10.1186/1748-5908-1-1.
    1. Koorts H, Eakin E, Estabrooks P, Timperio A, Salmon J, Bauman A. Implementation and scale up of population physical activity interventions for clinical and community settings: the PRACTIS guide. Int J Behav Nutr Phys Act. 2018;15(1):51. doi: 10.1186/s12966-018-0678-0.
    1. Goldstein H, Olswang L. Is there a science to facilitate implementation of evidence-based practices and programs? Evidence-Based Communication Assessment and Intervention. 2017;11(3-4):55–60. doi: 10.1080/17489539.2017.1416768.
    1. Bauer MS, Kirchner J. Implementation science: What is it and why should I care? Psychiatry Res. 2020;283:112376. doi: 10.1016/j.psychres.2019.04.025.
    1. Wolfenden L, Foy R, Presseau J, Grimshaw JM, Ivers NM, Powell BJ, et al. Designing and undertaking randomised implementation trials: guide for researchers. BMJ. 2021;372:m3721. doi: 10.1136/bmj.m3721.
    1. Wolfenden L, Williams CM, Wiggers J, Nathan N, Yoong SL. Improving the translation of health promotion interventions using effectiveness-implementation hybrid designs in program evaluations. Health Promotion J Aust. 2016;27(3):204–7. doi: 10.1071/HE16056.

Source: PubMed

Sponsorer og samarbejdspartnere

Medicinske tilstande

Narkotikainterventioner

3
Abonner