How to design, implement and evaluate organizational interventions for maximum impact: the Sigtuna Principles
Ulrica von Thiele Schwarz, Karina Nielsen, Kasper Edwards, Henna Hasson, Christine Ipsen, Carl Savage, Johan Simonsen Abildgaard, Anne Richter, Caroline Lornudd, Pamela Mazzocato, Julie E Reed, Ulrica von Thiele Schwarz, Karina Nielsen, Kasper Edwards, Henna Hasson, Christine Ipsen, Carl Savage, Johan Simonsen Abildgaard, Anne Richter, Caroline Lornudd, Pamela Mazzocato, Julie E Reed
Abstract
Research on organizational interventions needs to meet the objectives of both researchers and participating organizations. This duality means that real-world impact has to be considered throughout the research process, simultaneously addressing both scientific rigour and practical relevance. This discussion paper aims to offer a set of principles, grounded in knowledge from various disciplines that can guide researchers in designing, implementing, and evaluating organizational interventions. Inspired by Mode 2 knowledge production, the principles were developed through a transdisciplinary, participatory and iterative process where practitioners and academics were invited to develop, refine and validate the principles. The process resulted in 10 principles: 1) Ensure active engagement and participation among key stakeholders; 2) Understand the situation (starting points and objectives); 3) Align the intervention with existing organizational objectives; 4) Explicate the program logic; 5) Prioritize intervention activities based on effort-gain balance; 6) Work with existing practices, processes, and mindsets; 7) Iteratively observe, reflect, and adapt; 8) Develop organizational learning capabilities; 9) Evaluate the interaction between intervention, process, and context; and 10) Transfer knowledge beyond the specific organization. The principles suggest how the design, implementation, and evaluation of organizational interventions can be researched in a way that maximizes both practical and scientific impact.
Keywords: Academy-practice partnership; occupational health interventions; participation; recommendations; workplace-based interventions.
Conflict of interest statement
The authors declare no conflicts of interest.
© 2020 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.
Figures
References
- Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration Policy in Mental Health and Mental Health Services Research, 38(1), 4–23. 10.1007/s10488-010-0327-7
- Abildgaard, J. S., Hasson, H., von Thiele Schwarz, U., Løvseth, L. T., Ala-Laurinaho, A., & Nielsen, K. (2018). Forms of participation: The development and application of a conceptual model of participation in work environment interventions. Economic and Industrial Democracy, 41(3), 746–769. 10.1177/0143831X17743576
- Abildgaard, J. S., Nielsen, K., Wåhlin-Jacobsen, C. D., Maltesen, T., Christensen, K. B., & Holtermann, A. (2019). ‘Same, but different’–a mixed methods realist evaluation of a cluster-randomised controlled participatory organisational intervention. Human Relations. 10.1177/0018726719866896
- Argyris, C., & Schön, D. A. (1996). Organizational learning 11: Theory, method and practice. Addison-Wellesley.
- Bamberger, M., Tarsilla, M., & Hesse-Biber, S. (2016). Why so many “rigorous” evaluations fail to identify unintended consequences of development programs: How mixed methods can contribute. Evaluation and Program Planning, 55, 155–162. 10.1016/j.evalprogplan.2016.01.001
- Batalden, P. B., & Davidoff, F. (2007). What is “quality improvement” and how can it transform healthcare? Quality and Safety in Health Care, 16(1), 2–3. 10.1136/qshc.2006.022046
- Batalden, P. B., & Stoltz, P. K. (1993). A framework for the continual improvement of health care: Building and applying professional and improvement knowledge to test changes in daily work. The Joint Commission Journal on Quality Improvement, 19(10), 424–447. 10.1016/S1070-3241(16)30025-6
- Bauer, G. F., & Jenny, G. J. (2012). Moving towards positive organisational health: Challenges and a proposal for a research model of organisational health development. In: J. Houdmondt, S. Leka & R.R. Sinclair (Eds.). Contemporary Occupational Health Psychology: Global Perspectives on Research and Practice. Vol 2 (pp 126–145). Oxford, UK: Wiley-Blackwell.
- Benneyan, J., Lloyd, R., & Plsek, P. (2003). Statistical process control as a tool for research and healthcare improvement. Quality and Safety in Health Care, 12(6), 458–464. 10.1136/qhc.12.6.458
- Bernerth, J. B., Walker, H. J., & Harris, S. G. (2011). Change fatigue: Development and initial validation of a new measure. Work & Stress, 25(4), 321–337. 10.1080/02678373.2011.634280
- Biron, C., & Karanika-Murray, M. (2013). Process evaluation for organizational stress and well-being interventions: Implications for theory, method, and practice. International Journal of Stress Management, 21(1), 85–111. .
- Brown, K. G., & Gerhardt, M. W. (2002). Formative evaluation: An integrative practice model and case study. Personnel Psychology, 55(4), 951–983. 10.1111/j.1744-6570.2002.tb00137.x
- Brydon-Miller, M., Greenwood, D., & Maguire, P. (2003). Why action research? Action Research, 1(1), 9–28. 10.1177/14767503030011002
- Centers for Disease Control and Prevention . (1999). Framework for program evaluation in public health. MMWR 1999; 48 (No. RR-11).
- Chambers, D., Glasgow, R., & Stange, K. (2013). The dynamic sustainability framework: Addressing the paradox of sustainment amid ongoing change. Implementation Science, 8(1), 117. 10.1186/1748-5908-8-117
- Cox, T., Griffiths, A., & Randall, R. (2002). Interventions to control stress at work in hospital staff. HSE Contract Research Report.
- Cox, T., Griffiths, A., & Rial-González, E. (2000). Research on work-related stress Report: Office for Official Publications of the European Communities.
- Docherty, S. M., & Emden, C. (1997). Qualitative metasynthesis: Issues and techniques. Research in Nursing & Health, 20(4), 365–371. 10.1002/(SICI)1098-240X(199708)20:4<365::AID-NUR9>;2-E
- Edwards, K., & Jensen, P. L. (2014). Design of systems for productivity and well being. Applied Ergonomics, 45(1), 26–32. 10.1016/j.apergo.2013.03.022
- Fishbein, M., & Yzer, M. C. (2003). Using theory to design effective health behavior interventions. Communication Theory, 13(2), 164–183. 10.1111/j.1468-2885.2003.tb00287.x
- Flyvbjerg, B. (2006). Five misunderstandings about case-study research. Qualitative Inquiry, 12(2), 219–245. 10.1177/1077800405284363
- Framke, E., & Sørensen, O. H. (2015). Implementation of a participatory organisational-level occupational health intervention-focusing on the primary task. International Journal of Human Factors and Ergonomics, 3(3–4), 254–270. 10.1504/IJHFE.2015.072998
- Frykman, M., Hasson, H., Atlin Muntlin, Å., & von Thiele Schwarz, U. (2014). Functions of behavior change interventions when implementing teamwork at an emergency department: A comparative case study. BMC Health Services Research, 14(1), 218. 10.1186/1472-6963-14-218
- Gibbons, M., Limoges, C., Nowotony, H., Schwartzman, S., Scott, P., & Trow, M. (1994). The new production of knowledge: The dynamics of science and research in contemporary societies. Sage Publications.
- Greenhalgh, T., & Wieringa, S. (2011). Is it time to drop the ‘knowledge translation’metaphor? A critical literature review. Journal of the Royal Society of Medicine, 104(12), 501–509. 10.1258/jrsm.2011.110285
- Griffiths, A. (1999). Organizational interventions: Facing the limits of the natural science paradigm. Scandinavian Journal of Work, Environment & Health, 25(6), 589–596. 10.5271/sjweh.485
- Gustavsen, B. (2011). The Nordic model of work organization. Journal of the Knowledge Economy, 2(4), 463–480. 10.1007/s13132-011-0064-5
- Hasson, H., Lornudd, C., von Thiele Schwarz, U., & Richter, A. (2018). Supporting Interventions: Enabling senior management to enhance the effectiveness of a training program for line managers. In Nielsen K. & Noblet A. (Eds.), Organizational interventions for health and well-being: A handbook for evidence-based practice (pp. 220–236). Routledge.
- Hasson, H., Villaume, K., von Thiele Schwarz, U., & Palm, K. (2014). Managing implementation: Roles of line managers, senior managers, and human resource professionals in an occupational health intervention. Journal of Occupational and Environmental Medicine, 56(1), 58–65. 10.1097/JOM.0000000000000020
- Hasson, H., von Thiele Schwarz, U., Nielsen, K., & Tafvelin, S. (2016). Are we all in the same boat? The role of perceptual distance in organizational health interventions. Stress & Health, 32(4), 294–303. 10.1002/smi.2703
- Havermans, B. M., Schelvis, R. M., Boot, C. R., Brouwers, E. P., Anema, J. R., & van der Beek, A. J. (2016). Process variables in organizational stress management intervention evaluation research: A systematic review. Scandinavian Journal of Work, Environment & Health, 42(5), 371–381. 10.5271/sjweh.3570
- Holman, D., & Axtell, C. (2016). Can job redesign interventions influence a broad range of employee outcomes by changing multiple job characteristics? A quasi-experimental study. Journal of Occupational Health Psychology, 21(3), 284 doi:10.1037/a0039962
- Ipsen, C., Gish, L., & Poulsen, S. (2015). Organizational-level interventions in small and medium-sized enterprises: Enabling and inhibiting factors in the PoWRS program. Safety Science, 71, 264–274. 10.1016/j.ssci.2014.07.017
- Ipsen, C., & Jensen, P. L. (2012). Organizational options for preventing work-related stress in knowledge work. International Journal of Industrial Ergonomics, 42(4), 325–334. 10.1016/j.ergon.2012.02.006
- Jørgensen, T. H., Remmen, A., & Mellado, M. D. (2006). Integrated management systems–three different levels of integration. Journal of Cleaner Production, 14(8), 713–722. 10.1016/j.jclepro.2005.04.005
- Kaplan, H. C., Provost, L. P., Froehle, C. M., & Margolis, P. A. (2011). The model for understanding success in quality (MUSIQ): Building a theory of context in healthcare quality improvement. BMJ Quality & Safety, 21(1), 13–20.
- Karanika-Murray, M., & Biron, C. (2015b). Why do some interventions derail? Deconstructing the elements of organizational interventions for stress and well-being. In Karanika-Murray M. & Biron C. (Eds.), Derailed organizational interventions for stress and well-being (pp. 1–15). Springer. doi:10.1007/978-94-017-9867-9_1
- Karanika-Murray, M., & Biron, C. (2015a). Derailed organizational interventions for stress and well-being. Springer.
- Kompier, M., & Aust, B. (2016). Organizational stress management interventions: Is it the singer not the song? Scandinavian Journal of Work, Environment & Health, 42(5), 355–358. 10.5271/sjweh.3578
- Kotter, J. P. (1996). Leading Change. Harvard Business School Press.
- Kristensen, T. (2005). Intervention studies in occupational epidemiology. Occupational and Environmental Medicine, 62(3), 205–210. 10.1136/oem.2004.016097
- Lennox, L., Maher, L., & Reed, J. (2018). Navigating the sustainability landscape: A systematic review of sustainability approaches in healthcare. Implementation Science, 13(1), 27. 10.1186/s13012-017-0707-4
- Leviton, L. C., Khan, L. K., Rog, D., Dawkins, N., & Cotton, D. (2010). Evaluability assessment to improve public health policies, programs, and practices. Annual Review of Public Health, 31(1), 213–233. 10.1146/annurev.publhealth.012809.103625
- MacLean, D., MacIntosh, R., & Grant, S. (2002). Mode 2 management research. British Journal of Management, 13(3), 189–207. 10.1111/1467-8551.00237
- Malchaire, J. (2004). The SOBANE risk management strategy and the Déparis method for the participatory screening of the risks. International Archives of Occupational and Environmental Health, 77(6), 443–450. 10.1007/s00420-004-0524-3
- Malinen, S., Hatton, T., Naswall, K., & Kuntz, J. (2019). Strategies to enhance employee well‐being and organisational performance in a postcrisis environment: A case study. Journal of Contingencies and Crisis Management, 27(1), 79–86. 10.1111/1468-5973.12227
- March, J. G. (1991). Exploration and exploitation in organizational learning. Organization Science, 2(1), 71–87. 10.1287/orsc.2.1.71
- McNicholas, C., Lennox, L., Woodcock, T., Bell, D., & Reed, J. E. (2019). Evolving quality improvement support strategies to improve plan–do–study–act cycle fidelity: A retrospective mixed-methods study. BMJ Quality & Safety, 28(5), 356–365. 10.1136/bmjqs-2017-007605
- Montano, D., Hoven, H., & Siegrist, J. (2014). Effects of organisational-level interventions at work on employees’ health: A systematic review. BMC Public Health, 14(1), 135. 10.1186/1471-2458-14-135
- Nielsen, K. (2013). Review article: How can we make organizational interventions work? Employees and line managers as actively crafting interventions. Human Relations, 66(8), 1029–1050. 10.1177/0018726713477164
- Nielsen, K. (2017). Organizational occupational health interventions: What works for whom in which circumstances? Occupational Medicine, 67(6), 410–412. 10.1093/occmed/kqx058
- Nielsen, K., & Abildgaard, J. S. (2013). Organizational interventions: A research-based framework for the evaluation of both process and effects. Work & Stress, 27(3), 278–297. 10.1080/02678373.2013.812358
- Nielsen, K., & Miraglia, M. (2017). What works for whom in which circumstances? On the need to move beyond the ‘what works?’ question in organizational intervention research. Human Relations, 70(1), 40–62. 10.1177/0018726716670226
- Nielsen, K., Nielsen, M. B., Ogbonnaya, C., Känsälä, M., Saari, E., & Isaksson, K. (2017). Workplace resources to improve both employee well-being and performance: A systematic review and meta-analysis. Work & Stress, 31(2), 101–120. 10.1080/02678373.2017.1304463
- Nielsen, K., & Randall, R. (2013). Opening the black box: Presenting a model for evaluating organizational-level interventions. European Journal of Work and Organizational Psychology, 22(5), 601–617. 10.1080/1359432X.2012.690556
- Nielsen, K., Randall, R., Holten, A. L., & González, E. R. (2010). Conducting organizational-level occupational health interventions: What works?. Work & Stress, 24(3), 234–259. doi:10.1080/02678373.2010.515393
- Owen, H. (2008). Open space technology: A user’s guide. Berrett-Koehler Publishers.
- Palm, K., Lilja, J., & Wiklund, H. (2014). The challenge of integrating innovation and quality management practice. Total Quality Management and Business Excellence, 27(1–2), 34–47.
- Pawson, R. (2013). The science of evaluation: A realist manifesto. Sage.
- Pawson, R., & Tilley, N. (1997). Realistic evaluation. Sage.
- Pettigrew, A., & Whipp, R. (1993). Managing change for competitive success. Wiley-Blackwell.
- Priles, M. A. (1993). The fishbowl discussion: A strategy for large honors classes. English Journal, 82(6), 49. 10.2307/820165
- Reed, J. E., Howe, C., Doyle, C., & Bell, D . (2018). Simple rules for evidence translation in complex systems: a qualitative study. BMC medicine,, 16(1), 92.
- Reed, J. E., McNicholas, C., Woodcock, T., Issen, L., & Bell, D. (2014). Designing quality improvement initiatives: The action effect method, a structured approach to identifying and articulating programme theory. BMJ Quality & Safett, 23(12), 1040–1048. 10.1136/bmjqs-2014-003103
- Rogers, P. J. (2008). Using programme theory to evaluate complicated and complex aspects of interventions. Evaluation, 14(1), 29–48. 10.1177/1356389007084674
- Rosskam, E. (2009). Using participatory action research methodology to improve worker health. In Schnall P., Dobson M., & Rosskam E. (Eds.), Unhealthy work: Causes, consequences, cures (pp. 211_229). Baywood Publishing Company.
- Saunders, R. P., Evans, M. H., & Joshi, P. (2005). Developing a process-evaluation plan for assessing health promotion program implementation: A how-to guide. Health Promotion Practice, 6(2), 134–147. 10.1177/1524839904273387
- Savage, M., Storkholm, M. H., Mazzocato, P., & Savage, C. (2018). Effective physician leaders: An appreciative inquiry into their qualities, capabilities and learning approaches. BMJ Leader, 2(3), 95–102. 10.1136/leader-2017-000050
- Schelvis, R. M., Hengel, K. M. O., Burdorf, A., Blatter, B. M., Strijk, J. E., & van der Beek, A. J. (2015). Evaluation of occupational health interventions using a randomized controlled trial: Challenges and alternative research designs. Scandinavian Journal of Work, Environment & Health, 41(5), 491–503. 10.5271/sjweh.3505
- Semmer, N. K. (2006). Job stress interventions and the organization of work. Scandinavian Journal of Work, Environment & Health, 32(6), 515–527. 10.5271/sjweh.1056
- Senge, P. (2006). The fifth discipline, the art and practice of the learning organization, doubleday. Doubleday Currency.
- Shewhart, W. A. (1930). Economic quality control of manufactured product 1. Bell System Technical Journal, 9(2), 364–389. 10.1002/j.1538-7305.1930.tb00373.x
- Sørensen, O. H., & Holman, D. (2014). A participative intervention to improve employee well-being in knowledge work jobs: A mixed-methods evaluation study. Work & Stress, 28(1), 67–86. 10.1080/02678373.2013.876124
- Stenfors Hayes, T., Hasson, H., Augustsson, H., Hvitfeldt Forsberg, H., & von Thiele Schwarz, U. (2014). Merging occupational health, safety and health promotion with Lean: An integrated systems approach (the LeanHealth project). In Burke R., Cooper C., & Biron C. (Eds.), Creating healthy workplaces: Stress reduction, improved well-being, and organizational effectiveness, (pp. 281–299). Gower Publishing.
- Storkholm, M. H., Mazzocato, P., & Savage, C. (2019). Make it complicated: A qualitative study utilizing a complexity framework to explain improvement in health care. BMC Health Services Research, 19(1), 842. 10.1186/s12913-019-4705-x
- Storkholm, M. H., Mazzocato, P., Savage, M., & Savage, C. (2017). Money’s (not) on my mind: A qualitative study of how staff and managers understand health care’s triple aim. BMC Health Services Research, 17(1), 98. 10.1186/s12913-017-2052-3
- Storkholm, M. H., Savage, C., Tessma, M. K., Salvig, J. D., & Mazzocato, P. (2019). Ready for the triple aim? Perspectives on organizational readiness for implementing change from a Danish obstetrics and gynecology department. BMC Health Services Research, 19(1), 517. 10.1186/s12913-019-4319-3
- Straatmann, T., Rothenhöfer, L. M., Meier, A., & Mueller, K. (2018). A configurational perspective on the theory of planned behaviour to understand employees’ change‐supportive intentions. Applied Psychology, 67(1), 91–135. 10.1111/apps.12120
- Tafvelin, S., Nielsen, K., Abildgaard, J. S., Richter, A., von Thiele Schwarz, U., & Hasson, H. (2019). Leader-team perceptual distance affects outcomes of leadership training: Examining safety leadership and follower safety self-efficacy. Safety Science, 120(December), 25–31. 10.1016/j.ssci.2019.06.019
- Tafvelin, S., von Thiele Schwarz, U., Nielsen, K., & Hasson, H. (2019). Employees’ and line managers’ active involvement in participatory organizational interventions: Examining direct, reversed, and reciprocal effects on well-being.Stress and Health, 35(1), 69–80. 10.1002/smi.2841
- Taylor, M. J., McNicholas, C., Nicolay, C., Darzi, A., Bell, D., & Reed, J. E. (2014). Systematic review of the application of the plan–do–study–act method to improve quality in healthcare. BMJ Quality & Safety, 23, 290–298.
- Thiem, A. (2017). Conducting configurational comparative research with qualitative comparative analysis: A hands-on tutorial for applied evaluation scholars and practitioners. American Journal of Evaluation, 38(3), 420–433. 10.1177/1098214016673902
- Thor, J., Lundberg, J., Ask, J., Olsson, J., Carli, C., Härenstam, K. P., & Brommels, M. (2007). Application of statistical process control in healthcare improvement: Systematic review. BMJ Quality & Safety, 16(5)(5), 387–399. 10.1136/qshc.2006.022194
- Tsutsumi, A., Nagami, M., Yoshikawa, T., Nogi, K., & Kawakami, N. (2009). Participatory intervention for workplace improvements on mental health and job performance among blue-collar workers: A cluster randomized controlled trial. Journal of Occupational and Environmental Medicine, 51(5), 554–563. 10.1097/JOM.0b013e3181a24d28
- Van der Klink, J., Blonk, R., Schene, A. H., & Van Dijk, F. (2001). The benefits of interventions for work-related stress. American Journal of Public Health, 91(2), 270–276. doi:10.2105/ajph.91.2.270
- Visser, M. (2007). Deutero-learning in organizations: A review and a reformulation. Academy of Management Review, 32(2), 659–667. 10.5465/amr.2007.24351883
- Vogt, W. P., & Johnson, B. (2011). Dictionary of statistics & methodology: A nontechnical guide for the social sciences. Sage.
- von Thiele Schwarz, U., & Hasson, H. (2013). Alignment for achieving a healthy organization. In Bauer G. F. & Jenny G. J. (Eds.), Salutogenic organizations and change (pp. 107–125). Springer.
- von Thiele Schwarz, U., Richter, A., & Hasson, H. (2018). Getting on the same page - Co-created program logic (COP). In Nielsen K. & Noblet A. (Eds.), Organizational interventions for health and well-being: A handbook for evidence-based practice (pp. 42–67). Routledge.
- von Thiele Schwarz, U., Augustsson, H., Hasson, H., & Stenfors-Hayes, T. (2015). Promoting employee health by integrating health protection, health promotion, and continuous improvement: A longitudinal quasi-experimental intervention study. Journal of Occupational and Environmental Medicine, 57(2), 217–225. 10.1097/JOM.0000000000000344
- von Thiele Schwarz, U., Hasson, H., & Aarons, G. (2019). The Value Equation: Three complementary propositions for reconciling adaptation and fidelity in evidence-based practice implementation. BMC Health Service Research, 19, 868. 10.1186/s12913-019-4668-y
- von Thiele Schwarz, U., Lundmark, R., & Hasson, H. (2016). The Dynamic Integrated Evaluation Model (DIEM): Achieving Sustainability in Organizational Intervention through a Participatory Evaluation Approach. Stress & Health special issue on Organizational health interventions – advances in evaluation methodology, 32(4), 285–293. 10.1002/smi.2701
- von Thiele Schwarz, U., Nielsen, K. M., Stenfors-Hayes, T., & Hasson, H. (2017). Using kaizen to improve employee well-being: Results from two organizational intervention studies. Human Relations, 70(8), 966–993. 10.1177/0018726716677071
- Wåhlin-Jacobsen, C. D. (2019). The terms of “becoming empowered”: How ascriptions and negotiations of employee identities shape the outcomes of workplace voice activities. Scandinavian Journal of Management, 35(3), 101059. 10.1016/j.scaman.2019.101059
- Weick, K. E. (1995). Sensemaking in organizations (Foundations for organizational science). Sage Publications Inc.
- Wilder, D. A., Austin, J., & Casella, S. (2009). Applying behavior analysis in organizations: Organizational behavior management. Psychological Services, 6(3), 202–211. 10.1037/a0015393
- Wilkinson, G., & Dale, B. (1999). Integrated management systems: An examination of the concept and theory. The TQM Magazine, 11(2), 95–104. 10.1108/09544789910257280
- Wilkinson, G., & Dale, B. (2002). An examination of the ISO 9001: 2000 standard and its influence on the integration of management systems. Production Planning & Control, 13(3), 284–297. 10.1080/09537280110086361
- Woodcock, T., Adeleke, Y., Goeschel, C., Pronovost, P., & Dixon-Woods, M. (2020). A modified Delphi study to identify the features of high quality measurement plans for healthcare improvement projects. BMC Medical Research Methodology, 20(1), 1–9. 10.1186/s12874-019-0886-6
- Yin, R. K. (2013). Case study research: Design and methods. Sage.
- Zwetsloot, G. I. (1995). Improving cleaner production by integration into the management of quality, environment and working conditions. Journal of Cleaner Production, 3(1), 61–66. 10.1016/0959-6526(95)00046-H
Source: PubMed