A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance

Kathryn Skivington, Lynsay Matthews, Sharon Anne Simpson, Peter Craig, Janis Baird, Jane M Blazeby, Kathleen Anne Boyd, Neil Craig, David P French, Emma McIntosh, Mark Petticrew, Jo Rycroft-Malone, Martin White, Laurence Moore, Kathryn Skivington, Lynsay Matthews, Sharon Anne Simpson, Peter Craig, Janis Baird, Jane M Blazeby, Kathleen Anne Boyd, Neil Craig, David P French, Emma McIntosh, Mark Petticrew, Jo Rycroft-Malone, Martin White, Laurence Moore

Abstract

The UK Medical Research Council’s widely used guidance for developing and evaluating complex interventions has been replaced by a new framework, commissioned jointly by the Medical Research Council and the National Institute for Health Research, which takes account of recent developments in theory and methods and the need to maximise the efficiency, use, and impact of research.

Conflict of interest statement

Competing interests: All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/coi_disclosure.pdf and declare: support from the NIHR, MRC, and the funders listed above for the submitted work; KS has project grant funding from the Scottish Government Chief Scientist Office; SAS is a former member of the NIHR Health Technology Assessment Clinical Evaluation and Trials Programme Panel (November 2016 - November 2020) and member of the Chief Scientist Office Health HIPS Committee (since 2018) and NIHR Policy Research Programme (since November 2019), and has project grant funding from the Economic and Social Research Council, MRC, and NIHR; LMo is a former member of the MRC-NIHR Methodology Research Programme Panel (2015-19) and MRC Population Health Sciences Group (2015-20); JB is a member of the NIHR Public Health Research Funding Committee (since May 2019), and a core member (since 2016) and vice chairperson (since 2018) of a public health advisory committee of the National Institute for Health and Care Excellence; JMB is a former member of the NIHR Clinical Trials Unit Standing Advisory Committee (2015-19); DPF is a former member of the NIHR Public Health Research programme research funding board (2015-2019), the MRC-NIHR Methodology Research Programme panel member (2014-2018), and is a panel member of the Research Excellence Framework 2021, subpanel 2 (public health, health services, and primary care; November 2020 - February 2022), and has grant funding from the European Commission, NIHR, MRC, Natural Environment Research Council, Prevent Breast Cancer, Breast Cancer Now, Greater Sport, Manchester University NHS Foundation Trust, Christie Hospital NHS Trust, and BXS GP; EM is a member of the NIHR Public Health Research funding board; MP has grant funding from the MRC, UK Prevention Research Partnership, and NIHR; JR-M is programme director and chairperson of the NIHR’s Health Services Delivery Research Programme (since 2014) and member of the NIHR Strategy Board (since 2014); MW received a salary as director of the NIHR PHR Programme (2014-20), has grant funding from NIHR, and is a former member of the MRC’s Population Health Sciences Strategic Committee (July 2014 to June 2020). There are no other relationships or activities that could appear to have influenced the submitted work.

Figures

Fig 1
Fig 1
Framework for developing and evaluating complex interventions. Context=any feature of the circumstances in which an intervention is conceived, developed, evaluated, and implemented; programme theory=describes how an intervention is expected to lead to its effects and under what conditions—the programme theory should be tested and refined at all stages and used to guide the identification of uncertainties and research questions; stakeholders=those who are targeted by the intervention or policy, involved in its development or delivery, or more broadly those whose personal or professional interests are affected (that is, who have a stake in the topic)—this includes patients and members of the public as well as those linked in a professional capacity; uncertainties=identifying the key uncertainties that exist, given what is already known and what the programme theory, research team, and stakeholders identify as being most important to discover—these judgments inform the framing of research questions, which in turn govern the choice of research perspective; refinement=the process of fine tuning or making changes to the intervention once a preliminary version (prototype) has been developed; economic considerations=determining the comparative resource and outcome consequences of the interventions for those people and organisations affected

References

    1. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, Medical Research Council Guidance . Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 2008;337:a1655. 10.1136/bmj.a1655.
    1. Craig P, Dieppe P, Macintyre S, et al. . Developing and evaluating complex interventions: new guidance. Medical Research Council, 2006.
    1. Campbell M, Fitzpatrick R, Haines A, et al. . Framework for design and evaluation of complex interventions to improve health. BMJ 2000;321:694-6. 10.1136/bmj.321.7262.694.
    1. O’Cathain A, Croot L, Duncan E, et al. . Guidance on how to develop complex interventions to improve health and healthcare. BMJ Open 2019;9:e029954. 10.1136/bmjopen-2019-029954
    1. Craig P, Cooper C, Gunnell D, et al. . Using natural experiments to evaluate population health interventions: new Medical Research Council guidance. J Epidemiol Community Health 2012;66:1182-6. 10.1136/jech-2011-200375
    1. Craig P, Ruggiero ED, Frohlich KL, et al. . Taking account of context in population health intervention research: guidance for producers, users and funders of research. NIHR Journals Library, 201810.3310/CIHR-NIHR-01 .
    1. Moore GF, Audrey S, Barker M, et al. . Process evaluation of complex interventions: Medical Research Council guidance. BMJ 2015;350:h1258. 10.1136/bmj.h1258
    1. Moore G, Campbell M, Copeland L, et al. . Adapting interventions to new contexts-the ADAPT guidance. BMJ 2021;374:n1679. 10.1136/bmj.n1679
    1. Skivington K, Matthews L, Simpson SA, et al. . Framework for the development and evaluation of complex interventions: gap analysis, workshop and consultation-informed update. Health Technol Assess 2021. [forthcoming].
    1. Chng NR, Hawkins K, Fitzpatrick B, et al. . Implementing social prescribing in primary care in areas of high socioeconomic deprivation: process evaluation of the ‘Deep End’ community links worker programme. Br J Gen Pract 2021;1153:BJGP.2020.1153. 10.3399/BJGP.2020.1153
    1. Mercer SW, Fitzpatrick B, Grant L, et al. . Effectiveness of Community-Links Practitioners in Areas of High Socioeconomic Deprivation. Ann Fam Med 2019;17:518-25. 10.1370/afm.2429
    1. Hawe P, Shiell A, Riley T. Complex interventions: how “out of control” can a randomised controlled trial be? BMJ 2004;328:1561-3. 10.1136/bmj.328.7455.1561
    1. Blencowe NS, Mills N, Cook JA, et al. . Standardizing and monitoring the delivery of surgical interventions in randomized clinical trials. Br J Surg 2016;103:1377-84. 10.1002/bjs.10254
    1. Blencowe NS, Skilton A, Gaunt D, et al. ROMIO Study team . Protocol for developing quality assurance measures to use in surgical trials: an example from the ROMIO study. BMJ Open 2019;9:e026209. 10.1136/bmjopen-2018-026209
    1. Greenhalgh T, Papoutsi C. Studying complexity in health services research: desperately seeking an overdue paradigm shift. BMC Med 2018;16:95. 10.1186/s12916-018-1089-4
    1. Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. Am J Community Psychol 2009;43:267-76. 10.1007/s10464-009-9229-9
    1. Petticrew M. When are complex interventions ‘complex’? When are simple interventions ‘simple’? Eur J Public Health 2011;21:397-8. 10.1093/eurpub/ckr084
    1. Anderson R. New MRC guidance on evaluating complex interventions. BMJ 2008;337:a1937. 10.1136/bmj.a1937
    1. Pawson R, Tilley N. Realistic Evaluation. Sage, 1997.
    1. Campbell R, Starkey F, Holliday J, et al. . An informal school-based peer-led intervention for smoking prevention in adolescence (ASSIST): a cluster randomised trial. Lancet 2008;371:1595-602. 10.1016/S0140-6736(08)60692-3
    1. Randell R, Honey S, Hindmarsh J, et al. A realist process evaluation of robot-assisted surgery: integration into routine practice and impacts on communication, collaboration and decision-making. NIHR Journals Library, 2017. .
    1. Rutter H, Savona N, Glonti K, et al. . The need for a complex systems model of evidence for public health. Lancet 2017;390:2602-4. 10.1016/S0140-6736(17)31267-9
    1. The Health Foundation. Evidence Scan. Complex adaptive systems. Health Foundation 2010. .
    1. Wiggins M, Bonell C, Sawtell M, et al. . Health outcomes of youth development programme in England: prospective matched comparison study. BMJ 2009;339:b2534. 10.1136/bmj.b2534
    1. Robinson M, Geue C, Lewsey J, et al. . Evaluating the impact of the alcohol act on off-trade alcohol sales: a natural experiment in Scotland. Addiction 2014;109:2035-43. 10.1111/add.12701
    1. Raine R, Fitzpatrick R, de Pury J. Challenges, solutions and future directions in evaluative research. J Health Serv Res Policy 2016;21:215-6. 10.1177/1355819616664495
    1. Kessler R, Glasgow RE. A proposal to speed translation of healthcare research into practice: dramatic change is needed. Am J Prev Med 2011;40:637-44. 10.1016/j.amepre.2011.02.023
    1. Folegatti PM, Ewer KJ, Aley PK, et al. Oxford COVID Vaccine Trial Group . Safety and immunogenicity of the ChAdOx1 nCoV-19 vaccine against SARS-CoV-2: a preliminary report of a phase 1/2, single-blind, randomised controlled trial. Lancet 2020;396:467-78. 10.1016/S0140-6736(20)31604-4
    1. Voysey M, Clemens SAC, Madhi SA, et al. Oxford COVID Vaccine Trial Group . Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. Lancet 2021;397:99-111. 10.1016/S0140-6736(20)32661-1
    1. Soi C, Shearer JC, Budden A, et al. . How to evaluate the implementation of complex health programmes in low-income settings: the approach of the Gavi Full Country Evaluations. Health Policy Plan 2020;35(Supplement_2):ii35-46. 10.1093/heapol/czaa127
    1. Burgess RA, Osborne RH, Yongabi KA, et al. . The COVID-19 vaccines rush: participatory community engagement matters more than ever. Lancet 2021;397:8-10. 10.1016/S0140-6736(20)32642-8
    1. Paltiel AD, Schwartz JL, Zheng A, Walensky RP. Clinical Outcomes Of A COVID-19 Vaccine: Implementation Over Efficacy. Health Aff (Millwood) 2021;40:42-52. 10.1377/hlthaff.2020.02054
    1. Dalkin S, Lhussier M, Williams L, et al. . Exploring the use of Soft Systems Methodology with realist approaches: A novel way to map programme complexity and develop and refine programme theory. Evaluation 2018;24:84-97. 10.1177/1356389017749036 .
    1. Mann C, Shaw ARG, Guthrie B, et al. . Can implementation failure or intervention failure explain the result of the 3D multimorbidity trial in general practice: mixed-methods process evaluation. BMJ Open 2019;9:e031438. 10.1136/bmjopen-2019-031438
    1. French C, Pinnock H, Forbes G, Skene I, Taylor SJC. Process evaluation within pragmatic randomised controlled trials: what is it, why is it done, and can we find it?-a systematic review. Trials 2020;21:916. 10.1186/s13063-020-04762-9
    1. Penney T, Adams J, Briggs A, et al. Evaluation of the impacts on health of the proposed UK industry levy on sugar sweetened beverages: developing a systems map and data platform, and collection of baseline and early impact data. National Institute for Health Research, 2018. .
    1. Hoddinott P, Britten J, Pill R. Why do interventions work in some places and not others: a breastfeeding support group trial. Soc Sci Med 2010;70:769-78. 10.1016/j.socscimed.2009.10.067
    1. Funnell SC, Rogers PJ. Purposeful program theory: effective use of theories of change and logic models. 1st ed. Jossey-Bass, 2011.
    1. Lawless A, Baum F, Delany-Crowe T, et al. . Developing a Framework for a Program Theory-Based Approach to Evaluating Policy Processes and Outcomes: Health in All Policies in South Australia. Int J Health Policy Manag 2018;7:510-21. 10.15171/ijhpm.2017.121
    1. Stephens TJ, Peden CJ, Pearse RM, et al. EPOCH trial group . Improving care at scale: process evaluation of a multi-component quality improvement intervention to reduce mortality after emergency abdominal surgery (EPOCH trial). Implement Sci 2018;13:142. 10.1186/s13012-018-0823-9
    1. Bonell C, Jamal F, Melendez-Torres GJ, Cummins S. ‘Dark logic’: theorising the harmful consequences of public health interventions. J Epidemiol Community Health 2015;69:95-8. 10.1136/jech-2014-204671
    1. Maini R, Mounier-Jack S, Borghi J. How to and how not to develop a theory of change to evaluate a complex intervention: reflections on an experience in the Democratic Republic of Congo. BMJ Glob Health 2018;3:e000617. 10.1136/bmjgh-2017-000617
    1. Cook PA, Hargreaves SC, Burns EJ, et al. . Communities in charge of alcohol (CICA): a protocol for a stepped-wedge randomised control trial of an alcohol health champions programme. BMC Public Health 2018;18:522. 10.1186/s12889-018-5410-0
    1. Ebenso B, Manzano A, Uzochukwu B, et al. . Dealing with context in logic model development: Reflections from a realist evaluation of a community health worker programme in Nigeria. Eval Program Plann 2019;73:97-110. 10.1016/j.evalprogplan.2018.12.002
    1. White M, Cummins S, Raynor M, et al. Evaluation of the health impacts of the UK Treasury Soft Drinks Industry Levy (SDIL) Project Protocol. NIHR Journals Library, 2018. .
    1. National Institute for Health and Care Excellence. What is public involvement in research? – INVOLVE. .
    1. Jones S, Riste L, Barrowclough C, et al. . Reducing relapse and suicide in bipolar disorder: practical clinical approaches to identifying risk, reducing harm and engaging service users in planning and delivery of care – the PARADES (Psychoeducation, Anxiety, Relapse, Advance Directive Evaluation and Suicidality) programme. Programme Grants for Applied Research 2018;6:1-296. 10.3310/pgfar06060
    1. Moodie R, Stuckler D, Monteiro C, et al. Lancet NCD Action Group . Profits and pandemics: prevention of harmful effects of tobacco, alcohol, and ultra-processed food and drink industries. Lancet 2013;381:670-9. 10.1016/S0140-6736(12)62089-3
    1. Yardley L, Ainsworth B, Arden-Close E, Muller I. The person-based approach to enhancing the acceptability and feasibility of interventions. Pilot Feasibility Stud 2015;1:37. 10.1186/s40814-015-0033-z
    1. Barnett ML, Dopp AR, Klein C, Ettner SL, Powell BJ, Saldana L. Collaborating with health economists to advance implementation science: a qualitative study. Implement Sci Commun 2020;1:82. 10.1186/s43058-020-00074-w
    1. National Institute for Health and Care Excellence. Developing NICE guidelines: the manual. NICE, 2014. .
    1. Boyd KA, Balogun MO, Minnis H. Development of a radical foster care intervention in Glasgow, Scotland. Health Promot Int 2016;31:665-73. 10.1093/heapro/dav041
    1. Deidda M, Boyd KA, Minnis H, et al. BeST study team . Protocol for the economic evaluation of a complex intervention to improve the mental health of maltreated infants and children in foster care in the UK (The BeST? services trial). BMJ Open 2018;8:e020066. 10.1136/bmjopen-2017-020066
    1. Escoffery C, Lebow-Skelley E, Haardoerfer R, et al. . A systematic review of adaptations of evidence-based public health interventions globally. Implement Sci 2018;13:125. 10.1186/s13012-018-0815-9
    1. Stirman SW, Miller CJ, Toder K, Calloway A. Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implement Sci 2013;8:65. 10.1186/1748-5908-8-65
    1. Forsyth R, Purcell C, Barry S, et al. . Peer-led intervention to prevent and reduce STI transmission and improve sexual health in secondary schools (STASH): protocol for a feasibility study. Pilot Feasibility Stud 2018;4:180. 10.1186/s40814-018-0354-9
    1. White J, Hawkins J, Madden K, et al. . Adapting the ASSIST model of informal peer-led intervention delivery to the Talk to FRANK drug prevention programme in UK secondary schools (ASSIST + FRANK): intervention development, refinement and a pilot cluster randomised controlled trial. Public Health Research 2017;5:1-98. 10.3310/phr05070
    1. Evans RE, Moore G, Movsisyan A, Rehfuess E, ADAPT Panel. ADAPT Panel comprises of Laura Arnold . How can we adapt complex population health interventions for new contexts? Progressing debates and research priorities. J Epidemiol Community Health 2021;75:40-5. 10.1136/jech-2020-214468.
    1. Hawkes RE, Miles LM, French DP. The theoretical basis of a nationally implemented type 2 diabetes prevention programme: how is the programme expected to produce changes in behaviour? Int J Behav Nutr Phys Act 2021;18:64. 10.1186/s12966-021-01134-7
    1. Ogilvie D, Adams J, Bauman A, et al. . Using natural experimental studies to guide public health action: turning the evidence-based medicine paradigm on its head. SocArXiv 201910.31235/.
    1. Eldridge SM, Chan CL, Campbell MJ, et al. PAFS consensus group . CONSORT 2010 statement: extension to randomised pilot and feasibility trials. BMJ 2016;355:i5239. 10.1136/bmj.i5239
    1. Thabane L, Hopewell S, Lancaster GA, et al. . Methods and processes for development of a CONSORT extension for reporting pilot randomized controlled trials. Pilot Feasibility Stud 2016;2:25. 10.1186/s40814-016-0065-z
    1. Craig P, Campbell M. Evaluability Assessment: a systematic approach to deciding whether and how to evaluate programmes and policies. Evaluability Assessment working paper. 2015.
    1. Ogilvie D, Cummins S, Petticrew M, White M, Jones A, Wheeler K. Assessing the evaluability of complex public health interventions: five questions for researchers, funders, and policymakers. Milbank Q 2011;89:206-25. 10.1111/j.1468-0009.2011.00626.x
    1. Expected Value of Perfect Information (EVPI). YHEC - York Health Econ. Consort. .
    1. Deaton A, Cartwright N. Understanding and misunderstanding randomized controlled trials. Soc Sci Med 2018;210:2-21. 10.1016/j.socscimed.2017.12.005
    1. Rosas S, Knight E. Evaluating a complex health promotion intervention: case application of three systems methods. Crit Public Health 2019;29:337-52. 10.1080/09581596.2018.1455966 .
    1. McKee M, Britton A, Black N, McPherson K, Sanderson C, Bain C. Methods in health services research. Interpreting the evidence: choosing between randomised and non-randomised studies. BMJ 1999;319:312-5. 10.1136/bmj.319.7205.312
    1. Burnett T, Mozgunov P, Pallmann P, Villar SS, Wheeler GM, Jaki T. Adding flexibility to clinical trial designs: an example-based guide to the practical use of adaptive designs. BMC Med 2020;18:352. 10.1186/s12916-020-01808-2
    1. Collins LM, Murphy SA, Strecher V. The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART): new methods for more potent eHealth interventions. Am J Prev Med 2007;32(Suppl):S112-8. 10.1016/j.amepre.2007.01.022
    1. McDonald S, Quinn F, Vieira R, et al. . The state of the art and future opportunities for using longitudinal n-of-1 methods in health behaviour research: a systematic literature overview. Health Psychol Rev 2017;11:307-23. 10.1080/17437199.2017.1316672
    1. Green BB, Coronado GD, Schwartz M, Coury J, Baldwin LM. Using a continuum of hybrid effectiveness-implementation studies to put research-tested colorectal screening interventions into practice. Implement Sci 2019;14:53. 10.1186/s13012-019-0903-5
    1. Tugwell P, Knottnerus JA, McGowan J, Tricco A. Big-5 Quasi-Experimental designs. J Clin Epidemiol 2017;89:1-3. 10.1016/j.jclinepi.2017.09.010
    1. Egan M, McGill E, Penney T, et al. NIHR SPHR Guidance on Systems Approaches to Local Public Health Evaluation. Part 1: Introducing systems thinking. NIHR School for Public Health Research, 2019. .
    1. Bonell C, Fletcher A, Morton M, Lorenc T, Moore L. Realist randomised controlled trials: a new approach to evaluating complex public health interventions. Soc Sci Med 2012;75:2299-306. 10.1016/j.socscimed.2012.08.032
    1. McGill E, Marks D, Er V, Penney T, Petticrew M, Egan M. Qualitative process evaluation from a complex systems perspective: A systematic review and framework for public health evaluators. PLoS Med 2020;17:e1003368. 10.1371/journal.pmed.1003368
    1. Bicket M, Christie I, Gilbert N, et al. Magenta Book 2020 Supplementary Guide: Handling Complexity in Policy Evaluation. Lond HM Treas 2020.
    1. Pfadenhauer LM, Gerhardus A, Mozygemba K, et al. . Making sense of complexity in context and implementation: the Context and Implementation of Complex Interventions (CICI) framework. Implement Sci 2017;12:21. 10.1186/s13012-017-0552-5
    1. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care 2012;50:217-26. 10.1097/MLR.0b013e3182408812
    1. Landes SJ, McBain SA, Curran GM. An introduction to effectiveness-implementation hybrid designs. Psychiatry Res 2019;280:112513. 10.1016/j.psychres.2019.112513
    1. Imison C, Curry N, Holder H, et al. Shifting the balance of care: great expectations. Research report. Nuffield Trust.
    1. Remme M, Martinez-Alvarez M, Vassall A. Cost-Effectiveness Thresholds in Global Health: Taking a Multisectoral Perspective. Value Health 2017;20:699-704. 10.1016/j.jval.2016.11.009
    1. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet 2009;374:86-9. 10.1016/S0140-6736(09)60329-9

Source: PubMed

스폰서 및 공동 작업자

건강 상태

약물 개입

3
구독하다