Theory of Change: a theory-driven approach to enhance the Medical Research Council's framework for complex interventions

Mary J De Silva, Erica Breuer, Lucy Lee, Laura Asher, Neerja Chowdhary, Crick Lund, Vikram Patel, Mary J De Silva, Erica Breuer, Lucy Lee, Laura Asher, Neerja Chowdhary, Crick Lund, Vikram Patel

Abstract

Background: The Medical Research Councils' framework for complex interventions has been criticized for not including theory-driven approaches to evaluation. Although the framework does include broad guidance on the use of theory, it contains little practical guidance for implementers and there have been calls to develop a more comprehensive approach. A prospective, theory-driven process of intervention design and evaluation is required to develop complex healthcare interventions which are more likely to be effective, sustainable and scalable.

Methods: We propose a theory-driven approach to the design and evaluation of complex interventions by adapting and integrating a programmatic design and evaluation tool, Theory of Change (ToC), into the MRC framework for complex interventions. We provide a guide to what ToC is, how to construct one, and how to integrate its use into research projects seeking to design, implement and evaluate complex interventions using the MRC framework. We test this approach by using ToC within two randomized controlled trials and one non-randomized evaluation of complex interventions.

Results: Our application of ToC in three research projects has shown that ToC can strengthen key stages of the MRC framework. It can aid the development of interventions by providing a framework for enhanced stakeholder engagement and by explicitly designing an intervention that is embedded in the local context. For the feasibility and piloting stage, ToC enables the systematic identification of knowledge gaps to generate research questions that strengthen intervention design. ToC may improve the evaluation of interventions by providing a comprehensive set of indicators to evaluate all stages of the causal pathway through which an intervention achieves impact, combining evaluations of intervention effectiveness with detailed process evaluations into one theoretical framework.

Conclusions: Incorporating a ToC approach into the MRC framework holds promise for improving the design and evaluation of complex interventions, thereby increasing the likelihood that the intervention will be ultimately effective, sustainable and scalable. We urge researchers developing and evaluating complex interventions to consider using this approach, to evaluate its usefulness and to build an evidence base to further refine the methodology.

Trial registration: Clinical trials.gov: NCT02160249.

Figures

Figure 1
Figure 1
SHARE Theory of Change: peer counselling for maternal depression in Goa, India.
Figure 2
Figure 2
How Theory of Change can be used to strengthen the MRC framework. Adapted from Craig et al.[1].

References

    1. Craig P, Dieppe P, Macintyre S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. Br Med J. 2008;337:a1655.
    1. Craig P, Petticrew M. Developing and evaluating complex interventions: reflections on the 2008 MRC guidance. Int J Nurs Stud. 2013;50:585–587.
    1. Anderson R. New MRC guidance on evaluating complex interventions. Br Med J. 2008;337:a1937.
    1. Bonell C, Fletcher A, Morton M, Lorenc T, Moore L. Realist randomized controlled trials: a new approach to evaluating complex public health interventions. Soc Sci Med. 2012;75:2299–2306.
    1. Ling T. Evaluating complex and unfolding interventions in real time. Evaluation. 2012;18:79–91.
    1. Coryn CLS, Noakes LA, Westine CD, Schrote DC. A systematic review of theory-driven evaluation practice from 1990 to 2009. Am J Eval. 2011;32:199–226.
    1. Kirkpatrick DL. Techniques for Evaluating Training Programmes”. J Am Soc for Train Dev. 1959;11:1–13.
    1. Chen HT. Theory Driven Evaluation. Thousand Oaks, California: Sage; 1990.
    1. James C. Theory of Change Review: A report comissioned by Comic Relief. 2011.
    1. Weiss CH. In: New Approaches to Evaluating Community Initiatives Volume 1 Concepts, Methods and Contexts. Connell JP, Kubisch AC, Schorr LB, Weiss CH, editor. Washington DC: The Aspen Institute; 1995. Nothing As Practical As Good Theory: Exploring Theory-Based Evaluation For Comprehensive Community Initiatives For Children And Families; pp. 65–92.
    1. Connell JP, Kubisch AC. New Approaches to Evaluating Community Initiatives, Volume 2: Theory, Measurement, and Analysis. Washington DC: The Aspen Institute; 1998. Applying a Theory of Change Approach to the Evaluation of Comprehensive Community Initiatives: Progress, Prospects, and Problems.
    1. Anderson A. The Community Builder’s Approach to Theory of Change. A Practical Guide to Theory Development. New York: The Aspen Institute; 2005.
    1. Vogel I. Review of the use of Theory of Change in International Development. London: UK Department of International Development; 2012.
    1. Sullivan H, Barnes HM, Matka E. Building collaborative capacity through `Theories of Change': early lessons from the evaluation of Health Action Zones in England. Evaluation. 2002;8:205–226.
    1. Mason P, Barnes M. Constructing theories of Change: methods and sources. Evaluation. 2007;13:151–170.
    1. Weitzman BC, Mijanovich T, Silver D, Brecher C. Finding the impact in a messy intervention: using an integrated design to evaluate a comprehensive citywide health initiative. Am J Eval. 2009;30:495–514.
    1. Weitzman BC, Silver D, Dillman KN. Integrating a comparison group design into a Theory of Change evaluation: the case of the urban health initiative. Am J Eval. 2002;23:371–385.
    1. Cole M. The Health Action Zone initiative: lessons from Plymouth. Local Govern Stud. 2003;29:99–117.
    1. Kernik D. Wanted—new methodologies for health service research. Is complexity theory the answer? Fam Pract. 2006;23:385–390.
    1. Ajzen I. The theory of planned behavior. Organ Behav Hum Decis Process. 1991;50:179–211.
    1. Murray M, Treweek S, Pope C, MacFarlane A, Ballini L, Dowrick C, Finch T, Kennedy A, Mair F, O'Donnell C, Ong B, Rapley T, Rogers A, May C. Normalisation process theory: a framework for developing, evaluating and implementing complex interventions. BMC Med. 2010;8:63.
    1. Department for International Development. How to Note: Guidance on using revised Logical Framework. London; 2011.
    1. Clark H, Andersen A. Theories of Change and Logic Models: Telling Them Apart. 2004 Presented at the American Evaluation Association Conference. Atlanta, Georgia; 2004.
    1. Rahman A, Malik A, Sikander S, Roberts C, Creed F. Cognitive behaviour therapy-based intervention by community health workers for mothers with depression and their infants in rural Pakistan: a cluster-randomized controlled trial. Lancet. 2008;372:902–909.
    1. Taplin D, Rasic M. Theory of Change Technical Papers: A series of papers to support development of theories of change based on practice in the field. New York: ActKnowledge; 2013.
    1. Taplin DH, Rasic M. Facilitator's Source Book: Source Book for facilitators leading Theory of Change development sessions. New York: ActKnowledge; 2012.
    1. Grant A, Treweek S, Dreischulte T, Foy R, Guthrie B. Process evaluations for cluster-randomized trials of complex interventions: a proposed framework for design and reporting. Trials. 2013;14:15.
    1. Bagozzi R, Yi Y. Specification, evaluation, and interpretation of structural equation models. J Acad Market Sci. 2012;40:8–34.
    1. Robinson S. Simulation - The practice of model development and use. Chichester, UK: Wiley; 2004.
    1. Bonabeau E. Agent-based modeling: methods and techniques for simulating human systems. Proc National Acad Sci. 2002;99:7280–7287.
    1. Sterman J. System dynamics modeling: tools for learning in a complex world. Calif Manage Rev. 2001;43:8–25.
    1. Campbell NC, Murray E, Darbyshire J, Emery J, Farmer A, Griffiths F, Guthrie B, Lester H, Wilson P, AL K. Designing and evaluating complex interventions to improve health care. Br Med J. 2007;334:455–459.
    1. WHO. Community based rehabilitation: CBR guidelines. Geneva: World Health Organization; 2010.
    1. Chatterjee S, Naik SS, John S, Dabholkar H, Balaji M, Koschorke M, Varghese M, Thara R, Weiss HA, Williams P, McCrone P, Patel V, Thornicroft G. Effectiveness of a community-based intervention for people with schizophrenia and their caregivers in India (COPSI): a randomized controlled trial. Lancet. 2014;383:1385–1394.
    1. Lund C, Jordans M, Petersen I, Bhana A, Kigozi F, Prince M, Thornicroft G, Hanlon C, Kakuma R, McDaid D, Saxena S, Chisholm D, Raja S, Kippen-Wood S, Honikman S, Fairall L, Patel V. PRIME: A programme to reduce the treatment gap for mental disorders in five low- and middle-income countries. PLoS Med. 2012;9:e1001359.
    1. Breuer E, De Silva M, Fekadu A, Luitel N, Murhar V, Nakku J, Petersen I, Lund C. Using workshops to develop Theories of Change in five low and middle income countries: lessons from the Programme for Improving Mental Health Care (PRIME) Intl J Ment Health Syst. 2014;8:15.
    1. Hanlon C, Luitel NP, Kathree T, Murhar V, Shrivasta S, Medhin G, Ssebunnya J, Fekadu A, Shidhaye R, Petersen I, Jordans M, Kigozi F, Thornicroft G, Patel V, Tomlinson M, Lund C, Breuer E, De Silva M, Prince M. Challenges and opportunities for implementing integrated mental health care: a district level situation analysis from five low- and middle-income countries. PLoS One. 2014;9:e88437.
    1. Hawe P, Shiell A, Riley T, Gold L. Methods for exploring implementation variation and local context within a cluster randomized community intervention trial. J Epidemiol Community Health. 2004;58:788–793.
    1. Segal L, Sara Opie R, Dalziel K. Theory! The missing link in understanding the performance of neonate/infant home-visiting programs to prevent child maltreatment: a systematic review. Milbank Q. 2012;90:47–106.
    1. Anderson L, Anderson L, Petticrew M, Rehfuess E, Armstrong R, Ueffing E, Baker P, Francis D, Tugwell P. Using logic models to capture complexity in systematic reviews. Res Synth Meth. 2011;2:33–42.
    1. Afifi R, Makhoul J, El Hajj T, Nakkash R. Developing a logic model for youth mental health: participatory research with a refugee community in Beirut. Health Policy Plan. 2011;26:508–517.
    1. Hernandez M, Hodges S. Applying a theory of change approach to interagency planning in child mental health. Am J Community Psychol. 2006;38:165–173.
    1. Chandani Y, Noel M, Pomeroy A, Andersson S, Pahl M, Williams T. Factors affecting availability of essential medicines among community health workers in Ethiopia, Malawi, and Rwanda: solving the last mile puzzle. Am J Trop Med Hyg. 2012;87:120–126.
    1. Hawkins J, Brown E, Oesterle S, Arthur M, Abbott R, Catalano R. Early effects of Communities That Care on targeted risks and initiation of delinquent behavior and substance use. J Adolesc Health. 2008;43:15–22.
    1. Bickman L. The application of program theory to the evaluation of a managed mental health care system. Eval Program Plann. 1996;19:111–119.
    1. Van Belle S, Marchal B, Dubourg D, Kegels G. How to develop a theory-driven evaluation design? Lessons learned from an adolescent sexual and reproductive health programme in West Africa. BMC Public Health. 2010;10:741.

Source: PubMed

Sponsorer och medarbetare

Medicinska tillstånd

Läkemedelsinterventioner

3
Prenumerera