Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting

Aileen Grant, Shaun Treweek, Tobias Dreischulte, Robbie Foy, Bruce Guthrie, Aileen Grant, Shaun Treweek, Tobias Dreischulte, Robbie Foy, Bruce Guthrie

Abstract

Background: Process evaluations are recommended to open the 'black box' of complex interventions evaluated in trials, but there is limited guidance to help researchers design process evaluations. Much current literature on process evaluations of complex interventions focuses on qualitative methods, with less attention paid to quantitative methods. This discrepancy led us to develop our own framework for designing process evaluations of cluster-randomised controlled trials.

Methods: We reviewed recent theoretical and methodological literature and selected published process evaluations; these publications identified a need for structure to help design process evaluations. We drew upon this literature to develop a framework through iterative exchanges, and tested this against published evaluations.

Results: The developed framework presents a range of candidate approaches to understanding trial delivery, intervention implementation and the responses of targeted participants. We believe this framework will be useful to others designing process evaluations of complex intervention trials. We also propose key information that process evaluations could report to facilitate their identification and enhance their usefulness.

Conclusion: There is no single best way to design and carry out a process evaluation. Researchers will be faced with choices about what questions to focus on and which methods to use. The most appropriate design depends on the purpose of the process evaluation; the framework aims to help researchers make explicit their choices of research questions and methods.

Trial registration: Clinicaltrials.gov NCT01425502.

Figures

Figure 1
Figure 1
Framework model for designing process evaluations of cluster-randomised controlled trials.

References

    1. Medical Research Council. Developing and Evaluating Complex Interventions: New Guidance. London: Medical Research Council; 2008.
    1. Hawe P, Shiell A, Riley T, Gold L. Methods for exploring implementation variation and local context within a cluster randomised community intervention trial. J Epidemiol Community Health. 2004;58:788–793. doi: 10.1136/jech.2003.014415.
    1. Oakley A, Strange V, Bonell C, Allen E, Stephenson J. RIPPLE Study Team. Process evaluation in randomised controlled trials of complex interventions. Br Med J. 2006;332:413–416. doi: 10.1136/bmj.332.7538.413.
    1. Linnan L, Steckler A. In: Process Evaluation for Public Health Interventions and Research. Steckler A, Linnan L, editor. San Francisco: Jossey-Bass; 2002. Process evaluation for public health interventions and research: an overview; pp. 1–23.
    1. Wight D, Obasi A. In: Effective Sexual Health Interventions: Issues in Experimental Evaluation. Stephenson JM, Imrie J, Bonell C, editor. Oxford: Oxford University Press; 2003. Unpacking the 'black box': the importance of process data to explain outcomes; pp. 151–166.
    1. Ellard D, Parsons S. In: Evaluating Health Promotion Practice and Methods. Thorogood M, Coombes Y, editor. Oxford: Oxford University Press; 2010. Process evaluation: understanding how and why interventions work.
    1. Craigie A, Caswell S, Paterson C, Treweek S, Belch J, Daly F, Rodger J, Thompson J, Kirk A, Ludbrook A, Stead M, Wardle J, Steele R, Anderson A. Study protocol for BeWEL: the impact of a BodyWEight and physicaL activity intervention on adults at risk of developing colorectal adenomas. BMC Public Health. 2011;11:184. doi: 10.1186/1471-2458-11-184.
    1. Protheroe J, Bower P, Chew-Graham C. The use of mixed methodology in evaluating complex interventions: identifying patient factors that moderate the effects of a decision aid. Fam Pract. 2007;24:594–600. doi: 10.1093/fampra/cmm066.
    1. Avery AJ, Rodgers S, Cantrill JA, Armstrong S, Cresswell K, Eden M, Elliott RA, Howard R, Kendrick D, Morris CJ, Prescott RJ, Swanwick G, Franklin M, Putman K, Boyd M, Sheikh A. A pharmacist-led information technology intervention for medication errors (PINCER): a multicentre, cluster randomised, controlled trial and cost-effectiveness analysis. Lancet. 2012;379:1310–1319. doi: 10.1016/S0140-6736(11)61817-5.
    1. Medical Research Council. A Framework for the Development and Evaluation of RCTs for Complex Interventions to Improve Health. London: Medical Research Council; 2000.
    1. Byng R, Norman I, Redfern S, Jones R. Exposing the key functions of a complex intervention for shared care in mental health: case study of a process evaluation. BMC Health Serv Res. 2008;8:274. doi: 10.1186/1472-6963-8-274.
    1. Jansen Y, de Bont A, Foets M, Bruijnzeels M, Bal R. Tailoring intervention procedures to routine primary health care practice; an ethnographic process evaluation. BMC Health Serv Res. 2007;7:125. doi: 10.1186/1472-6963-7-125.
    1. Dreischulte T, Grant A, Donnan P, McCowan C, Davey P, Petrie D, Treweek S, Guthrie B. A cluster randomised stepped wedge trial to evaluate the effectiveness of a multifaceted information technology-bsed intervention in reducing high-risk prescribing of non-steroidal anti-inflammatory drugs and antiplatelets in primary care: the DQIP study protocol. Implement Sci. 2012;7:24. doi: 10.1186/1748-5908-7-24.
    1. Donnar A, Klar N. Design and Analysis of Cluster Randomization Trials in Health Research. London: Arnold; 2000.
    1. Donnar A, Klar N. Pitfalls of and controversies in cluster randomization trials. Am J Public Health. 2004;94:416–422. doi: 10.2105/AJPH.94.3.416.
    1. Ivers N, Jamtvedt G, Flottorp S, Young J, Odgaard-Jensen J, French S, O'Brien M, Johansen M, Grimshaw J, Oxman A. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;6:CD000259.
    1. Ellard D, Taylor S, Parsons S, Thorogood M. The OPERA trial: a protocol for the process evaluation of a randomised trial of an exercise intervention for older people in residential and nursing accommodation. Trials. 2011;12:28. doi: 10.1186/1745-6215-12-28.
    1. Underwood M, Eldridge S, Lamb S, Potter R, Sheehan B, Slowther A-M, Taylor S, Thorogood M, Weich S. The OPERA trial: protocol for a randomised trial of an exercise intervention for older people in residential and nursing accommodation. Trials. 2011;12:27. doi: 10.1186/1745-6215-12-27.
    1. Glasgow RE, McKay HG, Piette JD, Reynolds KD. The RE-AIM framework for evaluating interventions: what can it tell us about approaches to chronic illness management? Patient Educ Couns. 2001;44:119–127. doi: 10.1016/S0738-3991(00)00186-5.
    1. Akl EA, Treweek S, Foy R, Francis J, Oxman AD. NorthStar, a support tool for the design and evaluation of quality improvement interventions in healthcare. Implement Sci. 2007;2:19–26. doi: 10.1186/1748-5908-2-19.
    1. Francis J, Eccles M, Johnston M, Whitty P, Grimshaw J, Kaner E, Smith L, Walker A. Explaining the effects of an intervention designed to promote evidence-based diabetes care: a theory-based process evaluation of a pragmatic cluster randomised controlled trial. Implement Sci. 2008;3:50. doi: 10.1186/1748-5908-3-50.
    1. Durlak J. Why programme implementation is so important. J Prev Interv Community. 1998;17:5–18. doi: 10.1300/J005v17n02_02.
    1. Lewin S, Glenton C, Oxman AD. Use of qualitative methods alongside randomised controlled trials of complex healthcare interventions: methodological study. Br Med J. 2009;339:b3496. doi: 10.1136/bmj.b3496.
    1. Hulscher M, Laurant M, Grol R. In: Quality Improvement Research; Understanding The Science of Change in Healthcare. Grol R, Baker R, Moss F, editor. London: BMJ Publishing Group; 2004. Process evaluation of quality improvement interventions; pp. 165–183.
    1. Cook T. Emergent principles for the design, implementation, and analysis of cluster-based experiments in social science. Ann Am Acad Pol Soc Sci. 2005;599:176–198. doi: 10.1177/0002716205275738.
    1. Bonell C, Oakley A, Hargreaves J, Strange V, Rees R. Assessment of generalisability in trials of health interventions: suggested framework and systematic review. Br Med J. 2006;333:346–349. doi: 10.1136/bmj.333.7563.346.
    1. Murtagh MJ, Thomson RG, May CR, Rapley T, Heaven BR, Graham RH, Kaner EF, Stobbart L, Eccles MP. Qualitative methods in a randomised controlled trial: the role of an integrated qualitative process evaluation in providing evidence to discontinue the intervention in one arm of a trial of a decision support tool. Qual Saf Health Care. 2007;16:224–229. doi: 10.1136/qshc.2006.018499.
    1. Campbell MK, Elbourne DR, Altman DG. CONSORT statement: extension to cluster randomised trials. Br Med J. 2004;328:702–708. doi: 10.1136/bmj.328.7441.702.
    1. Hoddinott P, Britten J, Pill R. Why do interventions work in some places and not others: a breastfeeding support group trial. Soc Sci Med. 2010;70:769–778. doi: 10.1016/j.socscimed.2009.10.067.
    1. Nazareth I, Freemantle N, Duggan C, Mason J, Haines A, Nazareth I, Freemantle N, Duggan C, Mason J, Haines A. Evaluation of a complex intervention for changing professional behaviour: the Evidence Based Out Reach (EBOR) Trial. J Health Serv Res Policy. 2002;7:230–238. doi: 10.1258/135581902320432769.
    1. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. Br Med J. 2008;337:a1655. doi: 10.1136/bmj.a1655.
    1. Salter C, Holland R, Harvey I, Henwood K. ‘I haven't even phoned my doctor yet.’ The advice giving role of the pharmacist during consultations for medication review with patients aged 80 or more: qualitative discourse analysis. Br Med J. 2007;334:1101. doi: 10.1136/bmj.39171.577106.55.
    1. Fretheim A, Havelsrud K, Oxman A. Rational Prescribing in Primary care (RaPP): process evaluation of an intervention to improve prescribing of antihypertensive and cholesterol-lowering drugs. Implement Sci. 2006;1:19. doi: 10.1186/1748-5908-1-19.
    1. Farrin A, Russell I, Torgerson DJ, Underwood M. Differential recruitment in a cluster randomised trial in primary care: the experience of the UK Back pain, Exercise, Active management and Manipulation (UK BEAM) feasibility study. Clin Trials. 2005;2:119–124. doi: 10.1191/1740774505cn073oa.
    1. May C, Finch T. Implementing, embedding, and integrating practices: an outline of normalisation process theory. Sociology. 2009;43:535.
    1. Christie D, Hudson L, Mathiot A, Cole T, Karlsen S, Kessel A, Kinra S, Morris S, Nazareth I, Sovio U, Wong I, Viner R. Assessing the efficacy of the healthy eating and lifestyle programme (HELP) compared with enhanced standard care of the obese adolescent in the community: study protocol for a randomized controlled trial. Trials. 2011;12:242. doi: 10.1186/1745-6215-12-242.
    1. Hardeman W, Sutton S, Griffin S, Johnston M, White AJ, Wareham NJ, Kinmonth AL. A causal modelling approach to the development of theory-based behaviour change programmes for trial evaluation. Health Educ Res. 2005;20:676–687. doi: 10.1093/her/cyh022.
    1. Mitchie S, Johnson M, Francis J, Hardeman W, Eccles M. From theory to intervention: mapping theoretically derived behavioural determinants to behaviour change techniques. Appl Psychol Int Rev. 2008;57:660–680. doi: 10.1111/j.1464-0597.2008.00341.x.
    1. Wells M, Williams B, Treweek S, Coyle J, Taylor J. Intervention description is not enough: evidence from an in-depth multiple case study on the untold role and impact of context in randomised controlled trials of seven complex interventions. Trials. 2012;13:95. doi: 10.1186/1745-6215-13-95.
    1. Grant A, Dreischulte T, Treweek S, Guthrie B. Study protocol of a mixed-methods evaluation of a cluster randomised trial to improve the safety of NSAID and antiplatelet prescribing: data-driven quality improvement in primary care. Trials. 2012;13:154. doi: 10.1186/1745-6215-13-154.
    1. Foy R, Francis J, Johnston M, Eccles M, Lecouturier J, Bamford C, Grimshaw J. The development of a theory-based intervention to promote appropriate disclosure of a diagnosis of dementia. BMC Health Serv Res. 2007;7:207. doi: 10.1186/1472-6963-7-207.
    1. 11 Questions to Help You Make Sense of a Trial. .

Source: PubMed

Sponsorer og samarbejdspartnere

Medicinske tilstande

Narkotikainterventioner

3
Abonner