Improving the normalization of complex interventions: part 1 - development of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT)

Tim Rapley, Melissa Girling, Frances S Mair, Elizabeth Murray, Shaun Treweek, Elaine McColl, Ian Nicholas Steen, Carl R May, Tracy L Finch, Tim Rapley, Melissa Girling, Frances S Mair, Elizabeth Murray, Shaun Treweek, Elaine McColl, Ian Nicholas Steen, Carl R May, Tracy L Finch

Abstract

Background: Understanding and measuring implementation processes is a key challenge for implementation researchers. This study draws on Normalization Process Theory (NPT) to develop an instrument that can be applied to assess, monitor or measure factors likely to affect normalization from the perspective of implementation participants.

Methods: An iterative process of instrument development was undertaken using the following methods: theoretical elaboration, item generation and item reduction (team workshops); item appraisal (QAS-99); cognitive testing with complex intervention teams; theory re-validation with NPT experts; and pilot testing of instrument.

Results: We initially generated 112 potential questionnaire items; these were then reduced to 47 through team workshops and item appraisal. No concerns about item wording and construction were raised through the item appraisal process. We undertook three rounds of cognitive interviews with professionals (n = 30) involved in the development, evaluation, delivery or reception of complex interventions. We identified minor issues around wording of some items; universal issues around how to engage with people at different time points in an intervention; and conceptual issues around the types of people for whom the instrument should be designed. We managed these by adding extra items (n = 6) and including a new set of option responses: 'not relevant at this stage', 'not relevant to my role' and 'not relevant to this intervention' and decided to design an instrument explicitly for those people either delivering or receiving an intervention. This version of the instrument had 53 items. Twenty-three people with a good working knowledge of NPT reviewed the items for theoretical drift. Items that displayed a poor alignment with NPT sub-constructs were removed (n = 8) and others revised or combined (n = 6). The final instrument, with 43 items, was successfully piloted with five people, with a 100% completion rate of items.

Conclusion: The process of moving through cycles of theoretical translation, item generation, cognitive testing, and theoretical (re)validation was essential for maintaining a balance between the theoretical integrity of the NPT concepts and the ease with which intended respondents could answer the questions. The final instrument could be easily understood and completed, while retaining theoretical validity. NoMAD represents a measure that can be used to understand implementation participants' experiences. It is intended as a measure that can be used alongside instruments that measure other dimensions of implementation activity, such as implementation fidelity, adoption, and readiness.

Keywords: Complex interventions; Implementation process; Instrument development; NPT; NoMAD; Normalization process theory; Questionnaire.

Conflict of interest statement

Ethics approval and consent to participate

The work was approved by the Newcastle University Ethics Committee (Reference number 00555/2012; approval granted 1/09/2012). Written consent obtained was obtained in all three rounds.

Consent for publication

Not applicable.

Competing interests

CM and TR are members of the Editorial Board of Implementation Science. All other authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures

Fig. 1
Fig. 1
Overview of questionnaire development process

References

    1. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Admin Pol Ment Health. 2011;38(2):65–76. doi: 10.1007/s10488-010-0319-7.
    1. Woolf SH. The meaning of translational research and why it matters. JAMA. 2008;299(2):211–213. doi: 10.1001/jama.2007.26.
    1. Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10(1):53. doi: 10.1186/s13012-015-0242-0.
    1. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629. doi: 10.1111/j.0887-378X.2004.00325.x.
    1. Grol R, Bosch M, Hulscher M, Eccles M, Wensing M. Planning and studying improvement in patient care: the use of theoretical perspectives. Milbank Q. 2007;85(1):93–138. doi: 10.1111/j.1468-0009.2007.00478.x.
    1. Damschroder L, Aron D, Keith R, Kirsh S, Alexander J, Lowery J. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50. doi: 10.1186/1748-5908-4-50.
    1. May C. Towards a general theory of implementation. Implement Sci. 2013;8(1):18. doi: 10.1186/1748-5908-8-18.
    1. Cook JM, O’Donnell C, Dinnen S, Coyne JC, Ruzek JI, Schnurr PP. Measurement of a model of implementation for health care: toward a testable theory. Implement Sci. 2012;7(1):59. doi: 10.1186/1748-5908-7-59.
    1. Weiner BJ, Amick H, Lee SY. Conceptualization and measurement of organizational readiness for change: a review of the literature in health services research and other fields. Med Care Res Rev. 2008;65:379–436. doi: 10.1177/1077558708317802.
    1. Squires JE, Estabrooks CA, O'Rourke HM, Gustavsson P, Newburn-Cook CV, Wallin L. A systematic review of the psychometric properties of self-report research utilization measures used in healthcare. Implement Sci. 2011;6(1):83. doi: 10.1186/1748-5908-6-83.
    1. Gagnon M-P, Attieh R, Ghandour EK, Légaré F, Ouimet M, Estabrooks CA, Grimshaw J. A systematic review of instruments to assess organizational readiness for knowledge translation in health care. PLoS One. 2014;9(12):e114338. doi: 10.1371/journal.pone.0114338.
    1. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655. doi: 10.1136/bmj.a1655.
    1. Linton JD. Implementation research: state of the art and future directions. Technovation. 2002;22(2):65–79. doi: 10.1016/S0166-4972(01)00075-X.
    1. Martinez RG, Lewis CC, Weiner BJ. Instrumentation issues in implementation science. Implement Sci. 2014;9(1):118. doi: 10.1186/s13012-014-0118-8.
    1. Lewis CC, Fischer S, Weiner BJ, Stanick C, Kim M, Martinez RG. Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. Implement Sci. 2015;10(1):155. doi: 10.1186/s13012-015-0342-x.
    1. Clinton-McHarg T, Yoong SL, Tzelepis F, Regan T, Fielding A, Skelton E, Kingsland M, Ooi JY, Wolfenden L. Psychometric properties of implementation measures for public health and community settings and mapping of constructs against the consolidated framework for implementation research: a systematic review. Implement Sci. 2016;11(1):148. doi: 10.1186/s13012-016-0512-5.
    1. Shea CM, Jacobs SR, Esserman DA, Bruce K, Weiner BJ. Organizational readiness for implementing change: a psychometric assessment of a new measure. Implement Sci. 2014;9(1):7. doi: 10.1186/1748-5908-9-7.
    1. Glasgow RE, Riley WT. Pragmatic measures: what they are and why we need them. Am J Prev Med. 2013;45(2):237–243. doi: 10.1016/j.amepre.2013.03.010.
    1. May C, Finch T. Implementing, embedding, and integrating practices: an outline of normalization process theory. Sociology. 2009;43(3):535–554. doi: 10.1177/0038038509103208.
    1. McEvoy R, Ballini L, Maltoni S, O'Donnell C, Mair F, MacFarlane A. A qualitative systematic review of studies using the normalization process theory to research implementation processes. Implement Sci. 2014;9(1):2. doi: 10.1186/1748-5908-9-2.
    1. Atkins S, Lewin S, Ringsberg KC, Thorson A. Provider experiences of the implementation of a new tuberculosis treatment programme: a qualitative study using the normalisation process model. BMC Health Serv Res. 2011;11(1):275. doi: 10.1186/1472-6963-11-275.
    1. Blakeman T, Protheroe J, Chew-Graham C. Understanding the management of early-stage chronic kidney disease in primary care: a qualitative study. Br J Gen Pract. 2012;62(597):e233–e242. doi: 10.3399/bjgp12X636056.
    1. Elwyn G, Légaré F, Weijden T, Edwards A, May C. Arduous implementation: does the normalisation process model explain why it's so difficult to embed decision support technologies for patients in routine clinical practice. Implement Sci. 2008;3(1):57. doi: 10.1186/1748-5908-3-57.
    1. Forster DA, Newton M, McLachlan HL, Willis K. Exploring implementation and sustainability of models of care: can theory help? BMC Public Health. 2011;11(Suppl 5):S8. doi: 10.1186/1471-2458-11-S5-S8.
    1. Foy R, Ovretveit J, Shekelle PG, Pronovost PJ, Taylor SL, Dy S, Hempel S, McDonald KM, Rubenstein LV, Wachter RM. The role of theory in research to develop and evaluate the implementation of patient safety practices. BMJ Qual Saf. 2011;20(5):453–459. doi: 10.1136/bmjqs.2010.047993.
    1. Franx G, Oud M, de Lange J, Wensing M, Grol R. Implementing a stepped-care approach in primary care: results of a qualitative study. Implement Sci. 2012;7(1):8. doi: 10.1186/1748-5908-7-8.
    1. Gallacher K, May CR, Montori VM, Mair FS. Understanding Patients’ experiences of treatment burden in chronic heart failure using normalization process theory. Ann Fam Med. 2011;9(3):235–243. doi: 10.1370/afm.1249.
    1. Gask L, Bower P, Lovell K, Escott D, Archer J, Gilbody S, Lankshear A, Simpson A, Richards D. What work has to be done to implement collaborative care for depression? Process evaluation of a trial utilizing the normalization process model. Implement Sci. 2010;5(15):5908–5905.
    1. Gask L, Rogers A, Campbell S, Sheaff R. Beyond the limits of clinical governance? The case of mental health in English primary care. BMC Health Serv Res. 2008;8(1):63. doi: 10.1186/1472-6963-8-63.
    1. Gunn J, Palmer V, Dowrick C, Herrman H, Griffiths F, Kokanovic R, Blashki G, Hegarty K, Johnson C, Potiriadis M, et al. Embedding effective depression care: using theory for primary care organisational and systems change. Implement Sci. 2010;5(62):1–15.
    1. James DM. The applicability of normalisation process theory to speech and language therapy: a review of qualitative research on a speech and language intervention. Implement Sci. 2011;6(1):95. doi: 10.1186/1748-5908-6-95.
    1. Kennedy A, Chew-Graham C, Blakeman T, Bowen A, Gardner C, Protheroe J, Rogers A, Gask L. Delivering the WISE (whole systems informing self-management engagement) training package in primary care: learning from formative evaluation. Implement Sci. 2010;5(1):7. doi: 10.1186/1748-5908-5-7.
    1. MacFarlane A, O’Reilly-de Brun M. Using a theory-driven conceptual framework in qualitative Health Research. Qual Health Res. 2011;22(5):607–618. doi: 10.1177/1049732311431898.
    1. Mair FS, Hiscock J, Beaton SC. Understanding factors that inhibit or promote the utilization of telecare in chronic lung disease. Chronic Illness. 2008;4(2):110–117. doi: 10.1177/1742395308092482.
    1. Murray E, Burns J, May C, Finch T, O'Donnell C, Wallace P, Mair F. Why is it difficult to implement e-health initiatives? A qualitative study. Implement Sci. 2011;6(1):6. doi: 10.1186/1748-5908-6-6.
    1. Rogers A, Vassilev I, Sanders C, Kirk S, Chew-Graham C, Kennedy A, Protheroe J, Bower P, Blickem C, Reeves D, et al. Social networks, work and network-based resources for the management of long-term conditions: a framework and study protocol for developing self-care support. Implement Sci. 2011;6(1):56. doi: 10.1186/1748-5908-6-56.
    1. Sanders T, Foster NE, Ong B. Perceptions of general practitioners towards the use of a new system for treating back pain: a qualitative interview study. BMC Med. 2011;9(1):49. doi: 10.1186/1741-7015-9-49.
    1. Sooklal R, Papadopoulos T, Ojiako U. Information systems development: a normalisation process theory perspective. Ind Manag Data Syst. 2011;111(8):1270–1286. doi: 10.1108/02635571111170794.
    1. Wilkes S, Rubin G. Process evaluation of infertility management in primary care: has open access HSG been normalized? Prim Health Care Res Dev. 2009;10:290–298. doi: 10.1017/S1463423609990168.
    1. Finch T, Mair F, O'Donnell C, Murray E, May C. From theory to ‘measurement’ in complex interventions: methodological lessons from the development of an e-health normalisation instrument. BMC Med Res Methodol. 2012;12(1):69. doi: 10.1186/1471-2288-12-69.
    1. Mair F, May C, Murray E, Finch T, Anderson G, O’Donnell C, Wallace P, Sullivan F. Understanding the implementation and integration of E-health services. London: National co-ordinating Centre for the National Institute for Health Research Service delivery and organisation Programme (NCCSDO); 2009.
    1. Murray E, May C, Mair F. Development and formative evaluation of the e-health implementation toolkit (e-HIT) BMC Med Inform Decis Mak. 2010;10:61. doi: 10.1186/1472-6947-10-61.
    1. May C, Finch T, Ballini L, MacFarlane A, Mair F, Murray E, Treweek S, Rapley T. Evaluating complex interventions and health technologies using normalization process theory: development of a simplified approach and web-enabled toolkit. BMC Health Serv Res. 2011;11(1):245. doi: 10.1186/1472-6963-11-245.
    1. Finch T, Rapley T, Girling M, Mair F, Murray E, Treweek S, McColl E, Steen I, May C. Improving the normalization of complex interventions: measure development based on normalization process theory (NoMAD): study protocol. Implement Sci. 2013;8(1):43. doi: 10.1186/1748-5908-8-43.
    1. Finch TL, Girling M, May CR, Mair FS, Murray E, Treweek S, McColl E, Steen IN, Cook C, Vernazza C et al: Improving the normalization of complex interventions: part 2 - validation of the NoMAD survey tool for assessing implementation work based on normalization process theory (NPT) In: BMC Medical Research Methodology; 2017. 10.1186/s12874-018-0591-x.
    1. May C, Finch T, Mair F, Ballini L, Dowrick C, Eccles M, Gask L, MacFarlane A, Murray E, Rapley T, et al. Understanding the implementation of complex interventions in health care: the normalization process model. BMC Health Serv Res. 2007;7:148. doi: 10.1186/1472-6963-7-148.
    1. May CR. Building an interdisciplinary theory of implementation, embedding and integration: the development of normalisation process theory. Implement Sci. 2009;4:29. doi: 10.1186/1748-5908-4-29.
    1. Lessler JT, Forsyth BH. A coding system for appraising questionnaires. In: Schwarz N, Sudman S, editors. Answering questions: methodology for determining cognitive and communicative processes in survey research. San Francisco: Jossey-Bass; 1996. pp. 389–402.
    1. Graesser A, Wiemer-hastings K, Kreuz R. QUAID: a questionnaire evaluation aid for survey methodologists. Behav Res Methods. 2000;32(2):254–262. doi: 10.3758/BF03207792.
    1. Willis GB. Cognitive interviewing: a tool for improving questionnaire design. California: Sage Publications Inc.; 2005.
    1. Knafl K, Deatrick J, Gallo A. The analysis and interpretation of cognitive interviews for instrument development. Res Nurs Health. 2007;30(2):224–234. doi: 10.1002/nur.20195.
    1. Stamatakis KA, McQueen A, Filler C, Boland E, Dreisinger M, Brownson RC, Luke DA. Measurement properties of a novel survey to assess stages of organizational readiness for evidence-based interventions in community chronic disease prevention settings. Implement Sci. 2012;7(1):65. doi: 10.1186/1748-5908-7-65.
    1. Aarons GA, Ehrhart MG, Farahnak LR. The implementation leadership scale (ILS): development of a brief measure of unit level implementation leadership. Implement Sci. 2014;9(1):45. doi: 10.1186/1748-5908-9-45.
    1. Dyson J, Lawton R, Jackson C, Cheater F. Development of a theory-based instrument to identify barriers and levers to best hand hygiene practice among healthcare practitioners. Implement Sci. 2013;8(1):111. doi: 10.1186/1748-5908-8-111.
    1. Jacobs SR, Weiner BJ, Bunger AC. Context matters: measuring implementation climate among individuals and groups. Implement Sci. 2014;9(1):46. doi: 10.1186/1748-5908-9-46.
    1. Chaudoir SR, Dugan AG, Barr CH. Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures. Implement Sci. 2013;8:22. doi: 10.1186/1748-5908-8-22.

Source: PubMed

Sponsoren und Mitarbeiter

Krankheiten

Drogeninterventionen

3
Abonnieren