Measuring implementation: development of the implementation process assessment tool (IPAT)

M Hartveit, E Hovlid, M H A Nordin, J Øvretveit, G R Bond, E Biringer, J Assmus, G H Mariniusson, T Ruud, M Hartveit, E Hovlid, M H A Nordin, J Øvretveit, G R Bond, E Biringer, J Assmus, G H Mariniusson, T Ruud

Abstract

Background: Implementation science comprises a large set of theories suggesting interacting factors at different organisational levels. Development of literature syntheses and frameworks for implementation have contributed to comprehensive descriptions of implementation. However, corresponding instruments for measuring these comprehensive descriptions are currently lacking. The present study aimed to develop an instrument measuring care providers' perceptions of an implementation effort, and to explore the instrument's psychometric properties.

Methods: Based on existing implementation literature, a questionnaire was designed with items on individual and team factors and on stages of change in an implementation process. The instrument was tested in a Norwegian study on implementation of evidence based practices for psychosis. Item analysis, factor structure, and internal consistency at baseline were examined.

Results: The 27-item Implementation Process Assessment Tool (IPAT) revealed large variation between mean score of the items. The total scale scores were widely dispersed across respondents. Internal consistency for the total scale was high (Cronbach's alpha: .962), and all but one item contributed positively to the construct. The results indicated four underlying constructs: individual stages for behavioural change, individual activities and perceived support, collective readiness and support, and individual perceptions of the intervention.

Conclusions: The IPAT appears to be a feasible instrument for investigating the implementation process from the perspective of those making the change. It can enable examination of the relative importance of factors thought to be essential for implementation outcomes. It may also provide ongoing feedback for leaders tailoring support for teams to improve implementation. However, further research is needed to detect the instrument's properties later in the implementation process and in different contexts.

Trial registration: ClinicalTrials.gov code NCT03271242 (retrospective registered September 5, 2017).

Keywords: Implementation; Quality improvement; Readiness.

Conflict of interest statement

Two of the co-authors, Torleif Ruud and Eva Biringer, are members of the editorial board (Associate Editor) of this journal, but the decision and process of submission was completely transparent and the authors declare that they have no competing interests.

Figures

Fig. 1
Fig. 1
Illustration of the theoretical grounding for the Implementation Process Assessment Tool (IPAT). Factors of the major CFIR domains Outer setting, Inner setting, Intervention characteristics and Characteristics of individuals [4] are constantly being interpreted by care providers, implying readiness at two levels; the individual and the collective. The degree of readiness and the progression of stages of change are expected to interact
Fig. 2
Fig. 2
Implementation Process Assessment Tool (IPAT) scores at baseline. Mean score and confidence interval (CI) for each IPAT item at baseline across the four practices (N = 299)

References

    1. McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348(26):2635–2645. doi: 10.1056/NEJMsa022615.
    1. Drake RE, Essock SM. The science-to-service gap in real-world schizophrenia treatment: the 95% problem. Schizophr Bull. 2009;35(4):677–678. doi: 10.1093/schbul/sbp047.
    1. McGovern M, GJ HH, Drake R, Bond GR, Merrens M. Implementing Evidence-Based Practices in Behavioral Health. Minnesota: Dartmouth PRC Hazelden; 2013.
    1. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009. 10.1186/1748-5908-4-50.
    1. Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. Implementation research: a synthesis of the literature. Tampa: University of South Florida; 2005.
    1. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629. doi: 10.1111/j.0887-378X.2004.00325.x.
    1. Feldstein AC, Glasgow RE. A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. Jt Comm J Qual Patient Saf. 2008;34(4):228–243. doi: 10.1016/S1553-7250(08)34030-6.
    1. Kaplan Heather C, Provost Lloyd P, Froehle Craig M, Margolis Peter A. The Model for Understanding Success in Quality (MUSIQ): building a theory of context in healthcare quality improvement. BMJ Quality & Safety. 2011;21(1):13–20. doi: 10.1136/bmjqs-2011-000010.
    1. Lynch EA, Mudge A, Knowles S, Kitson AL, Hunter SC, Harvey G. “There is nothing so practical as a good theory”: a pragmatic guide for selecting theoretical approaches for implementation projects. BMC Health Serv Res. 2018;18:857. doi: 10.1186/s12913-018-3671-z.
    1. Øvretveit J. Quality health services. Middlesex: Brunel Institute of Organisation and Social Studies, Brunel University; 1990.
    1. Berwick DM. A primer on leading the improvement of systems. BMJ. 1996;312(7031):619. doi: 10.1136/bmj.312.7031.619.
    1. Damschroder LJ, Lowery JC. Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR) Implement Sci. 2013;8(1):51. doi: 10.1186/1748-5908-8-51.
    1. Stevens GW. Toward a process-based Appoach of conceptualizing change readiness. J Appl Behav Sci. 2013;49(3):333–360. doi: 10.1177/0021886313475479.
    1. Holt DT, Vardaman JM. Toward a comprehensive understanding of readiness for change: the case for an expanded conceptualization. J Chang Manag. 2013;13(1):9–18. doi: 10.1080/14697017.2013.768426.
    1. Rafferty AE, Jimmieson NL, Armenakis AA. Change readiness a multilevel review. J Manag. 2013;39(1):110–135.
    1. Weiner A. A theory of organizational readiness for change. Implement Sci. 2009;4:67. doi: 10.1186/1748-5908-4-67.
    1. Rogers EM. Diffusion of innovations. 4. New York: Free Press; 1995.
    1. Prochaska JO, DiClemente CC, Norcross JC. In search of how people change: applications to addictive behaviors. Am Psychol. 1992;47(9):1102. doi: 10.1037/0003-066X.47.9.1102.
    1. Grol R, Wensing M, Eccles M, Davis D. Improving patient care: the implementation of change in health care. 2. West Sussex: John Wiley & Sons; 2013.
    1. Grol R, Wensing M. What drives change? Barriers to and incentives for achieving evidence-based practice. Med J Aust. 2004;180(6):S57.
    1. Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the consolidated framework for implementation research. Implement Sci. 2015;11(1):72. doi: 10.1186/s13012-016-0437-z.
    1. Walker TJ, Rodriguez SA, Vernon SW, Savas LA, Frost EL, Fernandez ME. Validity and reliability of measures to assess constructs from the inner setting domain of the consolidated framework for implementation research in a pediatric clinic network implementing HPV programs. BMC Health Serv Res. 2019;19:205. doi: 10.1186/s12913-019-4021-5.
    1. Weiner Bryan J., Amick Halle, Lee Shoou-Yih Daniel. Review: Conceptualization and Measurement of Organizational Readiness for Change. Medical Care Research and Review. 2008;65(4):379–436. doi: 10.1177/1077558708317802.
    1. Gagnon MP, Attieh R, Ghandour EK, Légaré F, Ouimet M, Estabrooks CA, et al. A systematic review of instruments to assess organizational readiness for knowledge translation in health care. PLoS One. 2014;9(12):e114338. doi: 10.1371/journal.pone.0114338.
    1. Egeland KM, Ruud T, Ogden T, Lindstrøm JC, Heiervang KS. Psychometric properties of the Norwegian version of the evidence-based practice attitude scale (EBPAS): to measure implementation readiness. Health Res Policy Syst. 2016;14(1):47. doi: 10.1186/s12961-016-0114-3.
    1. Colla JB, Bracken AC, Kinney LM, Weeks WB. Measuring patient safety climate: a review of surveys. Qual Saf Health Care. 2005;14(5):364–366. doi: 10.1136/qshc.2005.014217.
    1. Aarons GA. Mental health provider attitudes toward adoption of evidence-based practice: the evidence-based practice attitude scale (EBPAS) Ment Health Serv Res. 2004;6(2):61–74. doi: 10.1023/B:MHSR.0000024351.12294.65.
    1. Chamberlain P, Brown CH, Saldana L. Observational measure of implementation progress in community based settings: the stages of implementation completion (SIC) Implement Sci. 2011;6:116. doi: 10.1186/1748-5908-6-116.
    1. Ruud T. Implementation of National Guidelines for Treatment of Psychoses. 2017.
    1. Prince M, Stewart R, Ford T, Hotopf M. Practical psychiatric epidemiology. New York: Oxford; 2003.
    1. DeVellis RF. Scale development theory and applications. 4. SAGE: Los Angeles; 2017.
    1. Yong AG, Pearce S. A beginner’s guide to factor analysis: focusing on exploratory factor analysis. Tutor Quant Methods Psychol. 2013;9(2):79. doi: 10.20982/tqmp.09.2.p079.
    1. Bowling A. Research methods in health. Investigating health and health services. 3rd ed. Mc Graw Hill: Berkshire; 2009.
    1. Aarons GA, Glisson C, Green PD, Hoagwood K, Kelleher KJ, Landsverk JA. The organizational social context of mental health services and clinician attitudes toward evidence-based practice: a United States national study. Implement Sci. 2012;7(1):56. doi: 10.1186/1748-5908-7-56.
    1. Shea CM, Jacobs SR, Esserman DA, Bruce K, Weiner BJ. Organizational readiness for implementing change: a psychometric assessment of a new measure. Implement Sci. 2014;9(7):1–15.
    1. Nilsen P, Bernhardsson S. Context matters in implementation science: a scoping review of determinant frameworks that describe contextual determinants for implementation outcomes. BMC Health Serv Res. 2019;19:189. doi: 10.1186/s12913-019-4015-3.
    1. Grol RPTM, Bosch MC, Hulscher MEJL, Eccles MP, Wensing M. Planning and studying improvement in patient care: the use of theoretical perspectives. Milbank Q. 2007;85(1):93–138. doi: 10.1111/j.1468-0009.2007.00478.x.
    1. Foy R, Ovretveit J, Shekelle PG, Pronovost PJ, Taylor SL, Dy S, et al. The role of theory in research to develop and evaluate the implementation of patient safety practices. BMJ Qual Saf. 2011;20(5):453. doi: 10.1136/bmjqs.2010.047993.
    1. Harvey G, Jas P, Walshe K, Skelcher C. Analysing organisational context: case studies on the contribution of absorptive capacity theory to understanding inter-organisational variation in performance improvement. BMJ Qual Saf. 2015;24(1):48. doi: 10.1136/bmjqs-2014-002928.

Source: PubMed

Sponsoren und Mitarbeiter

Krankheiten

Drogeninterventionen

3
Abonnieren