Measuring determinants of implementation behavior: psychometric properties of a questionnaire based on the theoretical domains framework

Johanna M Huijg, Winifred A Gebhardt, Elise Dusseldorp, Marieke W Verheijden, Nicolette van der Zouwe, Barend J C Middelkoop, Mathilde R Crone, Johanna M Huijg, Winifred A Gebhardt, Elise Dusseldorp, Marieke W Verheijden, Nicolette van der Zouwe, Barend J C Middelkoop, Mathilde R Crone

Abstract

Background: To be able to design effective strategies to improve healthcare professionals' implementation behaviors, a valid and reliable questionnaire is needed to assess potential implementation determinants. The present study describes the development of the Determinants of Implementation Behavior Questionnaire (DIBQ) and investigates the reliability and validity of this Theoretical Domains Framework (TDF)-based questionnaire.

Methods: The DIBQ was developed to measure the potential behavioral determinants of the 12-domain version of the TDF (Michie et al., 2005). We identified existing questionnaires including items assessing constructs within TDF domains and developed new items where needed. Confirmatory factor analysis was used to examine whether the predefined structure of the TDF-based questionnaire was supported by the data. Cronbach's alpha was calculated to assess internal consistency reliability of the questionnaire, and domains' discriminant validity was investigated.

Results: We developed an initial questionnaire containing 100 items assessing 12 domains. Results obtained from confirmatory factor analysis and Cronbach's alpha resulted in the final questionnaire consisting of 93 items assessing 18 domains, explaining 63.3% of the variance, and internal consistency reliability values ranging from .68 to .93. Domains demonstrated good discriminant validity, although the domains 'Knowledge' and 'Skills' and the domains 'Skills' and 'Social/professional role and identity' were highly correlated.

Conclusions: We have developed a valid and reliable questionnaire that can be used to assess potential determinants of healthcare professional implementation behavior following the theoretical domains of the TDF. The DIBQ can be used by researchers and practitioners who are interested in identifying determinants of implementation behaviors in order to be able to develop effective strategies to improve healthcare professionals' implementation behaviors. Furthermore, the findings provide a novel validation of the TDF and indicate that the domain 'Environmental context and resources' might be divided into several environment-related domains.

References

    1. Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012;7:50. doi: 10.1186/1748-5908-7-50.
    1. Glasgow RE, Lichtenstein E, Marcus AC. Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. Am J Public Health. 2003;93:1261–1267. doi: 10.2105/AJPH.93.8.1261.
    1. Haines A, Kuruvilla S, Borchert M. Policy and practice bridging the implementation gap between knowledge and action for health. Bull World Health Organ. 2004;82:724–732.
    1. Glasgow R, Klesges L, Dzewaltowski DA, Bull SS, Estabrooks PA. The future of physical activity behavior change research: what is needed to improve translation of research into health promotion practice? Ann Behav Med. 2004;27:3–12. doi: 10.1207/s15324796abm2701_2.
    1. Eccles MP, Armstrong D, Baker R, Cleary K, Davies H, Davies S, Glasziou P, Ilott I, Kinmonth A-L, Leng G, Logan S, Marteau T, Michie S, Rogers H, Rycroft-Malone J, Sibbald B. An implementation research agenda. Implement Sci. 2009;4:18. doi: 10.1186/1748-5908-4-18.
    1. Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362:1225–1230. doi: 10.1016/S0140-6736(03)14546-1.
    1. Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual framework for implementation fidelity. Implement Sci. 2007;2:40. doi: 10.1186/1748-5908-2-40.
    1. Fleuren M, Wiefferink K, Paulussen T. Determinants of innovation within health care organizations: literature review and Delphi study. Int J Qual Health Care. 2004;16:107–123. doi: 10.1093/intqhc/mzh030.
    1. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82:581–629. doi: 10.1111/j.0887-378X.2004.00325.x.
    1. Grol R, Wensing M, Eccles M. Improving Patient Care: The Implementation of Change in Clinical Practice. Oxford: Elsevier; 2005.
    1. Rogers EM. Diffusion of Innovations. New York: The Free Press; 1983.
    1. Chaudoir SR, Dugan AG, Barr CHI. Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level. Implement Sci. 2013;8:22. doi: 10.1186/1748-5908-8-22.
    1. Huijg JM, Crone MR, Verheijden MW, van der Zouwe N, Middelkoop BJC, Gebhardt WA. Factors influencing the adoption, implementation, and continuation of physical activity interventions in primary health care: a Delphi study. BMC Fam Pract. 2013;14:142. doi: 10.1186/1471-2296-14-142.
    1. Eccles M, Grimshaw J, Walker A, Johnston M, Pitts N. Changing the behavior of healthcare professionals: the use of theory in promoting the uptake of research findings. J Clin Epidemiol. 2005;58:107–112. doi: 10.1016/j.jclinepi.2004.09.002.
    1. Foy R, Eccles M, Grimshaw J. Why does primary care need more implementation 5esearch? Fam Pract. 2001;18:353–355. doi: 10.1093/fampra/18.4.353.
    1. French SD, Green SE, O’Connor DA, McKenzie JE, Francis JJ, Michie S, Buchbinder R, Schattner P, Spike N, Grimshaw JM. Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the theoretical domains framework. Implement Sci. 2012;7:38. doi: 10.1186/1748-5908-7-38.
    1. Michie S, Johnston M, Francis J, Hardeman W, Eccles M. From theory to intervention: mapping theoretically derived behavioural determinants to behaviour change techniques. Appl Psychol. 2008;57:660–680. doi: 10.1111/j.1464-0597.2008.00341.x.
    1. Presseau J, Johnston M, Francis J, Hrisos S, Stamp E, Steen N, Hawthorne G, Grimshaw J, Elovainio M, Hunter M, Eccles M. Theory-based predictors of multiple clinician behaviors in the management of diabetes. J Behav Med. 2013. epub ahead of print.
    1. Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M, Baker R, Eccles MP. A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. 2013;8:35. doi: 10.1186/1748-5908-8-35.
    1. Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, Grilli R, Harvey E, Oxman A, O’Brien MA. Changing provider behavior: an overview of systematic reviews of interventions. Med Care. 2001;39:II2–II45.
    1. Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, Whitty P, Eccles MP, Matowe L, Shirran L, Wensing M, Dijkstra R, Donaldson C. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004;8:6.
    1. Davies P, Walker AE, Grimshaw JM. A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implement Sci. 2010;5:14. doi: 10.1186/1748-5908-5-14.
    1. The Improved Clinical Effectiveness through Behavioural Research Group (ICEBeRG) Designing theoretically-informed implementation interventions. Implement Sci. 2006;1:4.
    1. Francis JJ, Tinmouth A, Stanworth SJ, Grimshaw JM, Johnston M, Hyde C, Stockton C, Brehaut JC, Fergusson D, Eccles MP. Using theories of behaviour to understand transfusion prescribing in three clinical contexts in two countries: development work for an implementation trial. Implement Sci. 2009;4:70. doi: 10.1186/1748-5908-4-70.
    1. Grol RPTM, Bosch MC, Hulscher MEJL, Eccles MP, Wensing M. Planning and studying improvement in patient care: the use of theoretical perspectives. Milbank Q. 2007;85:93–138. doi: 10.1111/j.1468-0009.2007.00478.x.
    1. Rothman AJ. “Is there nothing more practical than a good theory?”: why innovations and advances in health behavior change will arise if interventions are used to test and refine theory. Int J Behav Nutr Phys. 2004;1:11. doi: 10.1186/1479-5868-1-11.
    1. Francis JJ, Stockton C, Eccles MP, Johnston M, Cuthbertson BH, Grimshaw JM, Hyde C, Tinmouth A, Stanworth SJ. Evidence-based selection of theories for designing behaviour change interventions: using methods based on theoretical construct domains to understand clinicians’ blood transfusion behaviour. Br J Health Psychol. 2009;14:625–646. doi: 10.1348/135910708X397025.
    1. Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005;14:26–33. doi: 10.1136/qshc.2004.011155.
    1. Cane J, O’Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7:37. doi: 10.1186/1748-5908-7-37.
    1. McSherry LA, Dombrowski SU, Francis JJ, Murphy J, Martin CM, O’Leary JJ, Sharp L. ATHENS Group. ‘It’s a can of worms’: understanding primary care practitioners' behaviours in relation to HPV using the theoretical domains framework. Implement Sci. 2012;7:73. doi: 10.1186/1748-5908-7-73.
    1. Patey AM, Islam R, Francis JJ, Bryson GL, Grimshaw JM. Anesthesiologists’ and surgeons’ perceptions about routine pre-operative testing in low-risk patients: application of the Theoretical Domains Framework (TDF) to identify factors that influence physicians’ decisions to order pre-operative tests. Implement Sci. 2012;7:52. doi: 10.1186/1748-5908-7-52.
    1. Islam R, Tinmouth AT, Francis JJ, Brehaut JC, Born J, Stockton C, Stanworth SJ, Eccles MP, Cuthbertson BH, Hyde C, Grimshaw JM. A cross-country comparison of intensive care physicians’ beliefs about their transfusion behaviour: a qualitative study using the theoretical domains framework. Implement Sci. 2012;7:93. doi: 10.1186/1748-5908-7-93.
    1. Curran JA, Brehaut J, Patey AM, Osmond M, Stiell I, Grimshaw JM. Understanding the Canadian adult CT head rule trial: use of the theoretical domains framework for process evaluation. Implement Sci. 2013;8:25. doi: 10.1186/1748-5908-8-25.
    1. Dyson J, Lawton R, Jackson C, Cheater F. Does the use of a theoretical approach tell us more about hand hygiene behaviour? The barriers and levers to hand hygiene. J Infect Prev. 2011;12:17–24. doi: 10.1177/1757177410384300.
    1. Mazza D, Chapman A, Michie S. Barriers to the implementation of preconception care guidelines as perceived by general practitioners: a qualitative study. BMC Health Serv Res. 2013;13:36. doi: 10.1186/1472-6963-13-36.
    1. Michie S, Pilling S, Garety P, Whitty P, Eccles M, Johnston M, Simmons J. Difficulties implementing a mental health guideline: an exploratory investigation using psychological theory. Implement Sci. 2007;2:8. doi: 10.1186/1748-5908-2-8.
    1. Amemori M, Michie S, Korhonen T, Murtomaa H, Kinnunen TH. Assessing implementation difficulties in tobacco use prevention and cessation counselling among dental providers. Implement Sci. 2011;6:50. doi: 10.1186/1748-5908-6-50.
    1. Beenstock J, Sniehotta FF, White M, Bell R, Milne EM, Araujo-Soares V. What helps and hinders midwives in engaging with pregnant women about stopping smoking? A cross-sectional survey of perceived implementation difficulties among midwives in the northeast of England. Implement Sci. 2012;7:36. doi: 10.1186/1748-5908-7-36.
    1. Taylor N, Parveen S, Robins V, Slater B, Lawton R. Development and initial validation of the influences on patient safety behaviours questionnaire. Implement Sci. 2013;8:81. doi: 10.1186/1748-5908-8-81.
    1. Huijg JM, Gebhardt WA, Crone MR, Dusseldorp E, Presseau J. Discriminant content validity of a Theoretical Domains Framework questionnaire for use in implementation research. Implement Sci. 2014;9:11. doi: 10.1186/1748-5908-9-11.
    1. Ajzen I. The theory of planned behavior. Organ Behav Hum Decis. 1991;50:179–211. doi: 10.1016/0749-5978(91)90020-T.
    1. Bandura A. Health promotion from the perspective of social cognitive theory. Psychol Health. 1998;13:623–649. doi: 10.1080/08870449808407422.
    1. Huijg JM, van der Zouwe N, Gebhardt WA, Crone MR, Verheijden MW, Middelkoop BJC. Introducing physical activity interventions in primary health care: a qualitative study of perceived facilitators and barriers [abstract] Int J Behav Med. 2012;19:S1–S341.
    1. van Veldhoven M, Meijman T, Broersen J, Fortuin R. Handleiding VBBA. Amsterdam: SKB Vragenlijst Services; 2002.
    1. Frese M. Social support as a moderator of the relationship between work stressors and psychological dysfunctioning: a longitudinal study with objective measures. J Occup Health Psychol. 1999;4:179–192.
    1. Gardner B, Abraham C, Lally P, de Bruijn G-J. Towards parsimony in habit measurement: testing the convergent and predictive validity of an automaticity subscale of the self-report habit index. Int J Behav Nutr Phys. 2012;9:102. doi: 10.1186/1479-5868-9-102.
    1. Sniehotta FF, Schwarzer R, Scholz U, Schüz B. Action planning and coping planning for long-term lifestyle change: theory and assessment. Eur J Soc Psychol. 2005;35:565–576. doi: 10.1002/ejsp.258.
    1. Scheier MF, Carver CS, Bridges MW. Distinguishing optimism from neuroticism (and trait anxiety, self-mastery, and self-esteem): a reevaluation of the life orientation test. J Pers Soc Psychol. 1994;67:1063–1078.
    1. Cialdini R, Kallgren C, Reno R. A focus theory of normative conduct: a theoretical refinement and reevaluation of the role of norms in human behavior. Adv Exp Soc Psychol. 1991;24:202–232.
    1. Qualtrics. Qualtrics software, version 45433. 2013. Provo, Utah, USA: Qualtrics.
    1. Corp IBM. IBM SPSS Statistics for Windows, Version 19.0. IBM Corp.: Armonk, NY; 2010.
    1. Stuive I, Kiers HAL, Timmerman ME, ten Berge JMF. The empirical verification of an assignment of items to subtests: the oblique multiple group method versus the confirmatory common factor method. Educ Psychol Meas. 2008;68:923–939. doi: 10.1177/0013164408315264.
    1. Holzinger K. A simple method of factor analysis. Psychometrika. 1944;9:257–262. doi: 10.1007/BF02288737.
    1. Stuive I, Kiers HAL, Timmerman ME. Comparison of methods for adjusting incorrect assignments of items to subtests: oblique multiple group method versus confirmatory common factor method. Educ Psychol Meas. 2009;69:948–965. doi: 10.1177/0013164409332226.
    1. Jöreskog K. A general approach to confirmatory maximum likelihood factor analysis. Psychometrika. 1969;34:183–202. doi: 10.1007/BF02289343.
    1. Jöreskog K. Testing a simple structure hypothesis in factor analysis. Psychometrika. 1966;31:165–178. doi: 10.1007/BF02289505.
    1. Hotelling H. Analysis of a complex of statitical variables into principal components. J Educ Psychol. 1933;24:417–441.
    1. Cronbach L. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16:297–333. doi: 10.1007/BF02310555.
    1. Campbell D, Fiske D. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol Bull. 1962;59:257–272.
    1. Anderson JC, Gerbing DW. Structural equation modeling in practice: a review and recommended two-step approach. Psychological Bulletin. 1988;103:411–423.
    1. Spearman C. The proof and measurement of association between two things. Am J Psychol. 1904;15:72–101. doi: 10.2307/1412159.
    1. Timmerman M, Stuive I. Multiple Group Method with Corrections for Selfcorrelation and Test Length. 2007.
    1. R Development Core Team R. R: a Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2012.
    1. Falissard B. Psy: Various Procedures Used in Psychometry. Vienna, Austria: R Foundation for Statistical Computing; 2012. R Package Version 1.1.
    1. Cohen J. Statistical Powe Analysis for the Behavioral Sciences. Hillsdale, New Jersey: Lawrence Erlbaum; 1988.
    1. Wännström I, Peterson U, Asberg M, Nygren A, Gustavsson JP. Psychometric properties of scales in the General Nordic Questionnaire for Psychological and Social Factors at Work (QPS): confirmatory factor analysis and prediction of certified long-term sickness absence. Scand J Psychol. 2009;50:231–244. doi: 10.1111/j.1467-9450.2008.00697.x.
    1. American Psychological Association (APA) APA Dictionary of Psychology. Washington, DC: American Psychological Association; 2007.
    1. Taylor N, Lawton R, Conner M. Development and initial validation of the determinants of physical activity questionnaire. Int J Behav Nutr Phys Act. 2013;10:74. doi: 10.1186/1479-5868-10-74.
    1. Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol. 1988;54:1063–1070.
    1. Crawford JR, Henry JD. The positive and negative affect schedule (PANAS): construct validity, measurement properties and normative data in a large non-clinical sample. Br J Clin Psychol. 2004;43:245–265. doi: 10.1348/0144665031752934.
    1. Wood W, Neal DT. The habitual consumer. J Consum Psychol. 2009;19:579–592. doi: 10.1016/j.jcps.2009.08.003.
    1. Deci E, Ryan R. Intrinsic Motivation and Self-Determination in Human Behavior. New York: Plenum Publishing Co.; 1985.
    1. Maslach C, Jackson SE. The measurement of experienced burnout. J Organ Behav. 1981;2:99–113. doi: 10.1002/job.4030020205.
    1. Dannapfel P, Peolsson A, Nilsen P. What supports physiotherapists’ use of research in clinical practice? A qualitative study in Sweden. Implement Sci. 2013;8:31. doi: 10.1186/1748-5908-8-31.
    1. Forsman H, Rudman A, Gustavsson P, Ehrenberg A, Wallin L. Nurses’ research utilization two years after graduation: a national survey of associated individual, organizational, and educational factors. Implement Sci. 2012;7:46. doi: 10.1186/1748-5908-7-46.
    1. Shirley D, Van der Ploeg HP, Bauman AE. Physical activity promotion in the physical therapy setting: perspectives from practitioners and students. Phys Ther. 2010;90:1311–1322. doi: 10.2522/ptj.20090383.
    1. van der Wees PJ, Zagers CA, de Die SE, Hendriks EJ, Nijhuis-van der Sanden MW, de Bie RA. Developing a questionnaire to identify perceived barriers for implementing the Dutch physical therapy COPD clinical practice guideline. BMC Health Serv Res. 2013;13:159. doi: 10.1186/1472-6963-13-159.
    1. Barrett EM, Darker CD, Hussey J. Promotion of physical activity in primary care: knowledge and practice of general practitioners and physiotherapists. J Public Health. 2013;21:63–69. doi: 10.1007/s10389-012-0512-0.
    1. Stevens J. Applied Multivariate Statistics for the Social Sciences. Hillsdale, New Jersey: Lawrence Erlbaum; 1992.

Source: PubMed

Sponsor e collaboratori

Condizioni mediche

Interventi farmacologici

3
Sottoscrivi