Psychometric Properties of the General Organizational Index (GOI): A Measure of Individualization and Quality Improvement to Complement Program Fidelity

Kristin Sverdvik Heiervang, Karina Myhren Egeland, Matthew Landers, Torleif Ruud, Inge Joa, Robert E Drake, Gary R Bond, Kristin Sverdvik Heiervang, Karina Myhren Egeland, Matthew Landers, Torleif Ruud, Inge Joa, Robert E Drake, Gary R Bond

Abstract

To assess the implementation of effective practices, mental health programs need standardized measures. The General Organizational Index (GOI), although widely used for this purpose, has received minimal psychometric research. For this study, we assessed psychometric properties of the GOI scale administered four times over 18 months during the implementation of a new program in 11 sites. The GOI scale demonstrated high levels of interrater reliability (.97), agreement between assessors on item ratings (86% overall), internal consistency (.77-.80 at three time points), sensitivity to change, and feasibility. We conclude that the GOI scale has acceptable psychometric properties, and its use may enhance implementation and research on evidence-based mental health practices.Trial registration: REK2015/2169. ClinicalTrials.gov Identifier: NCT03271242.

Keywords: GOI general organizational index; Individualization and quality improvement; Psychometric properties.

Conflict of interest statement

The authors declare that they have no conflict of interest.

Figures

Fig. 1
Fig. 1
Mean score of the GOI over 18 months
Fig. 2
Fig. 2
Mean score of Individualization over 18 months
Fig. 3
Fig. 3
Mean score of Quality Improvement over 18 months

References

    1. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration & Policy in Mental Health. 2011;38(1):4–23. doi: 10.1007/s10488-010-0327-7.
    1. Becker DR, Baker SR, Carlson L, Flint L, Howell R, Lindsay S, et al. Critical strategies for implementing supported employment. Journal of Vocational Rehabilitation. 2007;27:13–20.
    1. Bond GR, Drake RE. Administration and Policy in Mental Health. 2019 doi: 10.1007/s10488-019-00991-6.
    1. Bond GR, Evans L, Salyers MP, Williams J, Kim HK. Measurement of fidelity in psychiatric rehabilitation. Mental Health Services Research. 2000;2(2):75–87. doi: 10.1023/A:1010153020697.
    1. Bond GR, Drake RE, McHugo GJ, Rapp CA, Whitley R. Strategies for improving fidelity in the National Evidence-Based Practices Project. Research on Social Work Practice. 2009;19(5):569–581. doi: 10.1177/1049731509335531.
    1. Bond GR, Drake RE, Rapp CA, McHugo GJ, Xie H. Individualization and quality improvement: Two new scales to complement measurement of program fidelity. Administration & Policy in Mental Health & Mental Health Services Research. 2009;36(5):349–357. doi: 10.1007/s10488-009-0226-y.
    1. Bond GR, Becker DR, Drake RE. Measurement of fidelity of implementation of evidence-based practices: Case example of the IPS Fidelity Scale. Clinical Psychology: Science & Practice. 2011;18(2):126–141. doi: 10.1111/j.1468-2850.2011.01244.x.
    1. Bond GR, Drake RE, McHugo GJ, Peterson AE, Jones AM, Williams J. Long-term sustainability of evidence-based practices in community mental health agencies. Administration & Policy in Mental Health & Mental Health Services Research. 2014;41(2):228–236. doi: 10.1007/s10488-012-0461-5.
    1. Egeland, K. M. (2018). The role of practitioners in the implementation of evidence-based practices in mental health services: Attitudes, participation, and experiences (Doctoral thesis, University of Oslo, Oslo, Norway). Retrieved from .
    1. Egeland KM, Ruud T, Ogden T, Färdig R, Lindstrøm C, Heiervang KS. How to implement illness management and recovery (IMR) in mental health service settings: Evaluation of the implementation strategy. International Journal of Mental Health Systems. 2017;11:13. doi: 10.1186/s13033-017-0120-z.
    1. Egeland KM, Hauge M, Ruud T, Ogden T, Heiervang KS. Significance of leaders for sustained use of evidence-based practices: A qualitative focus-group study with mental health practitioners. Community Mental Health Journal. 2019;55:1344–1353. doi: 10.1007/s10597-019-00430-8.
    1. Fixsen, D., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: Florida University of South Florida, Louis de la Parte Florida Mental Health Institute, National Implementation Research Network.
    1. Kilbourne AM, Beck K, Spaeth-Rublee B, Ramanuj P, O'Brien RW, Tomoyasu N, Pincus HA. Measuring and improving the quality of mental health care: a global perspective. World Psychiatry. 2018;17(1):30–38. doi: 10.1002/wps.20482.
    1. Lakens D. Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t tests and ANOVAs. Front Psychol. 2013;4:863. doi: 10.3389/fpsyg.2013.00863.
    1. Martinez RG, Lewis CC, Weiner BJ. Instrumentation issues in implementation science. Implementation Science. 2014;9:118. doi: 10.1186/s13012-014-0118-8.
    1. Marty D, Rapp C, McHugo G, et al. Administration and Policy in Mental Health. 2008;35:204. doi: 10.1007/s10488-007-0157-4.
    1. McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychological Methods. 1996;1(1):30–46. doi: 10.1037/1082-989X.1.1.30.
    1. McGuire AB, Salyers MP, White DA, Gilbride DJ, White LM, Kean J, Kukla M. Factors affecting implementation of an evidence-based practice in the Veterans Health Administration: Illness management and recovery. Psychiatric Rehabilitation Journal. 2015;38(4):300–305. doi: 10.1037/prj0000116.
    1. McHugo GJ, Drake RE, Whitley R, Bond G, Campbell K, Rapp CA, et al. Fidelity outcomes in the national implementing evidence-based practices project. Psychiatric Services. 2007;58(10):1279–1284. doi: 10.1176/appi.ps.58.10.1279.
    1. Monroe-DeVita M, Morse G, Bond GR. Program fidelity and beyond: Multiple strategies and criteria for ensuring quality of assertive community treatment. Psychiatric Services. 2012;63(8):743–750. doi: 10.1176/appi.ps.201100015.
    1. Moullin JC, Dickson KS, Stadnick NA, et al. Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Implementation Science. 2019;14:1. doi: 10.1186/s13012-018-0842-6.
    1. Mueser KT, Meyer PS, Penn DL, Clancy R, Clancy DM, Salyers MP. The Illness Management and Recovery Program: Rationale, development, and preliminary findings. Schizophrenia Bulletin. 2006;32(Suppl 1):S32–S43. doi: 10.1093/schbul/sbl022.
    1. Rapp CA, Etzel-Wise D, Marty D, Coffman M, Carlson L, Asher D, et al. Evidence-based practice implementation strategies: Results of a qualitative study. Community Mental Health Journal. 2008;44:213–224. doi: 10.1007/s10597-007-9109-4.
    1. Rapp CA, Goscha RJ, Carlson LS. Evidence-based practice implementation in Kansas. Community Mental Health Journal. 2010;46:461–465. doi: 10.1007/s10597-010-9311-7.
    1. Ross S, Naylor C. Quality improvement in mental health (Report) London: The King’s fund; 2017.
    1. Rychener M, Salyers M, Labriola S, Little N. Thresholds’ wellness management and recovery implementation. American Journal of Psychiatric Rehabilitation. 2009;12(2):172–184. doi: 10.1080/15487760902813186.
    1. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: What it is and what it isn’t. British Medical Journal. 1996;312(7023):71–72. doi: 10.1136/bmj.312.7023.71.
    1. Salyers MP, Bond GR, Teague GB, Cox JF, Smith ME, Hicks ML, Koop JI. Is it ACT yet? Real-world examples of evaluating the degree of implementation for assertive community treatment. Journal of Behavioral Health Services & Research. 2003;30(3):304–320. doi: 10.1007/bf02287319.
    1. Salyers MP, Godfrey JL, McGuire AB, Gearhart T, Rollins AL, Boyle C. Implementing the illness management and recovery program for consumers with severe mental illness. Psychiatric Services. 2009;60(4):483–490. doi: 10.1176/ps.2009.60.4.483.
    1. Streiner DL, Norman GR, Cairney J. Health measurement scales. A practical guide to their development and use. 5th Edition. Oxford: Oxford University Press; 2015.
    1. Taylor MS. The effects of feedback on the behavior of organizational personnel. Administration in Social Work. 1987;11:191–203. doi: 10.1300/J147v11n03_18.
    1. Torrey WC, Finnerty M, Evans A, Wyzik P. Strategies for leading the implementation of evidence-based practices. Psychiatric Clinics of North America. 2003;26(4):883–897. doi: 10.1016/S0193-953X(03)00067-4.

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe