A structured mixed method process evaluation of a randomized controlled trial of Individual Placement and Support (IPS)

Tonje Fyhn, Kari Ludvigsen, Silje E Reme, Frederieke Schaafsma, Tonje Fyhn, Kari Ludvigsen, Silje E Reme, Frederieke Schaafsma

Abstract

Background: Individual Placement and Support (IPS) is an evidence-based work rehabilitation program helping people with moderate to severe mental illness to obtain ordinary employment. Although IPS has proven superior to other work rehabilitation programs, in many studies, the majority of the participants remain unemployed. Structured process evaluations of IPS that use mixed methods are scarce, although they could identify implementation aspects that may enhance its effect. The aim of the current study is to assess reach, fidelity, and identify barriers and facilitators to implement IPS.

Methods: The process evaluation was conducted alongside a randomized controlled trial including six IPS centers, comparing IPS with treatment as usual in a population of patients in treatment for moderate to severe mental illness. Mixed methods were used in the process evaluation, including focus group interviews with service providers, individual interviews and survey data from participants, and fidelity reviews using the validated IPS Fidelity Scale.

Results: The intervention reached the intended target group. All centers reached fair to good fidelity according to the IPS Fidelity Scale within the project period (range 97-109, SD 8.1) (see Table 5). Certain fidelity items indicated implementation issues related to employer contact, community-based services, and integration with health services. Survey data showed that less than half of the participants regarded their illness as a barrier for participating in IPS and that freedom of disclosure was important. Participant interviews gave further insight into the role of the IPS specialist, emphasizing their availability and consistent job focus.

Conclusions: Indications of implementation challenges across centers during the first year suggest special attention should be given to these aspects in an early phase to ensure higher fidelity from the start and thus enhance the effectiveness of IPS. The IPS specialist played an important role for participants and was described as positive, pushing in a positive way, and encouraging. More knowledge on the characteristics of successful IPS specialists could further enhance the effectiveness of the intervention.

Trial registration: The study was registered on clinicaltrials.gov prior to the inclusion period (reg.no: NCT01964092, registered 17/07/2013).

Supplementary information: Supplementary information accompanies this paper at 10.1186/s43058-020-00083-9.

Keywords: IPS; Mental health services; Process evaluation; Psychiatric disorders; Recovery.

Conflict of interest statement

Competing interestsThe authors declare that they have no competing interests.

© The Author(s) 2020.

References

    1. Luciano A, Meara E. Employment status of people with mental illness: national survey data from 2009 and 2010. Psychiatr Serv. 2014;65(10):1201–1209. doi: 10.1176/appi.ps.201300335.
    1. WHO . Mental health action plan 2013–2020. Geneva: World Health Organization; 2013.
    1. Bond G. Supported employment: evidence for an evidence-based practice. Psychiatr Rehabil J. 2004;27(4):345. doi: 10.2975/27.2004.345.359.
    1. Drake RE. A brief history of the Individual Placement and Support model. Psychiatr Rehabil J. 1998;22(1):3. doi: 10.1037/h0095273.
    1. Becker DR, Drake RE. A working life for people with severe mental illness. New York: Oxford University Press; 2003.
    1. Becker DR, Swanson SJ, Reese SL, Bond GR, McLeman BM. Supported employment fidelity review manual. 2015.
    1. Bond G, Becker D, Drake R. Measurement of fidelity of implementation of evidence-based practices: case example of the IPS Fidelity Scale. Clin Psychol Sci Pract. 2011;18(2):126–141. doi: 10.1111/j.1468-2850.2011.01244.x.
    1. Catty J, Lissouba P, White S, Becker T, Drake RE, Fioritti A, et al. Predictors of employment for people with severe mental illness: results of an international six-centre randomised controlled trial. Br J Psychiatry. 2008;192(3):224–231. doi: 10.1192/bjp.bp.107.041475.
    1. Bond G, Peterson A, Becker D, Drake R. Validation of the revised individual placement and support fidelity scale (IPS-25) Psychiatr Serv. 2012;63(8):758–763. doi: 10.1176/appi.ps.201100476.
    1. de Winter L, Couwenbergh C, van Weeghel J, Bergmans C, Bond GR. Fidelity and IPS: does quality of implementation predict vocational outcomes over time for organizations treating persons with severe mental illness in the Netherlands? Soc Psychiatry Psychiatr Epidemiol. 2020. 10.1007/s00127-020-01890-0.
    1. Brinchmann B, Widding-Havneraas T, Modini M, Rinaldi M, Moe CF, McDaid D, et al. A meta-regression of the impact of policy on the efficacy of individual placement and support. Acta Psychiatr Scand. 2020; 141(3):206–20.
    1. Bond G, Kim S, Becker D, Swanson S, Drake R, Krzos I, et al. A controlled trial of supported employment for people with severe mental illness and justice involvement. Psychiatr Serv. 2015;66(10):1027–1034. doi: 10.1176/appi.ps.201400510.
    1. Heslin M, Howard L, Leese M, McCrone P, Rice C, Jarrett M, et al. Randomized controlled trial of supported employment in England: 2 year follow-up of the Supported Work and Needs (SWAN) study. World Psychiatry. 2011;10(2):132–137. doi: 10.1002/j.2051-5545.2011.tb00035.x.
    1. Oshima I, Sono T, Bond G, Nishio M, Ito J. A randomized controlled trial of individual placement and support in Japan. Psychiatr Rehabil J. 2014;37(2):137. doi: 10.1037/prj0000085.
    1. Reme S, Monstad K, Fyhn T, Sveinsdottir V, Løvvik C, Lie S, et al. A randomized controlled multicenter trial of individual placement and support for patients with moderate-to-severe mental illness. Scand J Work Environ Health. 2018;45(1):33–41. doi: 10.5271/sjweh.3753.
    1. Suijkerbuijk Y, Schaafsma F, van Mechelen J, Ojajärvi A, Corbière M, Anema J. Interventions for obtaining and maintaining employment in adults with severe mental illness, a network meta-analysis: Cochrane Database Syst Rev; 2017.
    1. Viering S, Jäger M, Bärtsch B, Nordt C, Rössler W, Warnke I, et al. Supported Employment for the reintegration of disability pensioners with mental illnesses: a randomized controlled trial. Front Public Health. 2015;3:237.
    1. Glasgow R, Green L, Klesges L, Abrams D, Fisher E, Goldstein M, et al. External validity: we need to do more. Ann Behav Med. 2006;31(2):105–108. doi: 10.1207/s15324796abm3102_1.
    1. Linnan L, Steckler A. Process evaluation for public health interventions and research. San Francisco: Jossey-Bass; 2002.
    1. Oakley A, Strange V, Bonell C, Allen E, Stephenson J. Process evaluation in randomised controlled trials of complex interventions. BMJ. 2006;332(7538):413–416. doi: 10.1136/bmj.332.7538.413.
    1. Bonfils I, Hansen H, Dalum H, Eplov L. Implementation of the individual placement and support approach–facilitators and barriers. Scand J Disabil Res. 2017;19(4):318–333. doi: 10.1080/15017419.2016.1222306.
    1. Linnemørken LTB, Sveinsdottir V, Knutzen T, Rødevand L, Hernæs KH, Reme SE. Protocol for the Individual Placement and Support (IPS) in Pain Trial: a randomized controlled trial investigating the effectiveness of IPS for patients with chronic pain. BMC Musculoskelet Disord. 2018;19(1):47. doi: 10.1186/s12891-018-1962-5.
    1. Sveinsdottir V, Lie SA, Bond GR, Eriksen HR, Tveito TH, Grasdal AL, et al. Individual placement and support for young adults at risk of early work disability (the SEED trial). A randomized controlled trial. Scand J Work Environ Health. 2019;45(1):33–41.
    1. Bach-Mortensen AM, Lange BCL, Montgomery P. Barriers and facilitators to implementing evidence-based interventions among third sector organisations: a systematic review. Implement Sci. 2018;13(1):103. doi: 10.1186/s13012-018-0789-7.
    1. Sveinsdottir V, Løvvik C, Fyhn T, Monstad K, Ludvigsen K, Øverland S, et al. Protocol for the effect evaluation of Individual Placement and Support (IPS): a randomized controlled multicenter trial of IPS versus treatment as usual for patients with moderate to severe mental illness in Norway. BMC Psychiatry. 2014;14(1):307. doi: 10.1186/s12888-014-0307-7.
    1. Moore G, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258. doi: 10.1136/bmj.h1258.
    1. Johnson R, Onwuegbuzie A. Mixed methods research: a research paradigm whose time has come. Educ Res. 2004;33(7):14–26. doi: 10.3102/0013189X033007014.
    1. Palinkas LA, Aarons GA, Horwitz S, Chamberlain P, Hurlburt M, Landsverk J. Mixed method designs in implementation research. Adm Policy Ment Health Ment Health Serv Res. 2011;38(1):44–53. doi: 10.1007/s10488-010-0314-z.
    1. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International Neuropsychiatric Interview (MINI): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59(Suppl 20):22–3.
    1. Boyatzis RE. Transforming qualitative information: thematic analysis and code development. Thousand Oaks: SAGE; 1998.
    1. Joffe H, Yardley L. Content and thematic analysis. In: Marks IDF, Yardley L, editors. Research methods for clinical and health psychology. London: SAGE Publications; 2004. pp. 56–68.
    1. Reme S, Monstad K, Fyhn T, Øverland S, Ludvigsen K, Sveinsdottir V, et al. Effektevaluering av Individuell jobbstøtte (IPS): sluttrapport. Bergen: Uni Research; 2016.
    1. Johnson R, Floyd M, Pilling D, Boyce M, Grove B, Secker J, et al. Service users’ perceptions of the effective ingredients in supported employment. J Ment Health. 2009;18(2):121–128. doi: 10.1080/09638230701879151.
    1. Glover CM, Frounfelker RL. Competencies of more and less successful employment specialists. Community Ment Health J. 2013;49(3):311–316. doi: 10.1007/s10597-011-9471-0.
    1. Cook J, Mulkern V, Grey D, Burke-Miller J, Blyler C, Razzano L, et al. Effects of local unemployment rate on vocational outcomes in a randomized trial of supported employment for individuals with psychiatric disabilities. J Vocat Rehabil. 2006;25(2):71–84.
    1. Moen EÅ, Walseth LT, Larsen IB. Experiences of participating in individual placement and support: a meta-ethnographic review and synthesis of qualitative studies. Scand J Caring Sci. 2020. 10.1111/scs.12848.
    1. Cocks E, Boaden R. Evaluation of an employment program for people with mental illness using the Supported Employment Fidelity Scale. Aust Occup Ther J. 2009;56(5):300–306. doi: 10.1111/j.1440-1630.2008.00762.x.
    1. Oldman J, Thomson L, Calsaferri K, Luke A, Bond G. A case report of the conversion of sheltered employment to evidence-based supported employment in Canada. Psychiatr Serv. 2005;56(11):1436–1440. doi: 10.1176/appi.ps.56.11.1436.
    1. van Erp N, Giesen F, van Weeghel J, Kroon H, Michon H, Becker D, et al. A multisite study of implementing supported employment in the Netherlands. Psychiatr Serv. 2007;58(11):1421–1426. doi: 10.1176/ps.2007.58.11.1421.
    1. Becker D, Smith J, Tanzman B, Drake R, Tremblay T. Fidelity of supported employment programs and employment outcomes. Psychiatr Serv. 2001;52(6):834–836. doi: 10.1176/appi.ps.52.6.834.
    1. Shepherd G, Lockett H, Bacon J, Grove B. Establishing IPS in clinical teams-Some key themes from a national implementation programme. J Rehabil. 2012;78(1):30–36.
    1. Leff H, Cook J, Gold P, Toprac M, Blyler C, Goldberg R, et al. Effects of job development and job support on competitive employment of persons with severe mental illness. Psychiatr Serv. 2005;56(10):1237–1244. doi: 10.1176/appi.ps.56.10.1237.
    1. Carlson L, Smith G, Rapp C. Evaluation of Conceptual Selling® as a job development planning process. Psychiatr Rehabil J. 2008;31(3):219. doi: 10.2975/31.3.2008.219.225.
    1. Marwaha S, Balachandra S, Johnson S. Clinicians’ attitudes to the employment of people with psychosis. Soc Psychiatry Psychiatr Epidemiol. 2009;44(5):349. doi: 10.1007/s00127-008-0447-5.
    1. Killackey E, Waghorn G. The challenge of integrating employment services with public mental health services in Australia: progress at the first demonstration site. Psychiatr Rehabil J. 2008;32(1):63–66. doi: 10.2975/32.1.2008.63.66.
    1. Bond G, Drake R, Becker D. Generalizability of the Individual Placement and Support (IPS) model of supported employment outside the US. World Psychiatry. 2012;11(1):32–39. doi: 10.1016/j.wpsyc.2012.01.005.
    1. Gold P, Meisler N, Santos A, Carnemolla M, Williams O, Keleher J. Randomized trial of supported employment integrated with assertive community treatment for rural adults with severe mental illness. Schizophr Bull. 2006;32(2):378–395. doi: 10.1093/schbul/sbi056.
    1. Metcalfe J, Drake R, Bond G. Economic, labor, and regulatory moderators of the effect of Individual Placement and Support among people with severe mental illness: a systematic review and meta-analysis. Schizophr Bull. 2017;44(1):22–31. doi: 10.1093/schbul/sbx132.
    1. Vukadin M, Schaafsma FG, Westerman MJ, Michon HWC, Anema JR. Experiences with the implementation of Individual Placement and Support for people with severe mental illness: a qualitative study among stakeholders. BMC Psychiatry. 2018;18(1):145. doi: 10.1186/s12888-018-1729-4.
    1. Latimer E. An effective intervention delivered at sub-therapeutic dose becomes an ineffective intervention. Br J Psychiatry. 2010;196(5):341–342. doi: 10.1192/bjp.bp.109.073338.

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe