Implementation of National Guidelines for Treatment of Psychoses

A Pairwise Randomized Study on Implementation of Guidelines and Evidence Based Treatments of Psychoses

The study is a combined health services research study and a clinical patient outcomes sub-study.

The aims of the study are to give new knowledge on (RQ1) current implementation in mental health services of four evidence based practices for treatment of psychoses, (RQ2) how and to what degree implementation support affects the implementation, and (RQ3) whether improved implementation is associated with better clinical course and higher patient satisfaction.

Pairwise randomized study in six health trusts on implementation of the four evidence based practices physical health care, antipsychotic medication, family psychoeducation, and illness management and recovery. Data on model fidelity and patient course/experience are collected at baseline and after 6, 12 and 18 months. 39 clinical units (CMHCs/departments) choose two practices and receive implementation support on one for 18 months after randomization. RQ1 is answered from baseline data, and RQ2 and RQ3 from data after 6-18 months.

Study Overview

• Status: Completed
• Conditions: Psychotic Disorders
• Intervention / Treatment: Other: Systematic implementation support; Other: No implementation support

Detailed Description

(Based on Protocol of 27 October 2015 approved by the Regional Committee for Medical and Health Research Ethics 17 December 2015, with some adjustments of June and August 2016.)

BACKGROUND: During the last two decades, several evidence-based practices and clinical guidelines have been developed to improve treatment for psychoses. But there is limited knowledge on to what extent guidelines and evidence-based practices for psychoses are implemented in everyday routine practice in Norway and elsewhere, and on how implementation support and readiness for change influence the implementation process and degree of implementation. There is even less knowledge whether implementation of clinical guidelines and evidence-based practices actually improves clinical outcome and patient satisfaction in routine practice. In efficacy studies that guidelines build upon, clinicians may be especially skilled and motivated, and patients with additional problems are often excluded. But in routine practice non-selected clinicians are expected to implement evidence-based practices in treatment of unselected patients. FIDELITY SCALES: Fidelity measures for implementation have been developed for several evidence-based practices in treatments for psychoses, giving data on whether a clinical team or program have implemented key components of the specific evidence-based model. IMPLEMENTATION SUPPORT: Research has shown that several strategies need to be combined both on the system level and for clinicians to achieve a successful implementation of a new practice. Strategies include engaging leaders and clinicians, understanding of the needs to change practice, supervision frequently over some time with small group discussions on feedback from fidelity assessments and other measures, toolkits with a practice manual and other tools to help implement the practice, a "kickoff" to build enthusiasm, and initial training for the practitioners. The implementation support must be reasonably intensive during the first months, must be sensitive to site-specific conditions, and helpful to implementers through three phases of implementation: building momentum for change, making the changes, and reinforcing the changes. Clinicians' readiness to change seems to be an important factor in the implementation process.

SETTING: The mental health clinics of six of the 19 Norwegian health trusts are partners in the project, representing three of the four health regions in Norway and serving a population of 1.9 million (38 % of Norway's population). Altogether they have 30 community mental health centers (CMHCs), each serving a specific area and population in collaboration with other departments. The target group for the project is persons with psychoses served by these mental health services.

RESEARCH QUESTIONS

RQ1.What is the current level of implementation of evidence-based practices recommended in the national clinical guidelines for treatment of persons with psychoses? RQ2.How and to what degree Is the implementation of evidence-based practices influenced by an implementation support program and by clinician readiness for change? RQ3.Does implementation of evidence-based practices improve patient clinical course and patient satisfaction with the services?

HYPOTHESES

(H1).Current implementation of evidence-based practices is low. (H2).Implementation support for an evidence-based practice gives significant increase in implementation of the practice compared to no such support. (H3).Higher implementation of evidence-based practices is associated with better patient clinical course and higher patient satisfaction with the services.

DESIGN

Pairwise randomized study in six health trusts on implementation of the four evidence-based practices physical health care, antipsychotic medication, family psychoeducation, and illness management and recovery. Data on model fidelity and patient course/experience are collected at baseline and after 6, 12 and 18 months. 39 clinical units (CMHCs/departments) choose two practices and receive implementation support on one for 18 months after randomization. RQ1 is answered from baseline data, and RQ2 and RQ3 from data after 6-18 months compared with baseline.

THE IMPLEMENTATION STUDY (Study of health services)

The unit of analysis for the implementation study is the CMHC/department. CMHCs are the primary type of clinical units because they give a whole range of outpatient and inpatient services over time for the population in a local area, including for persons with psychosis. But other departments serving patients with psychosis may also participate in the study.

Primary outcome measure is fidelity to models for the four evidence-based practices. Each practice covering a set of core recommendations in the Norwegian guidelines, has been chosen based on a set of specific criteria (see below) and on a survey among the units (CMHCs/departments) in May 2015 on their interest for five evidence-based practices. Each unit chooses two practices that they will implement. For each unit one practice is randomized to implementation support and the other to be control with no implementation support.

Power calculation with 5% two-tailed significance and 90% power based on data from the US National Evidence Based Project (US NEBP) shows that 8 units are needed in each arm to show that implementation support for a practice gives a significant increase in fidelity compared to baseline with low fidelity. Average fidelity scores in the US NEBP project for five practices and 51 sites with implementation support were 2.28 at baseline and 3.76 after 12 months, with effect size 1.70.

The four practices for randomization were selected based on specific criteria like covering guidelines recommendation with high evidence and/or importance, considered to be among core elements in services for persons with psychosis, fidelity scales were available or could be developed, relevance for most patients with psychosis, and the required competence is available or within reach by training. The four selected practices are described below.

Antipsychotic medication: The model for this practice includes limiting polypharmacy, avoiding high doses and fast dose increase, adjusting medication to phases and situations, monitoring effect and side effects, and assessing and supporting adherence. As no established fidelity scale were found to cover the specified practice, a fidelity scale had to be developed.

Family psychoeducation: Family and the patient is offered psychoeducation with frequent sessions over some time alone or together with other families. A model and fidelity scales existed for this evidence-based practice.

Physical health care: The need for physical health care is great due to the documented higher morbidity and mortality of persons with schizophrenia and related psychoses. As no established fidelity scale were found to cover the specified practice, a fidelity scale had to be developed.

Illness management and recovery (IMR): IMR is a training program with several modules with emphasis on illness management, independent living and personal recovery. The IMR manual and fidelity scales were translated to Norwegian and made available by another project.

Emphasis on personal recovery and shared decision making are recommended in the guidelines. These will also be measured and included in data analyses as additional outcome measures of patient experiences with the services.

Data collection in Implementation study: Fidelity assessment of the two practices in each clinical unit (CMHC/department) is done every 6 months independently by two fidelity assessors to calculate inter-rater reliability. Consensus score will be used in data analyses. The assessments are done at site visits where the researchers gather information from the sources specified in the fidelity measure (written material, interviews, observations and reading randomly selected patient records).

PATIENT OUTCOMES SUBSTUDY (Clinical study of patients)

A sub-study of patient clinical course and patient satisfaction with key elements of each evidence-based practice aims to measure whether the patients experience differences in implementation of the practices (RQ3). The unit of analysis for this sub-study is the patient.

Inclusion criteria: Patients 16+ years old with psychoses (diagnosis F20-29 in the ICD-10). Based on national statistics the aim is to include 20% of the persons with psychoses seen by the mental health services during a year, which would be a total of 1074 patients (96-280 per health trust based on population in the area). Only patients giving written informed consent will be included. There are no exclusion criteria.

The inclusion period of 6 months will start when the implementation support starts and will be coordinated by the local coordinator. Eligible patients already in contact with the clinic will be invited to take part, and new referred patients assessed to have psychosis will be invited consecutively until the requisite number is met. Each included patient will be followed for 18 months from inclusion.

Data collection will be coordinated by the local coordinator, and done by the clinical units at baseline (inclusion) and after 6, 12 and 18 months, as long as the patient is in contact with the mental health services. Clinicians will administer questionnaires to patients and do clinical ratings of outcome measures.

The project will order data extraction from official registers on use of health services, primary care, social benefits, prescriptions, and causes of death. This will be done for the whole period (18 months before inclusion and 18 months during the study) when such data will be available after the data collection of the study has been finished in 2018.

ANALYSES

Data analyses for the first available data (baseline data) will start in September 2017.

Analyses of RQ1: The percentage of sites achieving high fidelity (4.0) will be calculated for each practice at baseline. The distribution of fidelity scores and exploration of contributing factors will also be analyzed based on data collected on implementation support and readiness to change.

Analyses of RQ2: First, the experimental and control conditions on fidelity will be compared across time, controlling for baseline and ignoring the content of practice. This analysis will consist of 39 pairs of sites, measured 4 points in time. Baseline scores will be used as covariates. Various ANOVA analyses is considered. Second, for each practice all experimental sites will be compared to all control sites. The number of observation in this analysis will depend on how many units choose each practice. Baseline scores will be used as covariates. Third, within-group changes over time will be examined for each practice receiving implementation support, calculating effect size and determining percentage of sites achieving various levels of fidelity at each point in time. Readiness for change may be included as a moderator in these analyses.

Analyses of RQ3: Data analysis on primary and secondary patient outcomes will include multi-level analysis with patient as individual level and CMHC/department (clinical unit) as system level. Correlations between fidelity and various patient outcomes may be done as secondary analyses.

• Study Type: Observational
• Enrollment (Actual): 325

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 16 years and older (Child, Adult, Older Adult)
• Accepts Healthy Volunteers: No

• Sampling Method: Non-Probability Sample

Study Population

Patients in participating clinical units (community mental health centers or other departments in mental health services in six health trusts in Norway) assessed by clinicians to have a psychotic disorder (ICD-11 diagnosis F20-29) and age 16+.

Description

Inclusion Criteria:

• All patients assessed to have a psychotic disorder (ICD 11: F20-F29)

Exclusion Criteria:

• None

Study Plan

How is the study designed?

Design Details

• Observational Models: Cohort
• Time Perspectives: Prospective

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
Implementation support; Units and their patients where the unit according to pairwise randomization is offered systematic implementation support for 18 months for the specific practice.	Other: Systematic implementation support; Implementation support is offered by implementation trainers visiting each clinical unit every two weeks for 6 months and then monthly for 12 months. Support is given only for the practice randomly assigned to support. The aim is to engage leaders and clinicians in identifying and overcoming implementation barriers, and in building systems to support and sustain implementation. Implementation trainers from all sites are trained together and meet every 2-3 month with leaders of the implementation training for supervision, mutual discussion, and exchange of experiences. At the start of the implementation, leaders and clinicians were also invited to a workshop on the practice they have been randomly assigned to receive support for. Toolkits are made available on a website.
No implementation support; Units and their patients where the unit according to pairwise randomization is not offered systematic implementation support for 18 months for the specific practice.	Other: No implementation support; No implementation support is offered to clinical units for the practice randomly assigned to no implementation support, of the two practices (of the four practices in the project) that unit has chosen to implement.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Implementation Study: Fidelity to the model for each evidence-based practice	Fidelity scales for each practice, with 10-17 items rated from 1 (low) to 5 (high).	Baseline to 18 months (0, 6, 12 and 18 months)
Patient Sub-Study: Patient satisfaction with each evidence-based practice	Set of 5-7 questions developed for each practice, rated 1-5 on a likert scale by patients	Baseline to 18 months (0, 6, 12 and 18 months)

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Patient Sub-Study: Patient general satisfaction with the mental health services	The Client Satisfaction Questionnaire (CSQ8) with 8 questions rated 1-4 by patients.	Baseline (0 months) and 18 months
Patient Sub-Study: Patient experiences of their own mental health and functioning	The BASIS-24 questionnaire with 24 questions on mental health, functioning and substance use. Each question is answered by patients on a scale 1-5.	Baseline to 18 months (0, 6, 12 and 18 months)
Patient Sub-Study: Patient experiences of their own personal recovery	Process of recovery Questionnaire version 2 (QPR V2) with 15 questions completed by patients on a scale 1-5.	Baseline (0 months) and 18 months
Patient Sub-Study: Clinician assessment of patient mental health and functioning	Health of the Nation Outcome Scale (HoNOS) scale rated by clinicians on 12 items rated 0-4 (no problem - very serious problem)	Baseline to 18 months (0, 6, 12 and 18 months)
Patient Sub-Study: Clinician assessment of patient mental health	Clinical Global Impression scale (CGI) rated 1-7 by clinicians.	Baseline to 18 months (0, 6, 12 and 18 months)
Patient Sub-Study: Clinician assessment of patient practical and social functioning	Practical and Social Functioning version 2 (PSF2) scale rated by clinicians on 32 items rated 1-5	Baseline (0 months) and 18 months
Patient Sub-Study: Clinician assessment of patient global functioning	Global Assessment of Functioning Scale, spilt version for symptoms and functioning	Baseline to 18 months (0, 6, 12 and 18 months)
Patient Sub-Study: Clinician assessment of patient substance use last 6 months	Scales for use of alcohol and drugs last 6 months	Baseline to 18 months (0, 6, 12 and 18 months)

Other Outcome Measures

Outcome Measure	Measure Description	Time Frame
Patient experiences of the extent of emphasis on personal recovery by the services	The INSPIRE questionnaire (20 items rated 1-5) completed by the patients	Baseline (0 months) and 18 months
Patient experience of the extent of shared decision-making	The CollaboRATE questionnaire (3 items rated 0-9) completed by patients	Baseline to 18 months (0, 6, 12 and 18 months)
Implementation study: Clinician readiness for change	Implementation Process Assessment Tool (IPAT), a questionnaire (27 items rated 1-6) to clinicians regarding experience of implementation of a specified practice	Baseline to 18 months (0, 6, 12 and 18 months)

Collaborators and Investigators

• Sponsor: University Hospital, Akershus
• Collaborators: University Hospital of North Norway; Helse Stavanger HF; Sorlandet Hospital HF; Sykehuset Innlandet HF; Helse Fonna; Westat; Mental helse
• Investigators: Principal Investigator: Torleif Ruud, MD, PhD, University Hospital, Akershus

Publications and helpful links

General Publications

• Larsen DL, Attkisson CC, Hargreaves WA, Nguyen TD. Assessment of client/patient satisfaction: development of a general scale. Eval Program Plann. 1979;2(3):197-207. doi: 10.1016/0149-7189(79)90094-6. No abstract available.
• Elwyn G, Barr PJ, Grande SW, Thompson R, Walsh T, Ozanne EM. Developing CollaboRATE: a fast and frugal patient-reported measure of shared decision making in clinical encounters. Patient Educ Couns. 2013 Oct;93(1):102-7. doi: 10.1016/j.pec.2013.05.009. Epub 2013 Jun 12.
• Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. Int J Nurs Stud. 2013 May;50(5):587-92. doi: 10.1016/j.ijnurstu.2012.09.010. Epub 2012 Nov 15. No abstract available.
• McHugo GJ, Drake RE, Whitley R, Bond GR, Campbell K, Rapp CA, Goldman HH, Lutz WJ, Finnerty MT. Fidelity outcomes in the National Implementing Evidence-Based Practices Project. Psychiatr Serv. 2007 Oct;58(10):1279-84. doi: 10.1176/ps.2007.58.10.1279.
• Williams J, Leamy M, Bird V, Le Boutillier C, Norton S, Pesola F, Slade M. Development and evaluation of the INSPIRE measure of staff support for personal recovery. Soc Psychiatry Psychiatr Epidemiol. 2015 May;50(5):777-86. doi: 10.1007/s00127-014-0983-0. Epub 2014 Nov 20.
• Priebe S, Huxley P, Knight S, Evans S. Application and results of the Manchester Short Assessment of Quality of Life (MANSA). Int J Soc Psychiatry. 1999 Spring;45(1):7-12. doi: 10.1177/002076409904500102.
• Law H, Neil ST, Dunn G, Morrison AP. Psychometric properties of the questionnaire about the process of recovery (QPR). Schizophr Res. 2014 Jul;156(2-3):184-9. doi: 10.1016/j.schres.2014.04.011. Epub 2014 May 9.
• Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011 Mar;38(2):65-76. doi: 10.1007/s10488-010-0319-7.
• Eisen SV, Normand SL, Belanger AJ, Spiro A 3rd, Esch D. The Revised Behavior and Symptom Identification Scale (BASIS-R): reliability and validity. Med Care. 2004 Dec;42(12):1230-41. doi: 10.1097/00005650-200412000-00010.
• Wing JK, Beevor AS, Curtis RH, Park SB, Hadden S, Burns A. Health of the Nation Outcome Scales (HoNOS). Research and development. Br J Psychiatry. 1998 Jan;172:11-8. doi: 10.1192/bjp.172.1.11.
• Karterud S, Pedersen G, Løvdal H, Friis S. S-GAF: Global Funsjonsskåring - Splittet Versjon [Global Assessment of Functioning - Split version]. Bakgrunn og skåringsveiledning. Klinikk for psykiatri, Ullevål sykehus, Oslo, 1998.
• Torrey WC, Drake RE, Dixon L, Burns BJ, Flynn L, Rush AJ, Clark RE, Klatzker D. Implementing evidence-based practices for persons with severe mental illnesses. Psychiatr Serv. 2001 Jan;52(1):45-50. doi: 10.1176/appi.ps.52.1.45.
• Weinmann S, Koesters M, Becker T. Effects of implementation of psychiatric guidelines on provider performance and patient outcome: systematic review. Acta Psychiatr Scand. 2007 Jun;115(6):420-33. doi: 10.1111/j.1600-0447.2007.01016.x.
• Tansella M, Thornicroft G. Implementation science: understanding the translation of evidence into practice. Br J Psychiatry. 2009 Oct;195(4):283-5. doi: 10.1192/bjp.bp.109.065565.
• Torrey WC, Bond GR, McHugo GJ, Swain K. Evidence-based practice implementation in community mental health settings: the relative importance of key domains of implementation activity. Adm Policy Ment Health. 2012 Sep;39(5):353-64. doi: 10.1007/s10488-011-0357-9.
• Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health. 2009 Jan;36(1):24-34. doi: 10.1007/s10488-008-0197-4. Epub 2008 Dec 23.
• Horne R, Weinman J, Hankins M. The Beliefs about Medicines Questionnaire: The development and evaluation of a new method for assessing the cognitive representation of medication. Psychology and Health 14: 1-24, 1999
• Rishovd Rund B, Ruud T. Måling av funksjonsnivå og funksjonsendring i psykiatrisk behandling [Measuring functioning and change in functioning during psychiatric treatment. Norwegian Board of Health. Report.] Statens helsetilsyn. IK 2422, Oslo. 1994.
• Mueser KT, Drake RE, Clark RE, McHugo GJ, Mercer-McFadden C, Ackerson TH. Toolkit for Evaluating Substance Abuse in Person with Severe Mental Illness. 1995.
• Hartveit M, Hovlid E, Nordin MHA, Ovretveit J, Bond GR, Biringer E, Assmus J, Mariniusson GH, Ruud T. Measuring implementation: development of the implementation process assessment tool (IPAT). BMC Health Serv Res. 2019 Oct 21;19(1):721. doi: 10.1186/s12913-019-4496-0.
• Egeland KM, Heiervang KS, Landers M, Ruud T, Drake RE, Bond GR. Psychometric Properties of a Fidelity Scale for Illness Management and Recovery. Adm Policy Ment Health. 2020 Nov;47(6):885-893. doi: 10.1007/s10488-019-00992-5.
• Ruud T, Drivenes K, Drake RE, Haaland VO, Landers M, Stensrud B, Heiervang KS, Tanum L, Bond GR. The Antipsychotic Medication Management Fidelity Scale: Psychometric properties. Adm Policy Ment Health. 2020 Nov;47(6):911-919. doi: 10.1007/s10488-020-01018-1.
• Ruud T, Hoifodt TS, Hendrick DC, Drake RE, Hoye A, Landers M, Heiervang KS, Bond GR. The Physical Health Care Fidelity Scale: Psychometric Properties. Adm Policy Ment Health. 2020 Nov;47(6):901-910. doi: 10.1007/s10488-020-01019-0.
• Heiervang KS, Egeland KM, Landers M, Ruud T, Joa I, Drake RE, Bond GR. Psychometric Properties of the General Organizational Index (GOI): A Measure of Individualization and Quality Improvement to Complement Program Fidelity. Adm Policy Ment Health. 2020 Nov;47(6):920-926. doi: 10.1007/s10488-020-01025-2.
• Joa I, Johannessen JO, Heiervang KS, Sviland AA, Nordin HA, Landers M, Ruud T, Drake RE, Bond GR. The Family Psychoeducation Fidelity Scale: Psychometric Properties. Adm Policy Ment Health. 2020 Nov;47(6):894-900. doi: 10.1007/s10488-020-01040-3.
• Ruud T, Drake RE, Bond GR. Measuring Fidelity to Evidence-Based Practices: Psychometrics. Adm Policy Ment Health. 2020 Nov;47(6):871-873. doi: 10.1007/s10488-020-01074-7.
• Ruud T, Drake RE, Saltyte Benth J, Drivenes K, Hartveit M, Heiervang K, Hoifodt TS, Haaland VO, Joa I, Johannessen JO, Johansen KJ, Stensrud B, Woldsengen Haugom E, Clausen H, Biringer E, Bond GR. The Effect of Intensive Implementation Support on Fidelity for Four Evidence-Based Psychosis Treatments: A Cluster Randomized Trial. Adm Policy Ment Health. 2021 Sep;48(5):909-920. doi: 10.1007/s10488-021-01136-4. Epub 2021 Apr 19.
• Skar-Froding R, Clausen HK, Saltyte Benth J, Ruud T, Slade M, Sverdvik Heiervang K. The Importance of Personal Recovery and Perceived Recovery Support Among Service Users With Psychosis. Psychiatr Serv. 2021 Jun;72(6):661-668. doi: 10.1176/appi.ps.202000223. Epub 2021 Apr 22.
• Haugom EW, Stensrud B, Beston G, Ruud T, Landheim AS. Mental health professionals' experiences with shared decision-making for patients with psychotic disorders: a qualitative study. BMC Health Serv Res. 2020 Nov 27;20(1):1093. doi: 10.1186/s12913-020-05949-1.
• Hartveit M, Hovlid E, Ovretveit J, Assmus J, Bond G, Joa I, Heiervang K, Stensrud B, Hoifodt TS, Biringer E, Ruud T. Can systematic implementation support improve programme fidelity by improving care providers' perceptions of implementation factors? A cluster randomized trial. BMC Health Serv Res. 2022 Jun 22;22(1):808. doi: 10.1186/s12913-022-08168-y.
• Skar-Froding R, Clausen HK, Saltyte Benth J, Ruud T, Slade M, Sverdvik Heiervang K. Relationship between satisfaction with mental health services, personal recovery and quality of life among service users with psychosis: a cross-sectional study. BMC Health Serv Res. 2021 May 8;21(1):439. doi: 10.1186/s12913-021-06409-0.

Helpful Links

• Website for the study with information and resources (in Norwegian)

Study record dates

Study Major Dates

• Study Start (Actual): June 22, 2016
• Primary Completion (Actual): December 31, 2018
• Study Completion (Actual): December 31, 2018

Study Registration Dates

• First Submitted: August 31, 2017
• First Submitted That Met QC Criteria: September 1, 2017
• First Posted (Actual): September 5, 2017

Study Record Updates

• Last Update Posted (Actual): July 20, 2023
• Last Update Submitted That Met QC Criteria: July 19, 2023
• Last Verified: April 1, 2021

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Mental Disorders; Schizophrenia Spectrum and Other Psychotic Disorders; Psychotic Disorders
• Other Study ID Numbers: 2015106

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No