Systematic Evaluation of Psychotherapy Training: Supervision, Student Learning, and Patient Outcomes

Systematic Evaluation of Psychotherapy Training: Impact of Supervision and Training on Student Learning and Patient Outcomes

The project is a longitudinal observation study that aims to systematically evaluate selected parts of the material collected within the framework of the psychotherapy training conducted at the Center of Psychotherapy Education and Research unit at Karolinska Institutet and Region Stockholm, Sweden. Students at the unit, patients being administered psychotherapy by the students, and supervisors will participate in the study.

The aim of the present study is to investigate the associations between (1) content and quality of psychotherapy supervision, (2) student therapists' competence, and (3) patient outcomes following student-led psychotherapy.

Study Overview

• Status: Recruiting
• Conditions: Student Education
• Intervention / Treatment: Behavioral: Psychotherapy education

Detailed Description

The project is a longitudinal observation study that aims to systematically evaluate selected parts of the material collected within the framework of the psychotherapy training conducted at the Center of Psychotherapy Education and Research unit at Karolinska Institutet and Region Stockholm, Sweden, from 2023 - 2028. The courses/programs that will be included in the project are the psychotherapist program (six semesters), the specialist training for psychologists in adult psychiatry (six semesters), the basic training in evidence-based psychotherapeutic methods (three semesters), and the basic training in psychotherapy for residents in child, adolescent, and adult psychiatry (two semesters). The element of the programs that will be evaluated is student-led psychotherapy, in which students conduct psychotherapy with patients in the public health care system.

The study's research subjects are supervisors, students, and their patients from the unit's student-led psychotherapies. Data consists of material that the supervisors and students have collected within the framework of the educational setting; the data are placed and stored on a secure digital platform (BASS). For objective estimates, the project also intends to code selected audio-recorded psychotherapy and supervision sessions.

The primary research questions are:

1. Do psychotherapy students' general and specific psychotherapy competence increase over time during training?
2. Can statistically significant and/or clinically meaningful patient improvements be observed after student-led psychotherapy?
3. Is students' psychotherapy competence significantly associated with patient outcomes?
4. Are aspects (content and quality) of the psychotherapy supervision setting significantly related to improved student therapists' competence?

The project also aims to psychometrically evaluate (validity and reliability) some of the included instruments.

• Study Type: Observational
• Enrollment (Estimated): 417

Contacts and Locations

• Study Contact: Name: Maria Beckman, PhD; Phone Number: +46-70-797 22 65; Email: maria.beckman@ki.se
• Study Contact Backup: Name: Markus Jansson-Fröjmark, PhD; Phone Number: +46-76-146 82 29; Email: markus.jansson-frojmark@ki.se

Study Locations

• Sweden
  • Stockholm, Sweden
    • Recruiting
    • Center for Psychiatry Research
    • Contact: Maria Beckman, PhD; Phone Number: +46-70-797 22 65; Email: maria.beckman@ki.se

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: Yes

• Sampling Method: Non-Probability Sample

Study Population

Psychotherapy students at the Center for Psychotherapy Education and Research, Karolinska Institutet and Region Stockholm, Sweden

Description

Inclusion Criteria:

• Students at one of the following courses/programs: Primary training in evidence-based psychotherapeutic methods (three semesters) and specialty trainee (ST) for doctors in child, adolescent, and adult psychiatry (two semesters), the postgraduate psychotherapist program (six semesters), and the specialist training program for clinical psychologists (six semesters).
• Clinical supervisors for the students in the abovementioned courses/programs.
• Patients in psychotherapy with students in the abovementioned courses/programs as therapists. The patients are regular patients at public health care facilities in psychiatry and primary care.
• Students, supervisors, and patients must consent to inclusion.

Exclusion Criteria:

• No exclusion criteria are applied.

Study Plan

How is the study designed?

Number of groups / cohorts

1

Cohorts and Interventions

Group / Cohort

Intervention / Treatment

Psychotherapy students; The participating psychotherapy students will come from the following courses at the Center for Psychotherapy Education and Research: the psychotherapist program (six semesters), the specialist training for psychologists in adult psychiatry (six semesters), the basic training in evidence-based psychotherapeutic methods (three semesters), and the basic training in psychotherapy for residents in child, adolescent, and adult psychiatry (two semesters).

Behavioral: Psychotherapy education; Psychotherapy education, including supervision, lectures, and workshops

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
The Generalized Anxiety Disorder Assessment 7 (GAD-7)	The GAD-7 is originally a measure for generalized anxiety disorder, but it is also frequently used to assess general anxiety. The form consists of seven items with a four-point scale. The range is 0-21 points. An eight item describes how disruptive the anxiety is from not at all to very significant, which is not included in the total score. The GAD-7 has been translated into Swedish and has shown good psychometric properties in terms of both reliability and validity.	Change from pre-treatment to immediately after the end of psychotherapy
The Patient Health Questionnaire-9 (PHQ-9)	The PHQ-9 is a well-established depression questionnaire consisting of nine items that correspond to the diagnostic criteria for a major depressive episode according to the DMS-5. How often the symptoms have bothered the patient in the past two weeks is rated by the patient, at the start and end of treatment, on a four-point scale. The range is 0-27 points. The PHQ-9 has been translated into Swedish and has good internal consistency and test-retest reliability.	Change from pre-treatment to immediately after the end of psychotherapy
The WHO Disability Assessment Schedule (WHODAS 2.0)	The WHODAS 2.0 is a widely used generic assessment instrument that provides a standardized method for measuring health and disability across cultures. The form is based on a set of questions from the International Classification of Functioning, Disability, and Health (ICF) that are sufficiently reliable and sensitive to measure changes after interventions. The form is available in seven different versions with 36 questions, 12 questions, and 12+24 questions as interview-administered, self-administered, and proxy-administered, respectively. In the project, the self-administered 12-question form is used.	Change from pre-treatment to immediately after the end of psychotherapy
The Alcohol Use Disorders Identification Test (AUDIT)	The AUDIT is a widely used instrument for identifying hazardous and harmful alcohol consumption. The instrument was initially developed by the WHO and consists of a ten-item questionnaire designed to measure three domains: consumption, dependence, and alcohol-related harm. The project uses a cut-off point of 5 points as an indication of mild alcohol use disorder. The project uses the AUDIT-C, which includes the first three questions of the AUDIT.	Change from pre-treatment to immediately after the end of psychotherapy for those patients scoring above the cut-off at pre-treatment
The Sleep Condition Indicator (SCI)	The SCI will be used to measure sleep disorders. The SCI contains eight questions regarding nocturnal symptoms, function, performance, duration, frequency, and severity. The form is translated into Swedish, and the psychometric properties have been shown in studies to be acceptable to good.	Change from pre-treatment to immediately after the end of psychotherapy
The Liebowitz Social Anxiety Scale, self-report (LSAS-SR)	The LSAS-SR is only used for patients with social anxiety disorder or similar.	Change from pre-treatment to immediately after the end of psychotherapy
The Panic Disorder Severity Scale - Self-rated (PDSS-SR)	The PDSS-SR is only used for patients with panic disorder or similar.	Change from pre-treatment to immediately after the end of psychotherapy
The Penn State Worry Questionnaire (PSWQ)	The PSWQ is only used for patients with generalized anxiety disorder or similar.	Change from pre-treatment to immediately after the end of psychotherapy
The Posttraumatic Stress Disorder Checklist, version 5 (PCL-5)	The PCL-5 is only used for patients with posttraumatic stress disorder or similar.	Change from pre-treatment to immediately after the end of psychotherapy
The Obsessive Compulsive Inventory-Revised (OCI-R)	The OCI-R is only used for patients with obsessive-compulsive disorder or similar.	Change from pre-treatment to immediately after the end of psychotherapy
The Perceived Stress Scale-10 (PSS-10)	The PSS-10 is only used for patients with stress or burnout problems or similar.	Change from pre-treatment to immediately after the end of psychotherapy
The Short Health Anxiety Inventory-14 (SHAI-14)	The SHAI-14 is only used for patients with health anxiety problems or similar.	Change from pre-treatment to immediately after the end of psychotherapy
The Eating Disorder Examination Questionnaire (EDE-Q)	The EDE-Q is only used for patients with eating disorders or similar.	Change from pre-treatment to immediately after the end of psychotherapy
The Drug Use Disorders Identification Test (DUDIT)	The DUDIT is only used for patients with drug problems or similar.	Change from pre-treatment to immediately after the end of psychotherapy

Other Outcome Measures

Outcome Measure	Measure Description	Time Frame
Psychotherapy summary	At the end of each treatment, all students write a summary of the treatment consisting of questions about the student (socio-demographic and contextual), the patient (socio-demographic and clinical), and the treatment (e.g., content, duration, and compliance). The summary concludes by summarizing the results of the patient's pre- and post-measurements, which are guided by the patient's problem domain.	Student and patient data immediately after the end of psychotherapy
Cognitive Therapy Scale-Revised (CTS-R)	The CTS-R is a standard instrument for assessing CBT competence that measures both general psychotherapy aspects and specific CBT aspects. The instrument contains twelve items, which are rated on a scale from 0 to 6 points corresponding to six different categories of therapist competence (incompetent, novice, advanced, beginner, competent, knowledgeable, and expert). The CTS-R has been translated into Swedish and has shown acceptable psychometric properties in several studies.	At first session, first treatment session, fifth treatment session, and last treatment session
Supervision Adherence and Guidance Evaluation (Short-SAGE)	The Short-SAGE is an observational tool to evaluate supervisors' skills and adherence to CBT supervision. Short-SAGE describes 14 behaviors (competencies of the supervisor and supervisee), grouped on two main factors: the Supervisor's set (specific behaviors of the supervisor that are believed to facilitate optimal learning through experience) and the Supervisee's set (specific learning competencies that can be observed in the supervisee). The assessor observes a tutoring session and rates the competencies observed on a 7-point scale. For training purposes, a simpler 3-point scale is used. The instrument is available in Swedish and English, is fairly new, and has only undergone preliminary psychometric evaluations.	At first session, first treatment session, fifth treatment session, and last treatment session
Evaluation of Supervisory Skills and Adherence, 2019 (ESSA-19)	The ESSA-19 is a behavioral coding system to evaluate supervisors' skills and adherence to MI supervision. ESSA has been developed to be used as a tool for structured feedback to MI supervisors and as a measure of supervisory integrity in clinical trials. The manual has been developed at the unit 2019. The instrument is relatively new and has only undergone preliminary psychometric evaluations.	At first session, first treatment session, fifth treatment session, and last treatment session
Supervision evaluation	A unit-developed set of 10 questions focusing on how the students perceive the supervisor and the supervision setting, e.g., regarding feedback, skills training, and activity.	At the middle of the student term (week 10 out of 20)

Collaborators and Investigators

• Sponsor: Region Stockholm
• Collaborators: Karolinska Institutet
• Investigators: Principal Investigator: Maria Beckman, PhD, Center for Psychiatry Research, Karolinska Institutet and Region Stockholm

Study record dates

Study Major Dates

• Study Start (Actual): December 1, 2023
• Primary Completion (Estimated): May 1, 2028
• Study Completion (Estimated): May 1, 2028

Study Registration Dates

• First Submitted: August 25, 2023
• First Submitted That Met QC Criteria: October 23, 2023
• First Posted (Actual): October 27, 2023

Study Record Updates

• Last Update Posted (Estimated): November 25, 2024
• Last Update Submitted That Met QC Criteria: November 21, 2024
• Last Verified: November 1, 2024

More Information

Terms related to this study

• Keywords: psychotherapy; patient outcomes; supervision; student learning
• Other Study ID Numbers: 2023-03668-01

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO
• IPD Plan Description: There is no ethical approval in place to allow for sharing IPD.

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No