Use of role plays to assess therapist competency and its association with client outcomes in psychological interventions: A scoping review and competency research agenda

Abstract

A major challenge in scaling-up psychological interventions worldwide is how to evaluate competency among new workforces engaged in psychological services. One approach to measuring competency is through standardized role plays. Role plays have the benefits of standardization and reliance on observed behavior rather than written knowledge. However, role plays are also resource intensive and dependent upon inter-rater reliability. We undertook a two-part scoping review to describe how competency is conceptualized in studies evaluating the relationship of competency with client outcomes. We focused on use of role plays including achieving inter-rater reliability and the association with client outcomes. First, we identified 4 reviews encompassing 61 studies evaluating the association of competency with client outcomes. Second, we identified 39 competency evaluation tools, of which 21 were used in comparisons with client outcomes. Inter-rater reliability (intraclass correlation coefficient) was reported for 15 tools and ranged from 0.53 to 0.96 (mean ICC = 0.77). However, we found that none of the outcome comparison studies measured competency with standardized role plays. Instead, studies typically used therapy quality (i.e., session ratings with actual clients) as a proxy for competency. This reveals a gap in the evidence base for competency and its role in predicting client outcomes. We therefore propose a competency research agenda to develop an evidence-base for objective, standardized role plays to measure competency and its association with client outcomes. OPEN SCIENCE REGISTRATION #: https://osf.io/nqhu7/.

Keywords: Common mental disorders; Competence; Developing countries; Paraprofessionals; Psychological treatments; Training.

Conflict of interest statement

Declaration of competing interest The authors declare no conflicts of interest.

Copyright © 2019 Elsevier Ltd. All rights reserved.

Figures

Figure 1:

Search Strategy, Therapist Competency Reviews

Figure 2:

Search Strategy, Competency Assessment Tools Review

Figure 3:. Competency Assessment Tool Items

Abbreviations: Alcohol Intervention Mechanisms Scale (AIMS); Assessment of Core CBT Skills (ACCS); Behavioral Family Management Therapist Competency/Adherence Scale (BFM-TCAS); Behaviour Change Counselling Index (BECCI); CICCA Scale; Cognitive Therapy Adherence-Competence Scale (CTACS); Cognitive Therapy Competence Scale for Social Phobia (CTCS-SP); Cognitive Therapy Rating Scale for Children and Adolescents (CTRS-CA); Cognitive Therapy Scale for Psychoses (CTS-Psy); Cognitive Therapy Scale Revised (CTS-R); Cognitive-Behavioral Treatment for Anxiety in Youth Competence Scale (CBAY-C); Competence and Adherence Scale for Cognitive Behavioral Therapy (CAS-CBT) for anxiety disorders in youth; Competence Rating Scale for Cognitive Processing Therapy (CRS-CPT); Competence Rating Scale for PTSD (CRS-PTSD); Fidelity of Implementation Rating System (FIMP); Independent Tape Rater Scale (ITRS); IPT Adherence and Quality Scale (IPTAQS); Manual -Assisted Cognitive Behavior Therapy Rating Scale (MACT-RS); Manual for the Motivational Interviewing Skill Code Version 2.1 (MISC); Mentalization-Based Treatment for BPD (MBT-ACS); Mindfulness-based Interventions Teaching Assessment Criteria (MBI:TAC); Motivational Interviewing Target Scheme (MITS 2.1); Motivational Interviewing Treatment Integrity 3.1.1 (MITI 3.1.1); Guided Treatment Software to Teach Clinicians How to Deliver Problem-Solving Treatment for Depression (ePST); Quality of the Counselling for Alcohol Programme (Q-CAP) scale; Quality of the Healthy Activity Programme (Q-HAP) scale; Screening, Brief Intervention, and Referral to Treatment Coding Scale (MD3 SBIRT); Supervisor Rating Form (SRF); The Cognitive Behavior Therapy Scale for Children and Young People (CBTS-CYP); Therapeutic Competence Scale (TCS) for Developmentally Adapted Cognitive Processing Therapy for children and adolescents (D-CPT); Therapist Behavior Rating Scale--Competence (TBRSC); Therapist Empathy Scale (TES); Therapist Facilitating Scale (TFS); UCL Scale for Assessing Cognitive and Behavioural Therapy (UCL-CBT); UCL Scale for Assessing Core and Generic Therapeutic Competences (UCL-CGTC); UK Alcohol Treatment Trial Process Rating Scale (UKATT); Video Assessment of Simulated Encounters (VASE-R); Yale Adherence and Competence Scale Guidelines (II) (YACS).

Figure 4:. Chart of Findings for Evaluated Therapist Competency and Client Outcomes

HIC, high-income countries; LMIC, low- and middle-income countries

Figure 5.

Conceptual models for designing competency evaluation studies