Development of a scoring system for non-specialist ratings of clinical competence in global mental health: a qualitative process evaluation of the Enhancing Assessment of Common Therapeutic Factors (ENACT) scale

Brandon A Kohrt, Megan K Ramaiya, Sauharda Rai, Anvita Bhardwaj, Mark J D Jordans, Brandon A Kohrt, Megan K Ramaiya, Sauharda Rai, Anvita Bhardwaj, Mark J D Jordans

Abstract

Task-sharing is the involvement of non-specialist providers to deliver mental health services. A challenge for task-sharing programs is to achieve and maintain clinical competence of non-specialists, including primary care workers, paraprofessionals, and lay providers. We developed a tool for non-specialist peer ratings of common factors clinical competency to evaluate and optimize competence during training and supervision in global mental health task-sharing initiatives. The 18-item ENhancing Assessment of Common Therapeutic factors (ENACT) tool was pilot-tested with non-specialists participating in mental health Gap Action Programme trainings in Nepal. Qualitative process evaluation was used to document development of the peer rating scoring system. Qualitative data included interviews with trainers and raters as well as transcripts of pre- and post-training observed structured clinical evaluations. Five challenges for non-specialist peer ratings were identified through the process evaluation: (1) balance of training and supervision objectives with research objectives; (2) burden for peer raters due to number of scale items, number of response options, and use of behavioral counts; (3) capturing hierarchy of clinical skills; (4) objective v. subjective aspects of rating; and (5) social desirability when rating peers. The process culminated in five recommendations based on the key findings for the development of scales to be used by non-specialists for peer ratings in low-resource settings. Further research is needed to determine the ability of ENACT to capture the relationship of clinical competence with client outcomes and to explore the relevance of these recommendations for non-specialist peer ratings in high-resource settings.

Figures

Fig. 1.
Fig. 1.
Scoring guideline development process within the ENACT tool development process.
Fig. 2.
Fig. 2.
First (A), second (B), and final (C) iterations of scoring system for ENACT.
Fig. 3.
Fig. 3.
Total response decisions for therapist rating tools. Total response decisions refer to number of items multiplied by the number of response options per item.

References

    1. Abas M, Baingana F, Broadhead J, Iacoponi E, Vanderpyl J (2003). Common mental disorders and primary health care: current practice in low-income countries. Harvard Review of Psychiatry 11, 166–173.
    1. Alegría M, Chatterji P, Wells K, Cao Z, Chen C-n, Takeuchi D, Meng X-L (2008). Disparity in depression treatment among racial and ethnic minority populations in the United States. Psychiatric Services 59, 1264–1272.
    1. Anastasi A (1976). Psychological Testing. 4th edn. Macmillan: New York.
    1. Biernat M, Manis M, Nelson TE (1991). Stereotypes and standards of judgment. Journal of Personality and Social Psychology 60, 485.
    1. Blackburn IM, James IA, Milne DL, Baker C, Standart S, Garland A, Reichelt FK (2001). The revised cognitive therapy scale (CTS-R): psychometric properties. Behavioural and Cognitive Psychotherapy 29, 431–446.
    1. Cahill J, Barkham M, Hardy G, Gilbody S, Richards D, Bower P, Connell J (2008). A review and critical appraisal of measures of therapist-patient interactions in mental health settings. Health Technology Assessment 12, iii, ix–47.
    1. Chibanda D, Mesu P, Kajawu L, Cowan F, Araya R, Abas M (2011). Problem-solving therapy for depression and common mental disorders in Zimbabwe: piloting a task-shifting primary mental health care intervention in a population with a high prevalence of people living with HIV. BMC Public Health 11, 828.
    1. Corrigan JD, Schmidt LD (1983). Development and validation of revisions in the counselor rating form. Journal of Counseling Psychology 30, 64–75.
    1. Fairburn CG, Cooper Z (2011). Therapist competence, therapy quality, and therapist training. Behaviour Research and Therapy 49, 373–378.
    1. Frank JD, Frank JB (1991). Persuasion and Healing: A Comparative Study of Psychotherapy. 3rd edn. Johns Hopkins University Press: Baltimore.
    1. Fuller J, Perkins D, Parker S, Holdsworth L, Kelly B, Roberts R, Fragar L (2011). Building effective service linkages in primary mental health care: a narrative review part 2. BMC Health Services Research 11, 66.
    1. Garfield SL (1973). Basic ingredients or common factors in psychotherapy? Journal of Consulting and Clinical Psychology 41, 9–12.
    1. Greenberg RP (2004). Essential ingredients for successful psychotherapy: effect of common factors In The Art and Science of Brief Psychotherapies: A Practitioner's Guide (Eds Dewan M. J., Steenbarger B. N. and Greenberg R. P.), pp. 231–241. American Psychiatric Publishing, Inc.: Arlington, Virginia, USA.
    1. Heine SJ, Lehman DR, Peng K, Greenholtz J (2002). What's wrong with cross-cultural comparisons of subjective Likert scales?: The reference-group effect. Journal of Personality and Social Psychology 82, 903–918.
    1. Henry WP, Schacht TE, Strupp HH, Butler SF, Binder JL (1993). Effects of training in time-limited dynamic psychotherapy: mediators of therapists’ responses to training. Journal of Consulting & Clinical Psychology 61, 441–447.
    1. Hill CE, Gormally J (1977). Effects of reflection, restatement, probe, and nonverbal behaviors on client affect. Journal of Counseling Psychology 24, 92–97.
    1. Hoyt MF, Marmar CR, Horowitz MJ, Alvarez WF (1981). The therapist action scale and the patient action scale – instruments for the assessment of activities during dynamic psychotherapy. Psychotherapy-Theory Research and Practice 18, 109–116.
    1. Ivey AE, Authier J (1978). Microcounseling: Innovations in Interviewing, Counseling, Psychotherapy, and Psychoeducation. Charles C. Thomas: Springfield, Illinois, USA.
    1. Jenkins R, Othieno C, Okeyo S, Kaseje D, Aruwa J, Oyugi H, Kauye F (2013). Short structured general mental health in service training programme in Kenya improves patient health and social outcomes but not detection of mental health problems-a pragmatic cluster randomised controlled trial. International Journal of Mental Health Systems 7, 25.
    1. Jordans M, Luitel N, Tomlinson M, Komproe I (2013). Setting priorities for mental health care in Nepal: a formative study. BMC Psychiatry 13, 332.
    1. Jordans MJD, Luitel NP, Pokhrel P, Patel V (2015). Development and pilot testing of a mental healthcare plan in Nepal. The British Journal of Psychiatry. Supplement, S1–S8. doi: 10.1192/bjp.bp.114.153718.
    1. Kabura P, Fleming LM, Tobin DJ (2005). Microcounseling skills training for informal helpers in Uganda. International Journal of Social Psychiatry 51, 63–70.
    1. Kakuma R, Minas H, van Ginneken N, Dal Poz MR, Desiraju K, Morris JE, Scheffler RM (2011). Human resources for mental health care: current situation and strategies for action. The Lancet 378, 1654–1663.
    1. Karson M, Fox J (2010). Common skills that underlie the common factors of successful psychotherapy. American Journal of Psychotherapy 64, 269–281.
    1. Kohrt BA, Harper I (2008). Navigating diagnoses: understanding mind-body relations, mental health, and stigma in Nepal. Culture, Medicine & Psychiatry 32, 462–491.
    1. Kohrt BA, Jordans MJ, Tol WA, Luitel NP, Maharjan SM, Upadhaya N (2011). Validation of cross-cultural child mental health and psychosocial research instruments: adapting the depression self-rating scale and child PTSD symptom scale in Nepal. BMC Psychiatry 11, e127.
    1. Kohrt BA, Jordans MJD, Rai S, Shrestha P, Luitel NP, Ramaiya M, Patel V (2015). Therapist competence in Global Mental Health: Development of the Enhancing Assessment of Common Therapeutic Factors (ENACT) rating scale. Behaviour Research and Therapy 69, 11–21.
    1. Krosnick JA (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied cognitive psychology 5, 213–236.
    1. Lafromboise TD, Coleman HLK, Hernandez A (1991). Development and factor structure of the cross-cultural counseling inventory revised. Professional Psychology-Research and Practice 22, 380–388.
    1. Lipsitz J, Kobak K, Feiger A, Sikich D, Moroz G, Engelhard A (2004). The rater applied performance scale: development and reliability. Psychiatry Research 127, 147–155.
    1. Luborsky L, Barber JP, Siqueland L, Johnson S, Najavits LM, Frank A, Daley D (1996). The revised helping alliance questionnaire (HAq-II): psychometric properties. The Journal of Psychotherapy Practice and Research 5, 260–271.
    1. Lund C, Tomlinson M, De Silva M, Fekadu A, Shidhaye R, Jordans M, Patel V (2012). PRIME: a programme to reduce the treatment gap for mental disorders in five low- and middle-income countries. PLoS Medicine/Public Library of Science 9, e1001359.
    1. Makanjuola V, Doku V, Jenkins R, Gureje O (2012). Monitoring and evaluation of the activities of trainees in the ‘training of trainers’ workshop at Ibadan, south-west Nigeria. Mental Health in Family Medicine 9, 25–32.
    1. Margison FR, McGrath G, Barkham M, Clark JM, Audin K, Connell J, Evans C (2000). Measurement and psychotherapy: evidence-based practice and practice-based evidence. The British Journal of Psychiatry 177, 123–130.
    1. Maslach C, Jackson SE, Leiter MP (1996). Maslach Burnout Inventory Manual. 3rd edn. Consulting Psychologists Press, Inc.: Palo Alto, CA.
    1. Miller GA (1956). The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychological Review 63, 81–97.
    1. Montgomery EC, Kunik ME, Wilson N, Stanley MA, Weiss B (2010). Can paraprofessionals deliver cognitive-behavioral therapy to treat anxiety and depressive symptoms? Bulletin of the Menninger Clinic 74, 45–62.
    1. Murray LK, Dorsey S, Bolton P, Jordans MJ, Rahman A, Bass J, Verdeli H (2011). Building capacity in mental health interventions in low resource countries: an apprenticeship model for training local providers. International Journal of Mental Health Systems 5, 30.
    1. Murray LK, Tol W, Jordans M, Sabir G, Amin AM, Bolton P, Thornicroft G (2014a). Dissemination and implementation of evidence based, mental health interventions in post conflict, low resource settings. Intervention 12, 94–112.
    1. Murray LKP, Skavenski SMSWMPH, Bass JP, Wilcox HP, Bolton PMMPH, Imasiku MP, Mayeya JMPH (2014b). Implementing evidence-based mental health care in low-resource settings: a focus on safety planning procedures. Journal of Cognitive Psychotherapy 28, 168–185.
    1. O'Malley SS, Suh CS, Strupp HH (1983). The Vanderbilt psychotherapy process scale: a report on the scale development and a process-outcome study. Journal of Consulting and Clinical Psychology 51, 581–586.
    1. Patel V (2009). The future of psychiatry in low- and middle-income countries. Psychological Medicine 39, 1759–1762.
    1. Patel V (2012). Global mental health: from science to action. Harvard Review of Psychiatry 20, 6–12.
    1. Patel V, Weobong B, Nadkarni A, Weiss H, Anand A, Naik S, Kirkwood B (2014). The effectiveness and cost-effectiveness of lay counsellor-delivered psychological treatments for harmful and dependent drinking and moderate to severe depression in primary care in India: PREMIUM study protocol for randomized controlled trials. Trials 15, 101.
    1. QSR International (2012). NVivo qualitative data analysis software (Version 10). QSR International Pty Ltd.: Doncaster, Australia.
    1. Robiner WN (2006). The mental health professions: workforce supply and demand, issues, and challenges. Clinical Psychology Review 26, 600–625.
    1. Sadik S, Abdulrahman S, Bradley M, Jenkins R (2011). Integrating mental health into primary health care in Iraq. Mental Health in Family Medicine 8, 39–49.
    1. Samstag LW, Batchelder ST, Muran JC, Safran JD, Winston A (1998). Early identification of treatment failures in short-term psychotherapy: an assessment of therapeutic alliance and interpersonal behavior. Journal of Psychotherapy Practice & Research 7, 126–143.
    1. Scheppers E, van Dongen E, Dekker J, Geertzen J, Dekker J (2006). Potential barriers to the use of health services among ethnic minorities: a review. Family Practice 23, 325–348.
    1. Schindler L, Hohenberger-Sieber E, Hahlweg K (1989). Observing client—therapist interaction in behaviour therapy: development and first application of an observational system. British Journal of Clinical Psychology 28, 213–226.
    1. Singla DR, Weobong B, Nadkarni A, Chowdhary N, Shinde S, Anand A, Weiss H (2014). Improving the scalability of psychological treatments in developing countries: an evaluation of lay therapist led therapy quality in Goa, India. Behaviour Research and Therapy 60, 53–59.
    1. Sparks JA, Duncan BL, Miller SD (2008). Common factors in psychotherapy In Twenty-First Century Psychotherapies: Contemporary Approaches to Theory and Practice (ed. Lebow J. L.), pp. 453–497. John Wiley & Sons Inc: Hoboken, NJ, US.
    1. Suh DH, Hong JS, Lee DH, Gonnella JS, Hojat M (2012). The Jefferson scale of physician empathy: a preliminary psychometric study and group comparisons in Korean physicians. Medical Teacher 34, E464–E468.
    1. Tol WA, Patel V, Tomlinson M, Baingana F, Galappatti A, Silove D, Panter-Brick C (2012). Relevance or excellence? Setting research priorities for mental health and psychosocial support in humanitarian settings. Harvard Review of Psychiatry 20, 25–36.
    1. Unützer J, Schoenbaum M, Druss BG, Katon WJ (2006). Transforming mental health care at the interface with general medicine: report for the Presidents Commission. Psychiatric Services 57, 37–47.
    1. van Ginneken N, Tharyan P, Lewin S, Rao GN, Meera S, Pian J, Patel V (2013). Non-specialist health worker interventions for the care of mental, neurological and substance-abuse disorders in low-and middle-income countries. The Cochrane Database of Systematic Reviews 11, CD009149.
    1. Wampold BE (2011). The research evidence for common factors models: a historically situated perspective In The Heart and Soul of Change: Delivering What Works in Therapy, 2nd edn (ed. Duncan B. L., Miller S. D., Wampold B. E. and Hubble M. A.), pp. 49–82. American Psychological Association: Washington, DC.
    1. Wang PS, Demler O, Olfson M, Pincus HA, Wells KB, Kessler RC (2006). Changing profiles of service sectors used for mental health care in the United States. American Journal of Psychiatry 163, 1187–1198.
    1. WHO (2008). Task Shifting: Rational Redistribution of Tasks Among Health Workforce Teams: Global Recommendations and Guidelines. WHO Press: Geneva.
    1. WHO (2010). mhGAP Intervention Guide for Mental, Neurological and Substance-Use Disorders in Non-Specialized Health Settings: Mental Health Gap Action Programme (mhGAP) (pp. 83). WHO Press: Geneva.

Source: PubMed

Sponsor e collaboratori

Condizioni mediche

Interventi farmacologici

3
Sottoscrivi