School Health Implementation Network: Eastern Mediterranean (SHINE)

Evaluating the Effectiveness of Technology Assisted School Mental Health Program to Improve Socio-emotional Wellbeing of School Going Children in Public Schools of Rural Pakistan: A Cluster Randomized Controlled Trial (cRCT)

Background

An estimated 10-20% of children globally are affected by a mental health problem. Child mental health has been identified as a priority issue by the World Health Organization's Eastern Mediterranean Regional Office (WHO EMRO). Following consultations with international and regional experts and stakeholders, WHO EMRO developed an evidence-based School Mental Health Program (SMHP), endorsed by WHO EMRO member countries, including Pakistan. The federal and provincial health departments in Pakistan made recommendations for a phased implementation of the SMHP in a pilot district. In the formative phase of this program, a number of implementation challenges were identified by the stakeholders. Broadly, these included the need to operationalize and adapt the existing components of the intervention to the local context and to develop sustainable mechanisms for delivery of quality training and supervision.

Informed by the results of a formative phase investigations, the SHINE scale-up research team adapted the SMHP (henceforth called Conventional SMHP or cSMHP) to address these implementation challenges. The enhanced version of the intervention is called Enhanced School Mental Health Program (eSMHP). Enhancements to cSMHP have occurred at two levels: A) Content enhancements, such as a collaborative care model for engaging parents/primary caregivers, strategies for teacher's wellbeing, and adaptation and operationalization of particular clinical intervention strategies and B) Technological enhancements which include adaptation of the training manual for delivery using an online training platform, and a 'Chat-bot' to aid the implementation of intervention strategies in classroom settings.

Objectives

The primary objective of the study is to evaluate the effectiveness of eSMHP in reducing socio-emotional difficulties in school-going children, aged 8-13, compared to cSMHP in Gujar Khan, a rural sub-district of Rawalpindi, Pakistan.

The secondary objectives are to compare the cost-effectiveness, acceptability, adoption, appropriateness (including cultural appropriateness), feasibility, penetration and sustainability of scaled-up implementation of eSMHP and cSMHP. It is hypothesized that eSMHP will prove to be both more effective and more scalable than cSMHP.

Study population

The research is embedded within the phased district level implementation of the cSMHP in Rawalpindi, Pakistan. The study population will consist of children of both genders, aged 8-13 (n=960) with socio-emotional difficulties, studying in rural public schools of sub-district Gujar Khan in Rawalpindi.

Design

The proposed study design is a cluster randomized controlled trial (cRCT), embedded within the conventional implementation of the SMHP. Following relevant ethics committees and regulatory approvals, 80 eligible schools, stratified by gender, will be randomized into intervention and control arms with a 1:1 allocation ratio. Following informed consent from the parent/ primary caregiver, children will be screened for socio-emotional difficulties using Strengths and Difficulties Questionnaire (SDQ). 960 children scoring > 12 on the teacher-rated SDQ total difficulty scores and > 14 on the parent-rated SDQ total difficulty scores will be recruited and equally randomized into intervention and control arms (480 in each arm).

Teachers in the intervention arm will receive training in eSMHP, whereas teachers in the active control will be trained in cSMHP. Trained teachers will deliver the program to children in their respective arms.

Outcome measures

Primary Outcome: The primary outcome is reduction in socio-emotional total difficulties scores, measured with the parent-rated SDQ, 9 months after commencing intervention delivery.

Secondary Outcomes: Implementation data on acceptability, adoption, appropriateness (including cultural appropriateness), feasibility, penetration and sustainability outcomes will be collected from children, parents/primary caregivers, head teachers and teachers. In addition, data will be collected on self-reported Psychological Outcome Profiles (PSYCHLOPS)-KIDS to measure progress on psycho-social problems and wellbeing; annual academic performance; classroom absenteeism, stigmatizing experiences and parent-teacher interaction. Data on teachers' sense of efficacy and subjective well-being, and on the schools' psychosocial environment profile will be collected. All secondary outcome data will be collected at baseline and 9 months after commencing intervention delivery.

Outcomes will be analyzed on an intention to treat basis. The role of various factors as potential mediators and moderators eSMHP effectiveness will be explored.

Cost-effectiveness evaluation of SMHP shall be evaluated in terms of costs associated with implementation of eSMHP compared with cSMHP.

Study Overview

• Status: Completed
• Conditions: Behavioral Symptoms; Emotional Disorder; ADHD; Conduct Disorder; Emotional Stress; Depression, Anxiety; Emotional Problem; Behavioral Problem; Emotional Trauma
• Intervention / Treatment: Behavioral: WHO School Mental Health Program; Behavioral: Enhanced School Mental Health Program (eSMHP)

• Study Type: Interventional
• Enrollment (Actual): 971
• Phase: Not Applicable

Contacts and Locations

Study Locations

• Pakistan
  • Islamabad, Pakistan
    • Human Development Research Foundation

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 8 years to 13 years (Child)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

Description

Eligibility Criteria for Children

Inclusion criteria for children:

Children meeting the following criteria will be included in the study; i. Aged 8-13 years living with parents/primary caregivers ii. Written parent/primary caregiver informed consent or witnessed consent (in case the parent is unable to read and write, the informed consent will be obtained from both parent and witness) and child assent for participation in the study.

iii. Screen positive on teacher-rated SDQ (total difficulties score > 12) and parent-rated SDQ (total difficulties score ≥ 14).

Exclusion criteria for children:

i. Children at high risk of abuse or harm to self or others as reported by the students themselves, teachers or parents/primary caregivers, or identified by the trained assessment team during screening.

ii. Children who require immediate or on-going in-patient medical or psychiatric care, as reported by student themselves or teachers or parents/primary caregivers or identified by the trained assessment team during screening.

iii. Children with deafness, blindness and speech difficulties or with developmental disorders as defined by the WHO mhGAP intervention guide identified by the trained assessment team during screening.

Exclusion from the evaluation will not impact the care received by the school children as all children studying in public schools will receive intervention as part of routine implementation of WHO SMHP.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Prevention
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Double

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Active Comparator: School Mental Health Program-Conventional (cSMHP); The teachers of schools randomized to control arm will receive training in World Health Organization (WHO) School Mental Health Program (SMHP) by mental health experts at WHO collaborating center for mental health research and training, Institute of Psychiatry. The training of teachers will consist of a mix of both didactic and interactive training methodologies in the form of a workshop. The workshop will consist of lectures/presentations, incorporating group discussions/activities. Through didactic methods, teachers will be taught basic theoretical knowledge related to mental health in schools. Training will be followed by monthly supervision meeting of teachers for 9-months.	Behavioral: WHO School Mental Health Program; World Health Organization (WHO) School Mental Health Program (SMHP) is a manual based multi-component, multi-tiered and evidence-informed intervention for common mental health problems in school going children. SMHP is designed to be introduced into the normal classroom and school setting by trained teachers. The intervention has a universal component which takes a whole school approach that aims to promote mental health among all school children. It includes basic counseling skills for teachers, core values of mental health promoting schools and other health promoting efforts that impact upon mental health and can be administered to all students in school and classroom settings. The manual also contains targeted intervention strategies on anxiety, separation anxiety/ school refusal, post-trauma, depression, suicide, ADHD, autism, psychosis, conduct problems and substance use problems that can be implemented by teachers in classroom settings.
Experimental: Enhanced-School Mental Health Program (eSMHP); The teachers of schools randomized to intervention arm will receive online training in adapted version of School Mental Health Program. The online training in adapted School Mental Health Program consists of 4-5 hour, self-paced online training course for teachers. The teachers will register themselves in the online course in the form of a group of 4-5 teachers from each school. The teachers will complete the online training course in a group, with interactive group activities and role plays. Progress to the next module in the online training is conditional upon completion of post-module mental health literacy quiz. A certificate of training completion in adapted SMHP shall be awarded to those teachers who complete the post-test. Teachers will be supported online and in-person by the trainers who are trained in SMHP in monthly supervision meeting of teachers for 9-months.	Behavioral: Enhanced School Mental Health Program (eSMHP); Informed by the results of pilot implementation, a number of content and delivery adaptations have been made to the School Mental Health Program (SMHP) to address the implementation challenges to scale-up of program in Pakistan. The adapted version of the intervention is called Enhanced School Mental Health Program (eSMHP). Enhancements to conventional SMHP have occurred at two levels: A) Content enhancements, such as a collaborative care model for engaging parents/primary caregivers, strategies for teacher's well-being, and adaptation and operationalization of particular clinical intervention strategies and B) Technological enhancements which include adaptation of the training manual for delivery using an online training platform, and a 'Chat-bot' to aid the implementation of intervention strategies in classroom settings.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Strengths and Difficulties Questionnaire (SDQ)	The change in total difficulties scores of children will be measured at 9 months after commencing intervention delivery using the parent-rated Strengths and Difficulties Questionnaire (SDQ) . SDQ has 25 items and consists of sub-scales to measure emotional symptoms, conduct problems, hyperactivity/inattention, peer problems and prosocial behavior. Items are rated on a three-point Likert scale (0= not true, 1= somewhat true, 2=certainly true). Total difficulty score is calculated by summing the responses of each item in all domains except pro-social behavior items. SDQ has been culturally adapted and validated in Pakistan	Baseline and at 9 months after commencing intervention delivery

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Externalizing and internalizing problems on parent-rated SDQ	Externalizing problems in children will be measured at 9 months after commencing intervention delivery, by combining conduct and hyperactivity sub-scales scores on parent-rated SDQ. Internalizing problems in children will be measured at 6 months after commencing intervention delivery, by combining emotional and peer problems subscales on parent-rated SDQ.	Baseline and at 9 months after commencing intervention delivery
Child's psycho-social well being and functioning (PSYCHLOPS)-Kids	Child's insight into his/her problems and wellbeing would be measured using the self-administered PSYCHLOPS Kids. The outcome measure assesses three domains, including problems, functioning and well-being. PSYCHLOPS KIDS has three questionnaire forms i.e. pre-therapy, during therapy and post therapy version. Some of the items are qualitative and provide additional information about child's problem, functioning and wellbeing, but are not provided a score. Other items are rated on 0-4 scale. The maximum score for each question is 4 (scored 0-4). PSYCHLOPS pre, during and post intervention versions would be administered by assessment team at baseline, and at 9 months after commencing intervention delivery, respectively.	Baseline, at 3 and 9 months after commencing intervention delivery
Pediatric Quality of Life (Peds-QL)	Child's health related quality of life during past month will be measured by parent-rated Pediatric Quality of Life (Peds-QL). The Peds-QL is 23 item impact module scale that encompasses 4 sub-scales namely physical functioning, emotional functioning, social functioning and school functioning . Items are rated on a 4-point Likert scale (1 = no problem to 4 = almost always a problem). Items are then reverse-scored and linearly transformed to a 0-100, so that higher scores indicate better quality of life. This tool yields a total score (all 23 items), physical health Summary score (8 items), Psychosocial Health Summary score (10 items) and School Functioning score (5 items).	Baseline and at 9 months after commencing intervention delivery
WHO-Disability Assessment Scale Child Version (WHO-DAS Child 12)	WHODAS Child 12 is used to measure individuals' difficulties due to mental health problems across six domains including cognition, mobility, self-care, getting along, life activities, and participation, during the last 30 days. It has 12-items which are rated on a scale of 0 to 4, with summed total scores ranging from 0 to 48. The parent-rated WHO-DAS 12 - Child Version will be used to measure child's functioning at baseline and 9 months' after commencing intervention delivery.	Baseline and at 9 months after commencing intervention delivery
Academic performance and absenteeism	The record of attendance and academic grades will be obtained from the school records at 9 months after commencing intervention delivery	Baseline and at 9 months after commencing intervention delivery
Teachers' Sense of Efficacy Scale (TSES)	The 12 item teachers' sense of efficacy scale will be used to assess teacher's beliefs about his or her capabilities in enhancing students' learning and ability to get through to students who are difficult or unmotivated. The scale measures teacher's sense of efficacy on three subscales namely, instructional strategies, student engagement and classroom management. The items are rated on 9 points Likert scale ranging from (1) 'None at all' to (9) 'A great deal'.	Baseline, at 3 and 9 months after commencing intervention delivery
Self-Reporting Questionnaire (SRQ)	The Self-Reporting Questionnaire (SRQ) is a 20-item self-report measure to detect non-specific psychological distress, developed by the World Health Organization. Psychological distress is represented by subscales of physical symptoms and emotional symptoms . The SRQ items are scored 0 or 1. A score of 1 indicates the presence of symptoms of psychological distress during past month and a score of 0 indicates absence of symptoms. The maximum score indicates presence of higher psychological distress.	Baseline, at 3 and 9 months after commencing intervention delivery
Parent Teacher Involvement Questionnaire (PTIQ)	The Parent Teacher Involvement Questionnaire (PTIQ) is used to measure the frequency and quality of parental involvement in children's educational progress in school and at home. The parent version of PTIQ has 26 items while the teacher version has 21 items. The four subscales of parent-rated PTIQ include frequency of contact between parents/primary caregivers and teachers, quality of parent teacher relationship, parent's/primary caregivers' volunteering and involvement of parents/primary caregivers, parent's/primary caregivers' endorsement of their child's school, while the three subscales of teacher-rated PTIQ include quality and frequency of contact between parents/primary caregivers and teacher, parent's/primary caregivers volunteering and involvement of parents/primary caregivers, parent's/primary caregivers' endorsement of their child's school . It's a 5-point Likert scale with more than one type of response options. Higher scores indicate more parental involvement.	Baseline and at 9 months after commencing intervention delivery
Psycho-Social Environment (PSE) Profile	Psycho-Social Environment (PSE) Profile will be administered with head teachers and teachers to identify school's capacity to create healthy psycho-social environment for its staff, teachers and students. The Psychosocial Environment Profile is a 98 item scale developed by the World Health Organization to evaluate the extent to which a school's environment contributes to the social and emotional well-being of its students and staff. Each question is scored on a scale from 1 to 4, with 1 representing the lowest and 4 the highest rating of social and emotional support.	Baseline and at 9 months after commencing intervention delivery
Determinants of Implementation Behavior Questionnaire (DIBQ)	DIBQ will be used to measure the change in teachers' behavior to implement the SMHP in school and classroom settings. It is a tool based on 'The Theoretical Domains Framework (TDF)' to measure behavior change of providers. DIBQ will be used to explore the determinants that inhibit or promote the implementation of evidence-based intervention strategies by teachers. An adapted version of DIBQ, based on 18 domains of TDF and consisting of 93 items will be used to assess the change in teachers' behavior regarding implementation of SMHP. Responses on DIBQ are scored from 1 'strongly disagree' to 7 'strongly agree'. Total sum for each domain is calculated and divided by maximum score for the given domain.	At 3 months after commencing intervention delivery
Slef-rated Paediatric Self-Stigmatization Scale (PaedS)	The self-stigma subscale of the PaedS will be used to measure stigma in children and adolescents .It has 5 items that measure sense of shame, embarrassment, and worry about others' responses towards mental health problems. The items are rated on 4 point likert scale, where higher scores indicate greater stigmatization.	Baseline and at 9 months after commencing intervention delivery
Client Services Receipt Inventory	Service use and out-of-pocket expenditure of the research participants (costs for: seeing a doctor or other health care providers; admission to hospital, medicines, tests and extra help at home needed) will be collected using a validated version of the Client Services Received Inventory (CSRI). It has previously been adapted for childhood developmental disorders and autism in study settings.	Baseline and at 9 months after commencing intervention delivery

Collaborators and Investigators

• Sponsor: Human Development Research Foundation, Pakistan
• Collaborators: National Institute of Mental Health (NIMH); University of Washington; World Health Organization; Harvard University; University of Liverpool; Liverpool School of Tropical Medicine; Institute of Psychiatry, WHO Collaborating Center for Mental Health, Rawalpindi; Pakistan Ministry of Health
• Investigators: Principal Investigator: Atif Rahman, PhD, University of Liverpool; Principal Investigator: Lawrence Wissow, MD, University of Washington, Seattle, USA

Publications and helpful links

General Publications

• Goodman R. The Strengths and Difficulties Questionnaire: a research note. J Child Psychol Psychiatry. 1997 Jul;38(5):581-6. doi: 10.1111/j.1469-7610.1997.tb01545.x.
• Varni JW, Seid M, Rode CA. The PedsQL: measurement model for the pediatric quality of life inventory. Med Care. 1999 Feb;37(2):126-39. doi: 10.1097/00005650-199902000-00003.
• Godfrey E, Aubrey M, Crockford S, Haythorne D, Kordowicz M, Ashworth M. The development and testing of PSYCHLOPS Kids: a new child-centred outcome measure. Child Adolesc Ment Health. 2019 Feb;24(1):54-65. doi: 10.1111/camh.12271. Epub 2018 Mar 24.
• Janca A, Kastrup M, Katschnig H, Lopez-Ibor JJ Jr, Mezzich JE, Sartorius N. The World Health Organization Short Disability Assessment Schedule (WHO DAS-S): a tool for the assessment of difficulties in selected areas of functioning of patients with mental disorders. Soc Psychiatry Psychiatr Epidemiol. 1996 Nov;31(6):349-54. doi: 10.1007/BF00783424.
• Tschannen-Moran, M. and A.W. Hoy, Teacher efficacy: Capturing an elusive construct. Teaching and teacher education, 2001. 17(7): p. 783-805.
• Beusenberg, M., J.H. Orley, and W.H. Organization, A User's guide to the self reporting questionnaire (SRQ. 1994, Geneva: World Health Organization
• Organization, W.H., Creating an environment for emotional and social well-being: an important responsibility of a health promoting and child-friendly school. 2003
• Huijg JM, Gebhardt WA, Dusseldorp E, Verheijden MW, van der Zouwe N, Middelkoop BJ, Crone MR. Measuring determinants of implementation behavior: psychometric properties of a questionnaire based on the theoretical domains framework. Implement Sci. 2014 Mar 19;9:33. doi: 10.1186/1748-5908-9-33.
• Moses T. Stigma and self-concept among adolescents receiving mental health treatment. Am J Orthopsychiatry. 2009 Apr;79(2):261-74. doi: 10.1037/a0015696.
• Hamdani SU; Zill-e-Huma, Warraitch A, Suleman N, Muzzafar N, Minhas FA; F.R.C.Psych, Nizami AT; F.C.P.S., Sikander S; F.C.P.S., Pringle B, Hamoda HM, Wang D, Rahman A, Wissow LS. Technology-Assisted Teachers' Training to Promote Socioemotional Well-Being of Children in Public Schools in Rural Pakistan. Psychiatr Serv. 2021 Jan 1;72(1):69-76. doi: 10.1176/appi.ps.202000005. Epub 2020 Aug 25.

Study record dates

Study Major Dates

• Study Start (Actual): October 7, 2019
• Primary Completion (Actual): June 30, 2021
• Study Completion (Actual): April 22, 2022

Study Registration Dates

• First Submitted: September 14, 2019
• First Submitted That Met QC Criteria: September 14, 2019
• First Posted (Actual): September 17, 2019

Study Record Updates

• Last Update Posted (Actual): August 3, 2022
• Last Update Submitted That Met QC Criteria: August 2, 2022
• Last Verified: August 1, 2022

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Mental Disorders; Pathologic Processes; Attention Deficit and Disruptive Behavior Disorders; Neurodevelopmental Disorders; Disease; Problem Behavior; Behavioral Symptoms; Conduct Disorder
• Other Study ID Numbers: SHINE cRCT; U19MH109998-01 (U.S. NIH Grant/Contract)

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: Yes
• IPD Plan Description: In compliance with the data sharing agreement, the unidentifiable data set will be periodically submitted to the online NIMH data repository- National Database for Clinical Trials Related to Mental Illness (NDCT).
• IPD Sharing Time Frame: August, 2021
• IPD Sharing Supporting Information Type: Study Protocol; Statistical Analysis Plan (SAP); Informed Consent Form (ICF)

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No