A Cluster-randomized Control Trial of PAX Good Behavior Game in Swedish Elementary Schools

PAX Good Behavior Game - A Cluster-randomized Controlled Evaluation and Qualitative Analysis of Effects on Work Environment, Learning, Mental Health and Health Economic Aspects of Implementing a Classroom-based Intervention

Two important challenges that primary school in Sweden is confronted by are increasing levels of mental illness among children and adolescents and increasing rates of stress related symptoms and sick leave associated with mental illness in primary school teachers. Primary school is an important arena to address these issues.

PAX is implemented as a part of the regular classroom activities and gives the teacher tools to promote behaviors that are constructive and learning oriented. The programme has been shown to improve student health and performance both short- and long-term. The teachers also receive tools to lessen stressors in their working environment. A Swedish uncontrolled pilot study showed increased well-being in children and reduced teacher stress as well as improved work related well-being at post-intervention.

The aim of this project is to study the effects of PAX on student well-being and teacher stress. In a cluster-randomized design, 84 teachers from 28 schools (clusters) will be randomized to either implement PAX, or to an active control group that receives the same amount of support but with different content. The PAX group will get a two-day education in PAX at the start of the term, followed by an implementation-term. The measurements include self-assessment, blind observer assessments, objectively measuring sound levels, data on sick leave and qualitative interviews.

Baseline measurements are performed before implementation and primary end point is one year after. Costs and savings related to PAX will be measured and put in relation to the quality-adjusted life years of students and teachers. Long-term effects on student performance, health and participation in society will be evaluated in a follow-up project.

Study Overview

• Status: Recruiting
• Conditions: Well-being in Elementary School Children; Mental Health in Elementary School Children; Work-related Stress in Teachers; Academic Performance in Elementary School Children
• Intervention / Treatment: Behavioral: PAX Good Behavior Game; Behavioral: Count on Me!; Behavioral: Collegial Learning Community

Detailed Description

Design

A cluster-randomized controlled trial will be conducted randomizing approximately 28 schools (clusters) with a total of 84 teachers/classrooms into either receiving training and supervision in PAX GBG or receiving equally much training and supervision for implementing one of the control interventions.

Teachers in both arms will be implementing and using the intervention they were randomized to during the first year and then start implementing also the other intervention, while continuing using the first. When the baseline measure occurs at the end of grade 2 and the initial intervention is implemented for pupils during grade 3, the pupils in these classes will only receive one intervention since they move up to middle school where none of the methods are used. The teacher will instead start over with new pupils in grade 1 and use both methods with them. The sub-group of pupils that receives only one intervention will thus preserve the randomized controlled design also for long-term follow-ups.

Procedure

Ideally, half of the classrooms will be enrolled to start the intervention/control intervention during the spring semester and half during the autumn semester. The purpose of this is to control for the occasional effect of the point in time during the school year, and calendar year, when the intervention is introduced.

Credible control interventions requiring the same amount of training and supervision will be used, to control for unspecific effects of implementing a new method. It also makes it possible to promote the study to schools and teachers as a way to implement and compare several useful methods, although for different purposes, which helps to avoid too high expectation or placebo effects for PAX and to minimize possible disappointment and nocebo effects related to having to wait one year before starting with PAX. The school will have the possibility to choose one of the two control interventions - a math application, 'Count On Me!', or Collegial learning community. This serves two purposes; (1) the possibility to choose will further reduce the risk of nocebo or being dissatisfied if not randomized to PAX and (2) it will enable more schools to participate if one of the control interventions would be seen as impractical to implement. Since neither fo the control interventions are deemed to have the same specific effects as PAX, at least not to the same extent, they will both act as controls for unspecific effects.

A total of seven measures will be administered in relation to pupils, teachers and the classroom during the two years, with the baseline (Measurement 0) at the end of the semester before the implementation starts (Semester 0) and the primary endpoint (Measurement 4) one year afterwards. To reduce the burden for teachers and pupils, the full range of outcome measures will be conducted only on two occasions (Measurement 0 and Measurement 4) and otherwise a reduced set of measures will be used for pupils and teachers (For measures on classroom level, the full set of measures will always be applied). The reduced set of self-rating measures for teachers will include work related stress (primary outcome), classroom management, general stress, sleep problems, work ability and observed negative/positive effects of the method being implemented. For pupils, the reduced set will contain questions on well-being (primary measure). Teachers will also measure the well-being for each pupil at each measurement point.

Following time frames will apply to the measurement points:

Measurement 0 (M0) - semester before the first implementation semester, a 4-weeks administration window ending two weeks before the end of the semester.

Measurement 1 (M1) - the first implementation semester, a 6-weeks administration window starting two weeks after the start of the semester (and preferably two weeks after the completion of the intervention training).

Measurement 2 (M2) - the first implementation semester, a 4-weeks administration window ending two weeks before the end of the semester).

Measurement 3 (M3) - the second implementation semester, a 4-weeks administration window starting two weeks after the start of the semester).

Measurement 4 (M4) - primary endpoint, the second implementation semester, a 4-weeks administration window ending two weeks before the end of the semester.

Measurement 5 (M5) - the third implementation semester, a 4-weeks administration window ending two weeks before the end of the semester.

Measurement 6 (M6) - the fourth implementation semester, a 4-weeks administration window ending two weeks before the end of the semester.

Participants and recruitment

Schools, teachers, classrooms and other school staff

The cluster-randomization will be performed on the school-level, and will in turn decide which teachers and pupils that are included. The schools will be recruited via an established network of PAX GBG trainers in Sweden and the research groups other connections with school-related networks and the recruitment will be country wide. The schools will be recruited from at least two different Regions and at least four different municipalities (kommuner) to increase the generalizability of the results. In total, around 28 schools will be included, depending on the number of classrooms participating from each school.

A total of 84 classes, with an expected total of up to 100 teachers (as some classes have more than one teacher) will be included in the project.

Other school staff is recruited only to participate in the qualitative evaluation of expectations and experiences of the interventions. The selection of these will be pragmatic and theoretically lead to reaching a saturation of qualitative data, with a maximum of four extra staff from each school.

Pupils and their participation in interventions and data collection

In total, 2200 pupils will be included, with approximately 26 pupils included from each classroom. All the pupils in a class related to the included teachers/classrooms will participate in the intervention since those are all implemented in the regular operation/teaching at the school and focused on the teachers' way of teaching and managing the class and the classroom and is not possible to exclude specific pupils from. Pupils can however be included in the data collection in three different ways:

• Data from pupils to parents/custodians having signed the informed consent will have the data from their self-ratings, from the teachers ratings of them, and from registries connected to an ID-number, which in turn can be connected to their social security number (personnummer).
• Data on pupils whose parents have not filled out the consent will still be collected, but without any ID-number or other identifier, which makes it impossible to connect it to the specific pupil and hence make the data anonymous. Only the interventions the pupil has been randomized to, and at which time-point the data was collected, is connected to this data.
• Parents/custodians and the pupil himself/herself will be informed that the pupils do not need to fill out the self-ratings if they inform the teacher or researchers about their wish. Pupils will be informed of this before each self-rating.

Randomization

To ensure that the socioeconomic status (SES) of the schools do not differ too much, each school will be classified as being low, middle, or high SES and then, when there are enough schools in a cohort, randomized consecutively according to one of three corresponding block-randomized lists with block sizes varying randomly between 2, 4 and 6. The randomization will be handled by an independent third party, the Karolinska Trial Alliance, to ensure the researchers are not able to assess the lists or otherwise predict randomization results.

Interventions

The implementation procedure will be similar for each intervention, with smaller adjustments to fit specific needs of each method. Each starts with brief initial written information about the interventions, followed by one to two days of training/education for teachers. This is then followed by regular supervision face-to-face or via video/telephone depending on practical aspects and possible restrictions due to the Covid-19 pandemic. Supervision always includes some form of observation and monitoring of the use of the method by the supervisor, as a source to further inform the supervision. A total of six supervision sessions are planned: 4 sessions during the first implementation semester (held during the 1st to 4rth quarter of the semester respectively, starting with 2 individual sessions, followed by 1 group session and then by 1 individual session) and 2 sessions during the second implementation semester (held during the 1st and 2nd half of the semester respectively, 1 group session followed by 1 individual session). Teachers in all groups will also be able to receive briefer supervision on demand or additional supervision sessions initiated by the supervisor if deemed necessary.

PAX Good Behavior Game

PAX Good Behavior Game (GBG) is based on its precursor programme Good Behavior Game (GBG) that trains teachers to use principles of social learning in order to maximize the childrens' task-oriented and prosocial behaviors and minimize the occurrence of disruptive and off-task behaviors. PAX GBG is packaged into ten specific tools that the teachers are trained to introduce in their classroom management practice. The program introduces verbal and visual cues that facilitate prosocial and attentive behaviors One of the core elements of the program is the PAX-game, meaning that the children - while focusing on ordinary school tasks like math - get to play a game in teams where each team tries to make as few classroom rules infractions as possible during a certain period of time. The teams that succeed in achieving predetermined goals get a reward (often a fun and desirable activity) that serves as a reinforcer of prosocial and task-oriented behaviors.

Supervision is performed individually with each teacher (4 sessions) and in group (2 sessions) and includes that the PAX-trainer observes a lecture given by the teacher before each supervision session.

Count On Me! (Control condition)

Count On Me! is an adaptive application teaching fluency in addition and subtraction facts (i.e., math facts), equality, compose and decompose numbers and a few more basic mathematical skills taught in the first grades of schooling. Count on me! is set in adventure game where the child has an avatar that travels thru different worlds/platforms by using their mathematical skills. The child starts of in the home of the avatar, in a house like environment, where the child discovers a magical map that brings the child into a different world called Numberia. During the adventures in Numberia the child learns that it has certain skills, called the mathemagical powers, that will be used to liberate Numberia from the evil Prince Claw that has captured all the beautiful things (e.g., the birds that used to sing in the forest).

Count on me! was developed within a behavior analytic framework using a nonlinear programming process originally developed by Tiemann and Markle and later slightly revised by other researchers while creating Headsprout Early Reading and Headsprout Reading Comprehension. Instructional design elements from Tiemann and Markle, explicit instruction and the Morningside model of generative instruction were used to build the exercises. The math exercises contain at least one of three recurrent phases used in all training: the modeling, guide, and fluency phase. Modeling of new skills involves animations and hand-drawn examples along with audial instructions and requires frequent untimed student responding. Guided practice of newly modeled skills helps students achieve high levels of accuracy without timing. Finally, fluency building is aimed at developing proficiency with the skill at hand using timing. The guide and fluency phases comprise gamified elements. The application has been scientifically evaluated among Swedish low-performing elementary school pupils having shown significant improvements in math skills among the children having used the application with the largest improvement among the children with lower cognitive skills.

Prior to the implementation start, the teachers' session focus on the functionality and content of the application and the way it corresponds to the math study plan for the class they're teaching. The training session will include the methods to introduce the application in the classroom, move forward and overcome common challenges. The teachers will be encouraged to start using the application in the classroom directly after the training session. The teachers will be recommended to use the application at least 4 times each week, approximately 15 minutes each time.

Supervision will be held individually and in groups and will partly be based on the supervisors monitoring reports on how much and how the application is used.

Collegial learning community (Control condition)

Muijs and colleagues has defined five cyclical steps for professional learning for teachers. The first step is to assess the skill or knowledge gaps for students, using various data sources (for example testing, grades, surveys, student interviews). The second step is to reflect upon what skills the teacher needs to acquire in order to help the students to meet the goals. The third step is to deepen the professional knowledge and skills. The fourth step is engaging the students in the new practice, and in the fifth step to evaluate the impact the new practice has had on student outcomes.

According to Timperley and other researchers, a feasible format for this kind of learning is a Collegial learning community, where teachers take help from their peers in each of these steps in a structured way. For example, way of deepening tacit or procedural knowledge (in step two) is to visit a colleague during class and make observations on their classroom practice.

Timperley points out the importance of having a professional supervisor for maintaining the structure of the learning process. One common pitfall that the supervisor helps the teacher avoid is shifting focus from professional learning and instead ending up in entrenching current teacher practices, instead of learning new skills.

Collegial learning communities can focus on practically any kind of teacher skills. The current intervention provide material to focus on any of the following: 1) formative assessment, 2) how to formulate and follow up on homework in order to engage students 3) How to teach fact knowledge in order to facilitate higher reasoning with students. These themes are chosen to mimimize the overlap with the content of PAX GBG but still be highly relevant for teachers.

The intervention will be based on a web resource, and 6 supervision sessions with assignments in between. Supervision and observation will be combined in that the supervisor initially observes when the teachers perform a group supervision with their colleagues, and then gives immediate feedback to the whole group. This is followed by a further discussion of the method and brief, 10 to 15 minutes long individual sessions with each teacher.

Measures

Background data and the effectiveness and implementation outcomes of the intervention will be assessed with a range of different measures described below. The timing of the measures is described in table 1 above, and in some cases commented more on below.

Extra background measures at baseline (M0) for schools, classrooms and teachers The measures described here are not the only one used as background data, but those specifically measured only at baseline.

The SES for each school district measured in average income for parents and average level of education for parents (based on the data from Statistics Sweden) Number of pupils, teachers and extra resource persons in each class The pupil's need for additional help and extra resources, as well as presence of a neurodevelopmental diagnosis.

Extra COPSOQ-III questionnaire subscales Quantitative Demands, Work Pace, Influence at Work, Predictability, Social Support from Supervisor, Social Support from Colleagues, Sense of Community at Work.

Extra items from TALIS, the questions about employment and about other teachers at school.

Effectiveness measures

The effects of implementing PAX will be assessed on school level, teacher level, classroom level and student level.

Classroom-focused measures

Classroom-observations by a trained observer will be conducted during two or three lessons during the same or conceding days, for a minimum time of 20 minutes for each class. To increase standardization of the measure, only lessons where the focus is on theoretical subjects will be included. The observations will be based on an observational schedule that has already been tested in the PAX pilot trial and will be slightly adapted in a separate project prior to being implemented in the current trial. The schedule will focus on direct observation of the following aspects: 1) pupils being on-task/off-task (primary outcome), 2) number of pupils disruptive behaviors, 3) time for transition between different tasks and 4) number of positive/negative teacher-pupil interactions. The observations will be conducted by trained observers and will never identify single pupils, and hence produce anonymous data. However, the gender of the pupil will be recorded for assessment (1) - (4) to provide opportunity to conduct gender analyses.

After each observed lesson, both observers and teachers will rate the observed lecture using a questionnaire that is being developed together with the observational schedule focusing on overall ratings of aspects like severity of disruptive behaviors in the classroom, level of noise in the classroom, how focused the class was during the lecture etc. Again, this data will be rated at classroom-level, with no possibility to identify individual students.

During one week connected to the fixed measurement occasions, data on noise levels will be collected in each classroom. The noise levels will be assessed by a measuring device that will be placed in the classroom without disturbing classroom environment. The device will measure and store very short snippets of sound with random intervals of on average three seconds, making it impossible to identify source and content of the sound activity (e. g. hear a conversation) but being enough to calculate the decibel level at different ranges of frequency. Thus, this data is also anonymous.

School attendance and the need of additional personnel in the class will be measured at the class level.

The topic of the intervention's effect on the classroom, both positive and negative, will be covered by the qualitative interviews with the teachers and other school staff (described in the section for Implementation measures). Also, four free-text questions to the teacher at M3, M4 and M6 will cover this, prompting them to describe positive and negative effects of the intervention in relation to (a) themselves and (b) to the pupils, since the last measurement point. The pupils will also be asked about their attitude to PAX/control intervention.

Measures of teachers' health and work

A number of self-rated measures will be collected from teachers using a secure platform used for conducting online psychological treatments and requiring two-step authentication. The measures will include self-rated health-related measures, self-rated work-related data and registry-based data on health-care and prescription drugs consumption.

Pupil-focused measures

Self-rated questionnaires be answered by the pupils, with the aid of trained research assistants who will make sure that the students have understood the questions and the response alternatives and minimize the risk of them disturbing or influencing each other. Different ways of administering the questionnaire battery are available (administration via paper-and-pen or tablet in the whole class, in smaller groups, or individually, with the administrator giving instructions, sometimes also using a pre-recorded instruction) to be able to adapt to what would be most feasible for a specific classroom. The pupils will rate their health and wellbeing. The teachers will fill out the dysregulations scale (5 items) of the Strengths and Difficulties Questionnaire. Registry-based data will be used to analyse the pupils' health-care and prescription drugs consumption as well as the parents' need to take stay home from work to take care of a sick child. The children's academic attainment in mathematics and the Swedish language will be measured using standardized tests.

Implementation measures

The pre-conditions for the implementation process will be assessed using both quantitative and qualitative data collection. Semi-structured qualitative interviews will be conducted with randomly selected 25-50% of all the participating teachers. The goal of the interviews is to shed light on aspects that will not be captured by the standardized instruments, both in terms of the intervention's effectiveness and quality of implementation. The interview guide is being created as a part of a smaller pilot project. The interviews will be conducted by phone or video and will take approximately 20 minutes on average. The interviews will be conducted three times with each selected teacher: immediately after the PAX-training, at M2 and M4 (see the specification of the measurement points above). The interviews will be recorded, transcribed and analyzed using qualitative content analysis.

Expectations and experience of the implemented intervention

In order to study expectations, experiences and outcomes of the implemented intervention, the teachers and other school staff on several occasions will fill out the Normalisation Measure Development Questionnaire (NoMAD). The same dimensions will also be studied by by conducting interviews and collecting objective data, such as those on long-term use of the intervention according to the classroom observations, the proportion of the eligible teachers using the intervention in each school etc.

Intervention Fidelity

The teachers' fidelity to the method will be measured by direct observation conducted by the supervisor and by using a self-rated fidelity instrument for teachers to fill out.

Adverse events

Information on adverse events will be collected starting at M3 and onwards as open-ended questions that the teachers fill out in the platform. Questions on negative effects will also be included in the qualitative interviews.

Statistical Analyses

Statistical power of 80% for the pupils' psychological symptoms (the between-group effect size estimated from the pilot study equalling d=0.6) can be reached with a cluster-randomized design including 28 schools, 3 classes per school and 25 pupils in each class (a total of 2100 pupils), the expected data loss of 20% and the clustering effects known from similar studier in school environments of rho2=.1 (school level) and rho3=.25 (teacher level). Since the number of pupils can exceed 25, we expect the total number of included pupils to be 2200.

Primary endpoint for outcomes will be at the one-year follow up (M4), before the teachers starts to use also the method they were not randomised to initially. The measures focusing on the teachers' and students' health, well-being and work/study environment, as well as classroom-based measures that are collected repeatedly will be analyzed with a hierarchal general linear model with a maximum of three nested levels (school, classroom/teacher, and pupil). The difference in the slopes of change over time between the PAX group and the control interventions will be tested for each outcome variable. A set of mixed effect models will be performed to detect factors (background factors and implementation-related factors) that influence the outcomes of the implementation (fidelity, acceptability, appropriateness etc.). Mixed effect models will also be used to detect whether implementation-related variables (e g readiness for change, fidelity, acceptability etc) affect the main outcomes of the trial (e g teachers' and students' health, well-being and work/study environment).

The anonymous data from pupils will be used in two ways: (1) for missing data analyses by comparing the levels and changes in the anonymous data to the identifiable data and (2) to perform sensitivity analyses where all data are analysed on time and intervention group level only, i.e. ignoring information on school, classroom and pupil, even when those exists. The statistical methods will thus have less power, but on the other hand data from more pupils can be utilized and since the amount of missing data will be lower than in the main analyses, the risk for biases due to non-random attrition will be lower. For the anonymous data, if data from fewer than 10 pupils at one time-point and intervention are collected, the data will be discarded and deleted completely, in order to keep a safe margin from the possibility of in some way indirectly connect the data to a pupil.

Cost effectiveness

The teachers' mental health measured with GHQ-12 will be recalculated to quality adjusted life-years (QALY:s). The students health-related quality of life, measured with CHU9D, will be used to calculate QALY:s. The costs for the implementation (e g time for training and supervision), will be calculated together with the possible savings related to reduced sick leave absence for the teachers, as well as the need for additional personnel in the schools and reduced need of student health resources. The total QALY:s during the follow-up period will be studied using area under the curve method. The difference health outcome between the intervention group and the control condition will be analyzed using regression models. The costs and savings will be compared to the gains expressed in QALY:s, both for the teachers and the students. The results will be presented as a cost-effectiveness quota and illustrated as a cost-effectiveness plan, that will show the uncertainty in the results when it comes to the costs and health-effects for the intervention and the control group.

• Study Type: Interventional
• Enrollment (Estimated): 2184
• Phase: Not Applicable

Contacts and Locations

• Study Contact: Name: Viktor Kaldo, PhD; Phone Number: 0046707373859; Email: viktor.kaldo@lnu.se
• Study Contact Backup: Name: Ekaterina Ivanova, PhD; Phone Number: 0046765807416; Email: ekaterina.ivanova@ki.se

Study Locations

• Sweden
  • Stockholm, Sweden
    • Recruiting
    • Several Schools in Sweden will be recruited, although as participants and not independent sites
    • Contact: Viktor Kaldo; Email: viktor.kaldo@lnu.se

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 6 years to 10 years (Child)
• Accepts Healthy Volunteers: No

Description

ON SCHOOL LEVEL:

Inclusion Criteria:

• The school uses the general study plan (i.e. the school is not only working with students with special needs)
• There is at least one teacher that is willing to participate, signs the study consent, and will be able to start the implementation in his/her class at the earliest in the second semester in the first grade or at the latest the first semester in the third grade.
• The principal has signed an agreement to participate in the project, to allow some time for measurements to be administrated, aid in delivering information to the parents, and that the school are responsible for implementing the methods within their regular operation (given that supervision is provided by the research team)

Exclusion Criteria:

• Another method that specifically aims to improve study and classroom environment is being implemented
• Another method that is expected to be time and energy consuming for teachers is being implemented during the same period

ON TEACHER LEVEL:

Inclusion criteria:

• The teacher mentors a class of 1st or 2nd graders following the general study plan (ready to implement the program no later than the first semester of the 3rd grade) and deems the class suitable for inclusion in the study
• The teacher has signed the informed consent

Exclusion criteria:

• The teacher has expressed plans to terminate their employment during the upcoming two years
• A major change in the composition of the class is planned during the upcoming year.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Prevention
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: PAX Good Behavior Game; PAX Good Behavior Game (GBG) is based on its precursor programme Good Behavior Game (GBG) that trains teachers to use principles of social learning in order to maximize the childrens' task-oriented and prosocial behaviors and minimize the occurrence of disruptive and off-task behaviors.	Behavioral: PAX Good Behavior Game; PAX GBG is packaged into ten specific tools that the teachers are trained to introduce in in their classroom management practice. The program introduces verbal and visual cues that facilitate prosocial and attentive behaviors One of the core elements of the program is the PAX-game, meaning that the children - while focusing on ordinary school tasks like math - get to play a game in teams where each team tries to make as few classroom rules infractions as possible during a certain period of time. The teams that succeed in achieving predetermined goals get a reward (often a fun and desirable activity) that serves as a reinforcer of prosocial and task-oriented behaviors. Supervision is performed individually with each teacher and included that the PAX-trainer observes a lecture given by the teacher before each supervision session.
Active Comparator: Active control (choice between on of two active interventions); [The project has two alternative control interventions for the schools to choose between, in order to increase their engagement in the control intervention. They are hence only provided with one of these interventions, not both.] Count On Me! is an adaptive application teaching fluency in addition and subtraction facts (i.e., math facts), equality, compose and decompose numbers and a few more basic mathematical skills taught in the first grades of schooling. Collegial earning is based on five cyclical steps for professional learning for teachers.	Behavioral: Count on Me!; Count on me! is set in adventure game where the child has an avatar that travels thru different worlds/platforms by using their mathematical skills. The child starts of in the home of the avatar, in a house like environment, where the child discovers a magical map that brings the child into a different world called Numberia. During the adventures in Numberia the child learns that it has certain skills, called the mathemagical powers, that will be used to liberate Numberia from the evil Prince Claw that has captured all the beautiful things (e.g., the birds that used to sing in the forest). Count on me! was developed within a behavior analytic framework using a nonlinear programming process.; Behavioral: Collegial Learning Community; Collegial learning community is a feasible way of professional learning for teachers according to the five cyclical steps. The teachers go through the first step that is to assess the skill or knowledge gaps for students, using various data sources (for example testing, grades, surveys, student interviews). The second step is to reflect upon what skills the teacher needs to acquire in order to help the students to meet the goals. The third step is to deepen the professional knowledge and skills. The fourth step is engaging the students in the new practice, and in the fifth step to evaluate the impact the new practice has had on student outcomes. In Collegial learning community teachers take help from their peers in each of these steps in a structured way. For example, way of deepening tacit or procedural knowledge (in step two) is to visit a colleague during class and make observations on their classroom practice.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
[Pupil-focused] KIDSCREEN-10 extended with 3 items	The 10-items questionnaire for assessing children and adolescents health and well-being KIDSCREEN-10 and 7 additional items from the full KIDSCREEN-52 (self-assessment versions). A selection of 13 of these items will be summarized and used as the primary outcome on pupil level. The scores of the extended scale range between 13 and 65 with a higher score indicating higher level of health and well-being.	Up to 24 months. See Detailed Description section for the details of the measurement points.
[Pupil-focused] KIDSCREEN-10 extended Teacher-rated	N=13 items from KIDSCREEN-10 and KIDSCREEN-52 for assessing children and adolescents health and well-being (Parent versions completed by the teachers), corresponding to the 13 items rated by the pupils as the primary outcome measure. The items are selected based on relevance in the school context. The scores of the extended scale range between 13 and 65 with a higher score indicating higher level of health and well-being.	Up to 24 months. See Detailed Description section for the details of the measurement points.
[Teacher-focused] Teacher Stress Inventory (TSI) (an 8 items subscale).	TSI is a self assessment instrument measuring work-related stress in teachers. The subscale Discipline and motivation (8 items in total) will be used as the primary outcome measure. The total score of the subscale ranges between 8 and 40 with higher scores indicating more severe work-related stress in teachers.	Up to 24 months. See Detailed Description section for the details of the measurement points.
[Classroom-focused] Frequency of on-task/off-task behaviors occurring in the classroom during an observation by a research assistant.	Classroom-observations by a trained observer will be conducted during two or three lessons during the same or conceding days, for a minimum time of 20 minutes for each class according to a standardized observational schedule. Actual frequencies of on-task/off-task behaviors in the classroom, unrelated to the person performing the behavior, will be counted.	Up to 24 months. See Detailed Description section for the details of the measurement points.

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
[Pupil-focused] Child Health Utilities - 9 dimensions (CHU9D)	CHU9D is a self-rated health assessment and will be used in cost-effectiveness analysis. The instrument covers nine dimensions (worried, sad, pain, tired, annoyed, schoolwork/homework, sleep, daily routine, and activities) with the related statements corresponding to five levels of severity of impairment in each dimension ranging from no impairment at to absolute impairment.	At baseline, then after 12 and 24 months. See Detailed Description section for the details of the measurement points.
[Pupil-focused] KIDSCREEN-10	KIDSCREEN-10 is a 10-items questionnaire for assessing children and adolescents health and well-being.The scores range between 10 and 50 with a higher score indicating higher level of health and well-being.	At baseline, then after 6, 12 and 24 months. See Detailed Description section for the details of the measurement points.
[Pupil-focused, administered by the teacher] Strengths and Difficulties Questionnaire (SDQ) [the 5-items dysregulation profile]	SDQ is an assessment of children and adolescents' psychiatric health. The 5-item dysregulation profile is the selection of the items that have strongest association with cognitive, behavioral and affective difficulties. The scale will be administered by the class teacher who will fill out the questionnaire for each pupil individually. Each item is represented by a question with 3 response alternatives from "Don't agree" to "Fully agree" and the questionnaire is scored in accordance to a standardazed algorithm.	Up to 24 months. See Detailed Description section for the details of the measurement points.
[Pupil-focused, administered by the teacher] Strengths and Difficulties Questionnaire (SDQ) [the 15-items dysregulation profile]	SDQ is an assessment of children and adolescents' psychiatric health. The 15-item dysregulation profile is the selection of the items that have a strong association with cognitive, behavioral and affective difficulties. The scale will be administered by the class teacher who will fill out the questionnaire for each pupil individually. Each item is represented by a question with 3 response alternatives from "Don't agree" to "Fully agree" and the questionnaire is scored in accordance to a standardazed algorithm.	At baseline, then after 12 and 24 months. See Detailed Description section for the details of the measurement points.
[Pupil-focused] School absence	School absence measured in hours as reported by the school.	A total measure for comparison between the randomization groups 12 months after baseline, and for comparison to a matched control group 24 months after baseline.
[Pupil-focused] The parents' need to stay home from work in order to take care of a sick child	The total of hours of staying at home for care of a sick child as reported by Longitudinal Integrated Database for Health Insurance and Labour Market Studies (LISA) held by Statistics Sweden.	A total measure for comparison between the randomization groups 12 months after baseline, and for comparison to a matched control group 24 months after baseline.
[Pupil-focused] Number of primary health contacts	Total number of health-care contacts as reported by regional primary health care databases in regions where such databases exist, for example Stockholm Regional Health Care Data Warehouse (VAL), Skåne Healtcare Register (SHR) and Region Västra Götaland VEGA database. Specialist health care consumption will be analyzed using data from the National Patient Register held by the National Board for Health and Welfare.	A total measure for comparison between the randomization groups 12 months after baseline, and for comparison to a matched control group 24 months after baseline.
[Pupil-focused] Neurodevelopmental diagnoses	Total number of neurodevelopmental diagnoses confirmed by health care providers during the study's measurement periods as reported by the National Patient Register held by the National Board for Health and Welfare.	A total measure for comparison between the randomization groups 12 months after baseline, and for comparison to a matched control group 24 months after baseline.
[Pupil-focused] Heidelberger-Rechen Test 4 dimensions (HRT-4)	HRT-4 measures math skills by providing math tasks and is scored by counting the correct answers.	At baseline, then after 12 and 24 months. See Detailed Description section for the details of the measurement points.
[Pupil-focused] LegiLexi	LegiLexi measures Swedish language skills by providing tasks in the Swedish language and is scored by counting the correctly solved tasks.	At baseline, then after 12 and 24 months. See Detailed Description section for the details of the measurement points.
[Teacher-focused] Teacher Stress Inventory (TSI) (a 12 items subscale).	TSI is a self assessment instrument measuring work-related stress in teachers. Following subscales will be used in the current trial: Discipline and motivation, Behavioral Manifestations, Work-related stressor (16 items in total). The total score of the combination of the used subscales ranges between 16 and 80 with higher scores indicating more severe work-related stress in teachers.	Up to 24 months. See Detailed Description section for the details of the measurement points.
[Teacher-focused] The 7-item Stress subscale from the Depression, Anxiety ans Stress Scale (DASS)	The 7-item Stress subscale of the DASS-questionnaire consists of seven statements about general stress during the past week with four response alternatives for each statement ranging between "Did not apply to me at all" and "Applied to me very much or most of the time" and scored from 0 to 3 respectively. The total score ranges between 0 and 21 and higher score indicates more severe stress symptoms.	Up to 24 months. See Detailed Description section for the details of the measurement points.
[Teacher-focused] Single Item Stress	Single Item Stress is the question "How stressful is your job" scored on a 10-points Likert scale from Not at all stressful to Very stressful with a higher score indicating higher levels of stress.	Up to 24 months. See Detailed Description section for the details of the measurement points.
[Teacher-focused] Self-Rated Health-1	The question "In general, would you say that your health is excellent, very good, good, fair, or poor?" where the different answers are treated as an ordinal scale.	Up to 24 months. See Detailed Description section for the details of the measurement points.
[Teacher-focused] Insomnia Severity Index (ISI)	ISI is a self-rated instrument screening for symptoms of insomnia. It consists of 7 items with 5-step response scale scored between 0-4. The total score ranges from 0-28 with higher scores indicating more severe insomnia symptoms.	At baseline, then after 6, 12 and 24 months. See Detailed Description section for the details of the measurement points.
[Teacher-focused] General Health Questionnaire - 12 items (GHQ-12)	GHQ-12 is a self-rated instrument screening for general symptoms of mental illness. It consists of 12 items with 4-step response scale scored between 0-3. The total score ranges from 0-36 with higher scores indicating more severe symptoms of mental disorders.	At baseline, then after 12 and 24 months. See Detailed Description section for the details of the measurement points.
[Teacher-focused] Treatment Inventory for Costs in Psychiatric Patients (TIC-P)	TIC-P is constructed to measure self-rated health care consumption, primarily for heath-economic evaluations. The scale is adapted for the specific study and is scored according to a specific algorithm.	At baseline, then after 12 and 24 months. See Detailed Description section for the details of the measurement points.
[Teacher-focused] Copenhagen Psychosocial Questionnaire Version III (COPSOQ-III)	COPSOQ-III is a questionnaire the measures core aspects of organizational and social work environment. The subscales Meaning of Work, Work Life Conflict, Self Rated Health, Quality of Work, Work Engagement, Emotional Demands, Quantitative Demands will be used in the current project. The subscales are scored individually and the score for each subscale can range between 0-100, with higher score indicating more severe problems in the work environment.	At baseline, then after 12 and 24 months. See Detailed Description section for the details of the measurement points.
[Teacher-focused] Eighteen items from The Teaching and Learning International Survey (TALIS)	The 18 selected items focus on the attitude to the profession and include statements where the respondents need to indicate how much they agree with the statement using a four-step likert scale (from not at all agree to fully agree). In Sweden the results are usually presented item by item and not composed to a full scale score.	At baseline, then after 12 and 24 months. See Detailed Description section for the details of the measurement points.
[Teacher-focused] Work Ability Score-1	The Work Ability score (WAS) is a single-item self-assessment of the current ability compared to the lifetime best. It ranges from 0 to 10, namely: poor (0-5 points), moderate (6, 7), good (8, 9), excellent (10).	Up to 24 months. See Detailed Description section for the details of the measurement points.
[Teacher-focused] Teacher Self-Efficacy Scale (TSES)	The four items focus on the teachers' self-efficacy and include statements where the respondents need to indicate how much they agree with the statement using a four-step likert scale (from not at all agree to fully agree). The responses are scored from 0 to 3 and a higher score indicates higher self-efficacy.	Up to 24 months. See Detailed Description section for the details of the measurement points.
[Teacher-focused] Additional questions on the teachers' attitudes towards the profession and their inclination to continue working as teachers	A sample of questions differing in format and not composing a scale focusing on the teachers attitudes towards the profession and their inclination to continue working as teachers	At baseline, then after 6, 12 and 24 months. See Detailed Description section for the details of the measurement points.
[Teacher-focused] Data on short-term sick-leaves	Data on short-term sick-leaves for the involved teachers will be collected from the schools.	Total sick-leaves between the pre-measure and primary end-point 12 months after the start of the implementation. Long-term follow-up measures after 24 months.
[Teacher-focused] Health care consumption	Primary health care consumption for the participating teachers consist of number of days on sick leave, prescriptive and bought medications, and numbers, length, and types of visits in primary care and specialized care. Data will be collected from Longitudinal Integrated Database for Health Insurance and Labour Market Studies (LISA) held by Statistics Sweden, data from the Swedish Social Insurance Agency and regional primary health care databases in regions where such databases exist, for example Stockholm Regional Health Care Data Warehouse (VAL), Skåne Healtcare Register (SHR) and Region Västra Götaland VEGA database. Specialist health care consumption will be analyzed using data from the National Patient Register held by National Board for Health and Welfare.	Total health care consumption between the pre-measure and primary end-point 2 school semesters after the start of the implementation. Long-term follow-up measures after 4 semesters
[Teacher-focused] Consumption of prescription drugs	Consumption of prescription drugs will be analyzed using data from the Register over Prescription Drugs by the National Board for Health and Welfare.	Total consumption of prescription drugs between the pre-measure and primary end-point 2 school semesters after the start of the implementation. Long-term follow-up measures after 4 semesters
[Classroom-focused] Noise-levels in the classroom	During one week connected to the fixed measurement occasions, data on noise levels will be collected in each classroom. The noise levels will be assessed by a measuring device that will be placed in the classroom without disturbing classroom environment. The device will measure and store very short snippets of sound with random intervals of on average three seconds, making it impossible to identify source and content of the sound activity (e. g. hear a conversation) but being enough to calculate the decibel level at different ranges of frequency.	Up to 24 months. See Detailed Description section for the details of the measurement points.
[Classroom-focused] Frequency of disruptive behaviors, positive/negative interactions and time for transition from one task to another occurring in the classroom during an observation by a research assistant.	Classroom-observations by a trained observer will be conducted during two or three lessons during the same or conceding days, for a minimum time of 20 minutes for each class according to a standardized observational schedule. Actual frequencies of disruptive behaviors, transition times and positive/negative interactions between the teacher and the pupil in the classroom, unrelated to the person performing the behavior, will be counted and measures.	Up to 24 months. See Detailed Description section for the details of the measurement points.
[Teacher-focused] Stress, demands and influence - 4 items	Four questions on Stress, demands and influence selected from a survey conducted by the Swedish Work Environment Authority. The items are collected in order to compare the study population to the population of the the authority's survey.	At baseline, then after 12 and 24 months. See Detailed Description section for the details of the measurement points.
[Implementation measure] The Normalisation MeAsure Development questionnaire - Swedish version (S-NOMAD)	The Normalisation MeAsure Development questionnaire (NoMAD) is a set of 23 survey items for assessing implementation processes from the perspective of professionals directly involved in the work of implementing complex interventions in healthcare.	3-4, 6, 12 and 24 months after the baseline. See Detailed Description section for the details of the measurement points.
[Implementation measure] Teachers' fidelity to the intervention - observer-rated	The observer-rated intervention fidelity will be rated during classroom work using an instrument covering the quality of use of each PAX- tool, with several items addressing the same tool. Each item can be scored between 0 and 2 with a higher score indicating higher quality of the implementation. The total number of items is 71. The observation is conducted by the supervisor.	The observations will be conducted prior to the supervision sessions 1-6 (see Design -> Interventions for the time-frames).
[Implementation measure] Teachers' fidelity to the intervention - self-rated	The self-rated teacher intervention fidelity will address the week preceding the measurement point using an instrument covering the quality of use of each PAX- tool, with several items addressing the same tool. Each item can be scored between 0 and 2 with a higher score indicating higher quality of the implementation. The total number of items is 71.	The instrument will be filled out prior to the supervision sessions 1-6 (see Design -> Interventions for the time-frames).
[Implementation measure] Pupils' attitudes towards and acceptability of the method	Pupils' attitudes towards and acceptability of the method will be measured by a set of questions about the the pupils's general attitude to PAX- or the control intervention specific phenomena mixed with questions on the attitudes towards other school-specific phenomena (like school lunch, gym lessons, breaks etc) for a reference.	8-9, 12 and 24 months after the baseline. See Detailed Description section for the details of the measurement
[Implementation measure] The rate of use of PAX-tools: teacher self-rated	Eight yes/no questions on the use of PAX-tools during the period a research assistant observe the classroom, but rated by the teacher after the observed class.	Up to 24 months. See Detailed Description section for the details of the measurement points.
[Implementation measure] The rate of use of PAX-tools: observer-rated	Eight yes/no questions on the use of PAX-tools during the classroom observation conducted by a research assistant. (The questions are answered by the research assistants after the observed class.)	Up to 24 months. See Detailed Description section for the details of the measurement points.
[Implementation measure] Six questions on the use of the PAX-game (only intervention arm): teacher-rated	Six questions on the extend of the use of PAX-game during the two weeks prior to the measurement point.	At M3 through M6 (see Design section for the description of the measurement points).
[Implementation measure] Six questions on the use of Count on Me! (only Count on Me!-arm): teacher-rated	Three questions on the extend of the use of Count on Me! during the two weeks prior to the measurement point.	Starting at 3-4 months after baseline and up to 24 months (see Design section for the description of the measurement points).
[Implementation measure] Social validity of PAX/control: teacher-rated	Nine questions on perceived consequences of using PAX/control in the classroom with 7-steps likert-scale response alternatives.	Starting at 3-4 months after baseline and up to 24 months (see Design section for the description of the measurement points).
[Implementation measure] Positive and negative effects of PAX/control: teacher-rated	Four open-ended questions on positive and negative effects of PAX/control on teachers and pupils.	8-9, 12 and 24 months after the baseline. See Detailed Description section for the details of the measurement
[Implementation measure] Social validity of PAX/control: pupil-rated	18 questions (5 in the control arm) on the pupil's attitude towards PAX/control and other school-related activities for comparison with 5-steps likert-scale response alternatives.	Starting at 3-4 months after baseline and up to 24 months (see Design section for the description of the measurement points).
[Implementation measure] Social validity of PAX/control: pupil-focused teacher-rated	18 questions (5 in the Count on Me!-arm) on the pupil's attitude towards PAX/control and other school-related activities for comparison rated by the teacher with 5-steps likert-scale response alternatives.	Starting at 3-4 months after baseline and up to 24 months (see Design section for the description of the measurement points).
[Negative effects] Questions on negative effects of PAX/control: teacher-reported	Seven questions (six in the case of control group) on the number of occurrences of different adverse effects (e g "How many times a group of pupils having lost a PAX-game was teased by other pupils?")	Starting at 3-4 months after baseline and up to 24 months (see Design section for the description of the measurement points).
[Implementation measure] Training assessment (PAX/control): teacher-rated	A number of questions not collected into a scale about the teachers' perception of PAX/control training as well as their possible worries about the implementation before the implementation as well as their perception of the results of the implementation after the implementation is completed.	Right after the training as well as 12 and 24 months after baseline (see Design section for the description of the measurement points).

Collaborators and Investigators

• Sponsor: Linnaeus University
• Collaborators: Karolinska Institutet; Uppsala University; Stockholm University; KTH Royal Institute of Technology; Vestigio (NGO)
• Investigators: Principal Investigator: Viktor Kaldo, PhD, Linnaeus University

Study record dates

Study Major Dates

• Study Start (Actual): April 21, 2021
• Primary Completion (Estimated): December 15, 2025
• Study Completion (Estimated): December 15, 2026

Study Registration Dates

• First Submitted: April 23, 2021
• First Submitted That Met QC Criteria: June 1, 2022
• First Posted (Actual): June 7, 2022

Study Record Updates

• Last Update Posted (Actual): February 15, 2024
• Last Update Submitted That Met QC Criteria: February 13, 2024
• Last Verified: February 1, 2024

More Information

Terms related to this study

• Keywords: Prevention; Mental health; Learning; Work-related stress; Psychology; School-based research; Leadership; Pedagogy; Pedagogical leadership; Work environment; Qualitative reserach; Quantitative research; Good classroom environment; Math skills
• Additional Relevant MeSH Terms: Behavioral Symptoms; Stress, Psychological; Occupational Diseases; Occupational Stress
• Other Study ID Numbers: 2020-03848

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No