Evaluating Implementation and Impact of PeerOnCall, a Mobile Health Approach to Peer Support for Canadian Public Safety Personnel

April 8, 2024 updated by: McMaster University

Advancing Peer Support Programming to Address PTSD and Trauma Among Canadian Public Safety Personnel and Veterans

A prospective cohort study design will be implemented across four sectors and multiple organizations sites to evaluate implementation and impact of the apps. Pre/post impact data will be collected over a six-month implementation period to assess whether the apps lead to increased Public Safety Personnel outreach to peer support (primary outcome), and to increased mental health literacy, and decreased mental distress (secondary outcomes). Recruitment will take place at three levels; 1) organizations (including knowledge users/organization champions), 2) peer support providers, and 3) public safety personnel employed within each participating organization. Sources of implementation and impact data for the six-month trial will include: 1) app utilization data, 2) knowledge user/organization champion interviews, 3) surveys with public safety personnel, 4) focus groups with peer support providers, and 5) implementation costs. Analysis will focus on both implementation and effectiveness to address the research questions related to the impact of the apps, as well as how the process and context of implementation shapes effectiveness.

Study Overview

Status

Active, not recruiting

Intervention / Treatment

Detailed Description

PeerOnCall was developed to support frontline PSP and includes features such as anonymous calling or messaging to a choice of peer support providers, access to self-screening tools, a personal wellness toolbox, and "tips to cope" resources. PeerOnCall Support is designed for peer support providers to provide a secure, private platform for providing support to peers who reach out via text or telephone through the PeerOnCall app.

This study will explicitly scale implementation and evaluation of these apps in the context of day-to-day work within a range of PSP organizations across four different service sectors (corrections, emergency communicators, fire services and paramedic services).

The implementation period for the app is six-months. Prior to the end of the six-month implementation period, the sector lead will meet with each organization to review app utilization data and to discuss options for continued usage of the app within the organization.

The Consolidated Framework for Implementation Research (CFIR) will inform exploration, not only of whether the apps 'work', but also about how the apps are adopted, and the individual and organizational forces that shape adoption.

The study will deploy the apps to a purposive sample of approximately 24 organizations across four sectors: 1) correctional workers; 2) emergency communicators (e.g., 911 call takers, dispatchers); 3) fire services; and 4) paramedic services. All participating organizations must have an established internal peer support service (with procedures for vetting, training, and ongoing support of the peer support providers), an organization champion who will liaise between the research team and employees, and support from leadership to participate in the study. The process of recruitment will be facilitated by sector leads and knowledge users who have extensive networks within the sectors. The number of organizations recruited across each sector will be different, recognizing differences in how services are provided, and the need for understanding breadth versus depth of implementation in key stakeholder groups.

Within each sector, key knowledge users (at the sector level) and organization champions (at the organization level) will be recruited. The sector-level knowledge users will provide insights into general considerations for implementation and evaluation. In the participating organizations, one or two "champions" will be recruited who will be responsible for organization onboarding, including customizing content, securing participation of in-house peer support providers, and sharing information with co-workers via formal and informal communication channels.

Peer support providers within each organization will be recruited to provide services through the app. All trained peer providers within each organization will be invited to participate. Information meetings will be held in each organization for the peer supporters regarding the PeerOnCall Support app. The orientation will ensure that the peer supporteres are aware of how to use the app and the protocols for responding to PSP who reach out for support, including those who may be in crisis.

All eligible employees within each organization will be invited to participate in the research via established communication channels within the organization (e.g., recruitment posters distributed via email, social media etc.), and information meetings held in the organization in the month prior to distributing access to the PeerOnCall app. After orientation within the organization, employees will be provided with an organization code to access the PeerOnCall app once it has been downloaded from the Google Play or Apple store. Employees can choose whether to download the app and they are able to use the app without engaging in the baseline and follow-up surveys.

To ensure adequate power for evaluating impact of the app, sample size estimates were based on the primary outcome of interest: the frequency of outreach to peer support through the app (count of the number of times contact was initiated by PeerOnCall users via phone and/or text). The sample size was calculated using a one-sided Poisson test since the frequency of outreach via the app can only increase from no outreach at baseline. Assuming 90% power, a significance level of 0.025 for a one-sided test, a hypothesized rate of 0.2 (20% increase in rates of outreach) and alternative hypothesis rate of 0.1 (thus a difference of 0.1), 161 participants will be required. Considering an estimated 20% drop out rate, 202 participants will be needed.

Aggregate data collected will include participant adoptions rates across each organization (e.g., the number of times the app was downloaded) as well as aggregate data on how often the app was used, average durations of use, the number of times different features were accessed, and the number and duration of contacts with peer supporters.

Knowledge user/Organization Champion Interviews will be conducted at baseline, three-months and six-months after app implementation begins at each participating organization to explore the context for implementation. Interview questions will be congruent with the Consolidated Framework for Implementation Research. These data will be used to identify facilitators and barriers to app use and inform interpretation of the adoption rates and outcome data both within and across participating organizations.

To track immediate feedback about the peer support service, a 'pop-up' feedback survey is presented to users within the application following each peer support encounter asking for a rating of their level of distress before and after the interaction using a 10-point Visual Analogue Scale. There is also an option to provide comments about the encounter. Data from the feedback surveys will help understand the immediate impact of peer support on level of distress.

To evaluate user impact over time, employees in each organization will be invited to complete baseline, three-month, and six-month follow-up surveys via a link to complete a survey.

Focus groups will be scheduled at three-months and six months after the implementation period with PeerOnCall Support peer support providers to explore their experiences with providing peer support through the app. Three sets of focus groups will be conducted within each sector with six to eight participants in each group.

Data regarding implementation costs within each organization will include: the organization champion's time invested on onboarding and communication, administrative time for uploading users and responding to requests, and metrics regarding peer support service provision. The cost of server hosting, troubleshooting, and regular upgrades to the apps will also be tracked across sites.

Implementation within each organization will be tracked via descriptive data regarding adoption rates (proportion of employees who download & use the app), and usage rates over time, including the number of times users reach out for peer support via phone/text, and the number of repeated calls.

To evaluate effectiveness, analysis of the pre/post survey data will examine the change in scores across the dataset in dimensions of mental health literacy, levels of mental distress (standardized assessments of anxiety, depression, and Post Traumatic Stress Disorder), and frequency of outreach behaviors in seeking mental health support. Multiple hierarchical linear regression analyses will be conducted to examine differences in effectiveness based on gender, service, and job tenure.

Effectiveness of the peer support service will also be evaluated by comparing change scores in level of distress before and after each peer support encounter via the in-app surveys.

Identification of barriers and facilitators to adoption of the technology and peer support service will occur through qualitative content analysis of the interview transcripts with the organizational champions and focus groups with the peer supporters. The CFIR framework will guide the analysis process considering organization internal and external forces, as well as how communication strategies influence implementation, and consistency in providing the peer support service.

All the data from the evaluation process will be reviewed to generate recommendations for optimizing uptake and impact of the app across different types of organizations.

Study Type

Observational

Enrollment (Actual)

950

Contacts and Locations

This section provides the contact details for those conducting the study, and information on where this study is being conducted.

Study Contact

Study Contact Backup

Study Locations

    • Newfoundland and Labrador
      • Saint John's, Newfoundland and Labrador, Canada
        • Memorial University of Newfoundland
    • Ontario
      • London, Ontario, Canada, N6A 3K7
        • Western University
      • Waterloo, Ontario, Canada, N2L 3C5
        • Wilfrid Laurier University

Participation Criteria

Researchers look for people who fit a certain description, called eligibility criteria. Some examples of these criteria are a person's general health condition or prior treatments.

Eligibility Criteria

Ages Eligible for Study

18 years to 70 years (Adult, Older Adult)

Accepts Healthy Volunteers

Yes

Sampling Method

Non-Probability Sample

Study Population

Individuals employed by participating Public Safety Organizations (fire services, paramedic services, emergency communications, correctional services).

Description

Inclusion Criteria:

  • Public Safety Personnel working in fire services, paramedic services, emergency communications and correctional services
  • Employed (full or part-time) by the participating organizations
  • Agree to download the PeerOnCall app on their mobile phone (Android or Apple)

Exclusion Criteria:

- Not able to access the app on their mobile health device

Study Plan

This section provides details of the study plan, including how the study is designed and what the study is measuring.

How is the study designed?

Design Details

What is the study measuring?

Primary Outcome Measures

Outcome Measure
Measure Description
Time Frame
Accessing peer support via phone or text via the PeerOnCall app (Frequency count)
Time Frame: Monthly intervals from 1 to 6 months.
Number of times PeerOnCall users within an organization create a connection with peer support providers via the app platform (Software analytics)
Monthly intervals from 1 to 6 months.

Secondary Outcome Measures

Outcome Measure
Measure Description
Time Frame
Change in Mental Health Literacy (6 items, 7-point Likert scale rating)
Time Frame: Baseline, 3 months, 6 months
Rating using a 7-point Likert scale of perceived improvement in dimensions of mental health literacy (Promote mental health, knowledge of how to identify and address mental health issues in self and others, how to identify relevant resources). Range 7-42 with higher scores indicating higher levels of perceived literacy.
Baseline, 3 months, 6 months
Changes in symptoms of anxiety (GAD-7 screening tool)
Time Frame: Baseline, 3 months, 6 months
The Generalised Anxiety Disorder (GAD-7) scale is a 7-item tool to evaluate anxiety symptoms over the past two weeks (4-point Likert scale ratings). Scores ranges from 0-21 with a score of 10 or more indicative of potential generalized anxiety disorder.
Baseline, 3 months, 6 months
Change in symptoms of depression (PHQ-9 screening tool)
Time Frame: Baseline, 3 months, 6 months
The Patient Health Questionnaire (PHQ-9) is a 9-item screening tool for depression. Scores range from 0-27, with a score of 10 or more indicative of major depression.
Baseline, 3 months, 6 months
Change in symptoms of trauma (PTSD Checklist, PCL-5 short version)
Time Frame: Baseline, 3 months, 6 months
The short version of the Post-Traumatic Stress Disorder Checklist (PCL-5) is a four-item PTSD screening checklist exploring responses to a stressful experience in terms of symptoms over the past month. Scores range from 0-16 with higher scores indicating more severe impairment.
Baseline, 3 months, 6 months
Change in help-seeking behaviour
Time Frame: Baseline, 3 months, 6 months
Rating of how likely to reach out for support related to four different types of support (In-person peer support, app-based peer support, professional support, and employee assistance program). Ratings on a 5-point scale from not at all likely to extremely likely.
Baseline, 3 months, 6 months
Perceived value of the app
Time Frame: 3 months, 6 months
Questions about perceived impact adapted from the Mobile Rating Application Rating Scale (MARS) asks users to rate (on a 5-point Likert scale) the perceived value of the app on 7 items (Awareness of the importance of addressing mental health, reaching out for help and impact on mental health, perceived support, and ability to cope at work).
3 months, 6 months
Change in perceived level of stress
Time Frame: Baseline, 3 months, 6 months
Rating of current level of stress on a 7-point Likert scale from very low to very high.
Baseline, 3 months, 6 months
Change in perceptions of workplace psychological health and safety climate
Time Frame: Baseline, 3 months, 6 months
Rating of psychological health and safety climate in the workplace on a 7-point Likert scale from toxic to excellent.
Baseline, 3 months, 6 months
Change in current mental health challenges
Time Frame: Baseline, 3 months, 6 months
Rating of current mental health challenges on a 100-point Visual Analogue scale from ill to healthy.
Baseline, 3 months, 6 months
Impact on work
Time Frame: Baseline, 3 months, 6 months
Questions about the number of hours users were scheduled/expected to work and the number of hours of work users missed because of the way they were feeling over the past 2 weeks.
Baseline, 3 months, 6 months

Other Outcome Measures

Outcome Measure
Measure Description
Time Frame
Digital literacy
Time Frame: Baseline, 3 months, 6 months
Rating of knowledge and comfort level in using apps. Ratings from low to high levels of digital literacy. Hypothesis is that lower digital literacy will predict lower app use. The baseline scale provides descriptive data about levels of digital literacy across the organization. The follow-up scale provides descriptive data about perceived digital literacy.
Baseline, 3 months, 6 months
Frequency of app use
Time Frame: 3 months, 6 months
Report of how often the app was used over the past 4 weeks, ranging from not at all to more than once a day. This data can be used to note dose/response (I.e., Higher app use might lead to greater impact) and to track app user over time.
3 months, 6 months
Timing of app use
Time Frame: 3 months, 6 months
Descriptive data of when the app is used the most. Can be used to report trends within/across sectors and organizations. This can also be used to cross-check with/against app utilization metrics.
3 months, 6 months
Rating of app
Time Frame: 3 months, 6 months
Users are asked to rate the value of 8 different features of the app on a 5-point Likert scale from very poor to excellent. Plus, an overall rating of the likelihood of continued use of the app on a 5-point Likert scale from definitely not to definitely.
3 months, 6 months

Collaborators and Investigators

This is where you will find people and organizations involved with this study.

Publications and helpful links

The person responsible for entering information about the study voluntarily provides these publications. These may be about anything related to the study.

General Publications

Study record dates

These dates track the progress of study record and summary results submissions to ClinicalTrials.gov. Study records and reported results are reviewed by the National Library of Medicine (NLM) to make sure they meet specific quality control standards before being posted on the public website.

Study Major Dates

Study Start (Actual)

May 2, 2023

Primary Completion (Actual)

March 31, 2024

Study Completion (Estimated)

April 1, 2024

Study Registration Dates

First Submitted

June 19, 2022

First Submitted That Met QC Criteria

June 19, 2022

First Posted (Actual)

June 23, 2022

Study Record Updates

Last Update Posted (Actual)

April 9, 2024

Last Update Submitted That Met QC Criteria

April 8, 2024

Last Verified

April 1, 2024

More Information

Terms related to this study

Plan for Individual participant data (IPD)

Plan to Share Individual Participant Data (IPD)?

UNDECIDED

IPD Plan Description

May share aggregate utilisation data without personal identifiers.

Drug and device information, study documents

Studies a U.S. FDA-regulated drug product

No

Studies a U.S. FDA-regulated device product

No

This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.

Clinical Trials on Post Traumatic Stress Disorder

3
Subscribe