Effectiveness and Implementation of eScreening in Post 9/11 Transition Programs

Electronic screening is effective for timely detection of, and intervention for, suicidal ideation and other mental health symptoms. The VA eScreening program is a patient self-report electronic screening system that has shown promise for the efficient and effective collection of mental and physical health information among Veterans. However, additional effectiveness and implementation research is warranted to evaluate the impact of eScreening within VHA. This study addresses questions of the impact of eScreening compared to screening as usual, while evaluating a multi-component implementation strategy (MCIS) for optimal enterprise rollout of eScreening in VA Transition Care Management clinics.

Study Overview

• Status: Completed
• Conditions: Depression; Stress Disorders, Post-Traumatic; Suicide
• Intervention / Treatment: Other: Screening As Usual; Other: eScreening

Detailed Description

This mixed methods hybrid 2, effectiveness-implementation, stepped-wedge (SW) trial evaluates the effectiveness and implementation of an electronic screening program called eScreening compared to standard of care paper and/or verbal screening methods in VHA Transition Care Management (TCM) programs. eScreening is a VHA program that allows Veterans to answer self-report screening questions via an iPad connected to the VHA secure Wi-Fi. eScreening reads from and writes to the VHA EMR. The highlighted features of eScreening include: 1) the ability for Veterans to enter screening information directly without the involvement of a clinician; 2) immediate scoring of measures; 3) an editable note generated in the EMR; and 4) clinician alerts for positive mental health screens that require follow-up for suicide risk.

Aim 1 involved evaluating eScreening, compared to paper and verbal screening (Screening As Usual), guided by the RE-AIM outcomes of PRISM in 8 VHA Transition Care Management (TCM) programs, using a cluster randomized, stepped wedge design. This stepped-wedge trial relied on sequential roll-out of eScreening to participating sites over time, while using other sites as controls until they began implementation. The 8 participating sites were stratified by size (a combination of number of TCM staff and average number of post-9/11 Veterans enrolled per month) and block randomized to four step/crossover cohorts of two sites each. All step/crossover cohorts went through a 3-month pre-implementation phase which served as the Screening As Usual comparison condition, as eScreening was not yet being implemented. This phase was followed by a 9-month active implementation using our eScreening MCIS. The eScreening MCIS will began with a 3-month period that included eScreening software provision, training, RPIW, and blended facilitation followed by 6 months of ongoing blended facilitation (9 months total). After the active implementation, all sites had a 9-month sustainment period.

Below is a detailed description of each phase:

Pre-implementation phase (Screening As Usual condition): lasted 3 months during which the research team worked with the internal facilitators to: 1) gather pre-implementation information including detailed information on the processes in place for TCM screening upon enrollment ; 2) identify points of contact for iPads and other logistical needs; 3) establish communication with TCM staff and others working with the TCM staff; 4) recruit staff participants for the study; and 5) begin ongoing tracking of process data. TCM staff names, clinic names, note titles, scheduling the RPIW, and clinical reminders completed by program staff were gathered and used for subsequent development of user accounts and content customization during the implementation phase. The implementation team also provided psychoeducation to the staff on the importance of screening. During pre-implementation, TCM teams continued usual screening procedures. These involve interview or self-report, paper-based collection of post-9/11 screening measures, including the system-wide standardized assessments of depression, PTSD and alcohol use (PHQ-2, PCPTSD, and Audit-C, respectively). Patients who score positive on the PHQ-2 and PCPTSD were then administered the Columbia Suicide Severity Rating Scale (C-SSRS), which collects more information regarding risk of suicide. Veterans who are positive on C-SSRS then receive a Comprehensive Suicide Risk Evaluation and are referred for appropriate follow-up. A detailed description and flow map of the current screening process at each site were developed by the external facilitator (research team) and internal facilitator (site staff) with information from the TCM staff prior to the implementation phase.

eScreening Implementation phase: during this 9 month phase, the MCIS was used to implement eScreening. This strategy consisted of eScreening software provision, user training, RPIW, and facilitation. Virtual and asynchronous eScreening training was provided and included a 1-hour instructional PowerPoint presented by the external facilitator. The PowerPoint was followed by an hour of tutorial videos showing all steps of (creating assessments, adding Veterans, saving to VistA, searching for assessments, creating scheduled appointment assessments, accessing reports). Hands-on training for users was available in group format or individually by the training staff as requested by the TCM site staff. Additional training materials could be accessed via the eScreening Pulse site which include a series of quick guides to address eScreening customization, assessment set-up and dashboard use. Frequently asked questions were also available on the Pulse site. The 3-day RPIW was facilitated virtually by the external facilitator with assistance from the onsite internal facilitator and included the TCM team, related staff (i.e., medical support staff, clerks), and other site stakeholders. The first day of the process trained participants in the RPIW principles and introduced a summary of the information gathered in the Pre-Implementation Phase, including a graphic of the current state process map which was then refined and finalized. The second day consisted of collective efforts to map a targeted future state, conduct a gap analysis, and identify relevant factors and barriers unique to the site. The third day was dedicated to the repetition of action planning, execution, and reevaluation to finalize the targeted state and identify clinically meaningful goals for improvement. Using a Plan-Do-Study-Act (PDSA) framework, the plans to achieve the target state are enacted with a detailed plan that includes who, what, when for each step in the plan. Due to the flexibility of eScreening and the implementation strategy, each TCM program could choose to integrate eScreening into their workflow based on the specific needs of their program, available resources, and other factors. This was followed by 6 months of ongoing facilitation, in which the primary external facilitator from the eScreening team worked with the site internal facilitator to schedule meetings, training sessions, and phone calls. The external facilitator was the main point of contact for implementation-related questions.

Sustainment Phase: this was a 9-month phase that did not involve any active implementation.

Throughout all study phases, patient level data was collected from EMR (for which the investigators had a HIPPA waiver), and no Veterans directly participated. EMR data was extracted from the Corporate Data Warehouse (CDW) database. EMR data consisted of 1) the number of Veterans that enrolled in the healthcare system and the date and time they enrolled, 2) the date and time that they received PC-PTSD-5, PHQ-2, AUDIT-C, CSSR-S, and the disposition (positive/negative screen), 3) date and time they received comprehensive suicide risk evaluation (CSRE), and 4) mental health care referrals. These data were used to calculate the overall rate of screening completion and referral to mental health care and the average length of time to screening completion.

To test Aim1 - whether the likelihood of receiving mental health screening differed between eScreening and Screening As Usual (Reach aim), we conducted logistic regression analyses comparing the proportion of newly enrolled veterans who received PTSD, PHQ-9, AUDIT, and CSSRS screening during the eScreening implementation phase and the sustainment phase to the pre-implementation/Screening As Usual phase. To test whether the number of days it took to receive mental health screening differed between eScreening and Screening As Usual (Effectiveness aim), we conducted Cox regression analyses comparing the number of days from enrollment to PTSD, PHQ-9, AUDIT, and CSSRS screening among newly enrolled veterans during the eScreening implementation phase and the sustainment phase to the pre-implementation/Screening As Usual phase. Finally, to test whether the likelihood of receiving a mental health consult request differed between eScreening and Screening As Usual (Effectiveness aim), we conducted logistic regression analyses comparing the proportion of newly enrolled veterans who received PTSD, PHQ-9, AUDIT, and CSSRS screening within 30 days of enrollment and screened positive for PTSD, depression, suicidality, or substance use during the eScreening implementation phase and the sustainment phase to the pre-implementation/Screening As Usual phase.

Aim 2 of this study involved evaluating the feasibility, acceptability, and potential impact of the MCIS, guided by the RE-AIM outcomes of PRISM, adoption, implementation, and maintenance using mixed methods. In addition to the patient level data above, Transition Care Management staff at each of the 8 VHA sites who had direct involvement with the implementation of eScreening were asked to take part in this research. These participants completed a quantitative survey called the "PRSIM Contextual Survey Instrument" (PSCI) focused on assessing the feasibility and acceptability of eScreening; and factors affecting adoption, implementation, and sustained use of eScreening. The PCSI included 29 items across the six PRISM domains: Organizational Perspective, Patient Perspective, Implementation and Sustainability Infrastructure, Organizational Characteristics, Patient Characteristics, and External Environment. Items were rated on a 5-point Likert scale. Surveys were administered at three phases (pre-implementation, implementation, and sustainment). Investigators examined staff responses on the PCSI surveys by implementation phase (pre-implementation, implementation, and sustainment) using non-parametric tests.

• Study Type: Interventional
• Enrollment (Actual): 72
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • California
    • San Diego, California, United States, 92161-0002
      • VA San Diego Healthcare System, San Diego, CA

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

Staff inclusion criteria:

• Direct or indirect involvement with implementation of eScreening at the site
• Capable of informed consent

Exclusion Criteria:

Staff exclusion criteria:

• Not involved in or directly impacted by eScreening involvement at each site

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: Randomized
• Interventional Model: Crossover Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Active Comparator: Screening As Usual/Pre-implementation; During this 3 month pre-implementation period, usual screening methods were used (i.e., verbal and paper-based screening).	Other: Screening As Usual; Assessment of depression, PTSD and alcohol use using verbal and paper-based screening methods.
Experimental: eScreening Implementation; During this 9 month active implementation phase, eScreening was implemented using the MCIS.	Other: eScreening; eScreening is a clinical patient self report system that allows Veterans to complete self report screens, including assessments of depression, PTSD and alcohol use, using a secure connection from any internet connected device.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Compare Rate of PTSD Screening During eScreening Implementation and Sustainment vs. Screening as Usual/Pre-implementation	Proportion of newly enrolled Veterans who received PTSD screening (i.e., The Primary Care PTSD Screen for DSM-5 [PC-PTSD-5]) during the eScreening implementation and sustainment phases compared to the screening as usual/pre-implementation phase. For this outcome, we calculated the rate of screening (i.e., the proportion of Veterans who received the screening measure versus those who did not) during each phase, and then compared the proportion of Veterans who received screening during the eScreening implementation and sustainment phases vs. screening as usual/pre-implementation phase.	within 30 days of enrollment at the VA study site during each phase
Compare Rate of Depression (PHQ-2) Screening During eScreening Implementation and Sustainment vs. Screening as Usual/Pre-implementation	Proportion of newly enrolled Veterans who received depression screening (Patient Health Questionnaire-2; PHQ-2) during the eScreening implementation and sustainment phases compared to the screening as usual/pre-implementation phase. For this outcome, we calculated the rate of screening (i.e., the proportion of Veterans who received the screening measure versus those who did not) during each phase, and then compared the proportion of Veterans who received screening during the eScreening implementation and sustainment phases vs. screening as usual/pre-implementation phase.	within 30 days of enrollment at the VA study site during each phase
Compare Rate of Alcohol Screening (AUDIT) During eScreening Implementation and Sustainment vs. Screening as Usual/Pre-implementation	Proportion of newly enrolled Veterans who received alcohol screening (AUDIT) during the eScreening implementation and sustainment phases compared to the screening as usual/pre-implementation phase. For this outcome, we calculated the rate of screening (i.e., the proportion of Veterans who received the screening measure versus those who did not) during each phase, and then compared the proportion of Veterans who received screening during the eScreening implementation and sustainment phases vs. screening as usual/pre-implementation phase.	within 30 days of enrollment at the VA study site during each phase
Compare Rate of Suicide Screening (CSSRS) During eScreening Implementation and Sustainment vs. Screening as Usual/Pre-implementation	Proportion of newly enrolled Veterans who received suicide screening (Columbia-Suicide Severity Rating Scale; CSSRS) during the eScreening implementation and sustainment phases compared to the screening as usual/pre-implementation phase. For this outcome, we calculated the rate of screening (i.e., the proportion of Veterans who received the screening measure versus those who did not) during each phase, and then compared the proportion of Veterans who received screening during the eScreening implementation and sustainment phases vs. screening as usual/pre-implementation phase.	within 30 days of enrollment at the VA study site during each phase.
Time to PTSD Screening (PC-PTSD-5) Completion During eScreening Implementation and Sustainment vs. Screening as Usual/Pre-implementation	Number of days to receive PTSD screening (PC-PTSD-5) during the eScreening implementation and sustainment phases compared to the screening as usual/pre-implementation phase. For this outcome, we calculated the time to screening (in days) during each phase, and then compared the time to screening for the eScreening implementation and sustainment phases vs. screening as usual/pre-implementation phase.	within 30 days of enrollment at the VA study site during each phase.
Time to Depression (PHQ-2) Screening Completion During eScreening Implementation and Sustainment vs. Screening as Usual/Pre-implementation	Number of days to depression screening (PHQ-2) during the eScreening implementation and sustainment phases compared to the screening as usual/pre-implementation phase. For this outcome, we calculated the time to screening (in days) during each phase, and then compared the time to screening for the eScreening implementation and sustainment phases vs. screening as usual/pre-implementation phase.	within 30 days of enrollment at the VA study site during each phase.
Time to Alcohol Screening (AUDIT) Completion During eScreening Implementation and Sustainment vs. Screening as Usual/Pre-implementation	Number of days to alcohol screening (AUDIT) during the eScreening implementation and sustainment phases compared to the screening as usual/pre-implementation phase. For this outcome, we calculated the time to screening (in days) during each phase, and then compared the time to screening for the eScreening implementation and sustainment phases vs. screening as usual/pre-implementation phase.	within 30 days of enrollment at the VA study site during each phase.
Time to Suicide Screening (CSSRS) Completion During eScreening Implementation and Sustainment vs. Screening as Usual/Pre-implementation	Number of days to suicide screening (CSSRS) during the eScreening implementation and sustainment phases compared to the screening as usual/pre-implementation phase. For this outcome, we calculated the time to screening (in days) during each phase, and then compared the time to screening for the eScreening implementation and sustainment phases vs. screening as usual/pre-implementation phase.	within 30 days of enrollment at the VA study site during each phase.
Change in Rate of Referral to Mental Health Care Following Positive PTSD Screen (PC-PTSD-5) During eScreening Implementation and Sustainment vs. Screening as Usual/Pre-implementation.	Change in rate of referral to mental health follow up care (BHIP) following a positive PTSD screen (PC-PTSD-5) during the eScreening implementation and sustainment phases compared to the screening as usual/pre-implementation phase.	30 days from the date the screener was administered during each phase.
Change in Rate of Referral to Mental Health Care Following Positive Depression Screen (PHQ-2) During eScreening Implementation and Sustainment vs. Screening as Usual/Pre-implementation	change in rate of referral to mental health follow up care (BHIP) following a positive depression (PHQ-2) screen during the eScreening implementation and sustainment phases compared to the screening as usual/pre-implementation phase.	30 days from the date the screener was administered during each phase.
Change in Rate of Referral to Substance Use Treatment Following Positive Alcohol Screen (AUDIT) During eScreening Implementation and Sustainment vs. Screening as Usual/Pre-implementation	change in rate of referral to substance use treatment following a positive alcohol screen (AUDIT) during the eScreening implementation and sustainment phases compared to the screening as usual/pre-implementation phase.	30 days from the date the screener was administered during each phase

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
PRISM Contextual Survey Instrument (PCSI)	Staff survey responses on the Patient Perspectives domain of the PRISM Contextual Survey Instrument (PCSI). The PCSI includes 29 items rated on a 5-point Likert scale, where 1 = strongly disagree and 5= strongly agree; thus higher scores represent more positive perspectives.	Surveys were administered within 60 days of initiating each phase: pre-implementation, implementation, and sustainment

Collaborators and Investigators

• Sponsor: VA Office of Research and Development
• Investigators: Principal Investigator: James Pittman, PhD MSW, VA San Diego Healthcare System, San Diego, CA

Publications and helpful links

General Publications

• Pittman JOE, Lindamer L, Afari N, Depp C, Villodas M, Hamilton A, Kim B, Mor MK, Almklov E, Gault J, Rabin B. Implementing eScreening for suicide prevention in VA post-9/11 transition programs using a stepped-wedge, mixed-method, hybrid type-II implementation trial: a study protocol. Implement Sci Commun. 2021 Apr 29;2(1):46. doi: 10.1186/s43058-021-00142-9.

Study record dates

Study Major Dates

• Study Start (Actual): June 18, 2021
• Primary Completion (Actual): December 29, 2023
• Study Completion (Actual): December 31, 2024

Study Registration Dates

• First Submitted: August 5, 2020
• First Submitted That Met QC Criteria: August 5, 2020
• First Posted (Actual): August 10, 2020

Study Record Updates

• Last Update Posted (Actual): February 17, 2026
• Last Update Submitted That Met QC Criteria: January 28, 2026
• Last Verified: January 1, 2026

More Information

Terms related to this study

• Keywords: mental health; suicide; veterans; screening; technology; self report
• Additional Relevant MeSH Terms: Trauma and Stressor Related Disorders; Mental Disorders; Behavioral Symptoms; Self-Injurious Behavior; Stress Disorders, Traumatic; Behavior; Personal Satisfaction; Suicide; Depression; Stress Disorders, Post-Traumatic; Psychological Well-Being; Health Care Quality, Access, and Evaluation; Investigative Techniques; Epidemiologic Methods; Diagnostic Techniques and Procedures; Diagnosis; Data Collection; Health Care Evaluation Mechanisms; Quality of Health Care; Public Health; Environment and Public Health; Health Services; Health Care Facilities Workforce and Services; Preventive Health Services; Diagnostic Services; Health Surveys; Surveys and Questionnaires; Public Health Practice; Mass Screening
• Other Study ID Numbers: SDR 19-443; 1I01HX003079-01A1 (U.S. NIH Grant/Contract)

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO
• IPD Plan Description: Only data sets without individual identifiers will be generated and shared.

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No
• product manufactured in and exported from the U.S.: No