Designing a Mobile App to Support Academic Success for Student Veterans

Compared to civilian students in higher education, student Veterans have high rates of mental health disorders (e.g., Depression: 24% vs 12.1%). As a result, Veterans with mental health disorders can be more likely to experience academic issues, such as lower enrollment rates and slower degree attainment on average. Though many student Veterans could benefit from programming embedded in supported education interventions, current in-person VA supported education treatments are often difficult to access - or not available locally - for these Veterans. In addition, there is no widely available, VA-specific online or mobile-app based resource for students. The goal of this project is to develop and evaluate a comprehensive mobile app for student Veterans with mental health disorders. VetEd will provide a resource to (1) orient student Veterans with mental health disorders to successfully transition to the role of student as defined by their self-created educational roadmap, which will include helping students acquire (2) academic skills, (3) mental health management skills, and (4) up-to-date information on psychiatric, academic, and financial resources to help them successfully meet higher-education expectations.

This overall study involves three aims: 1) Developing a Veteran-centered educational support app to help student Veterans with mental health disorders to identify their perceived academic needs, app preferences, and evaluate Veteran-centered content; 2) Testing and iteratively revising the VetEd app (n =15) by assessing app software, content, human-computer interface, usability, satisfaction data, and preliminary exploration of changes in educational functioning (course activity completion, academic self-efficacy, and retention; and 3) Completing final revisions of the VetEd app for a future grant application of a larger randomized controlled trial. This pilot project is significant and innovative in three key respects: (1) it extends services based in previous, effective supported education research to address both psychiatric and academic concerns for Veterans with mental health disorders; (2) is potentially cost-effective and easy to disseminate nationally; and (3) focuses on improving Veterans' academic functioning and quality of life is substantially different than current VA mobile app offerings.

Study Overview

• Status: Completed
• Conditions: Depression; Anxiety; Substance Use Disorder; Mental Health; Post-Traumatic Stress Disorder
• Intervention / Treatment: Behavioral: VetEd Mobile Application

Detailed Description

The goals of this phase are to: 1) gain feedback on the VetEd mobile app via Veteran usability testing; 2) assess Veteran interest, satisfaction, and usability of VetEd; 3) iteratively address any technological concerns, and 4) collect preliminary information on the impact of the VetEd app on academic functioning (retention, perceived course task ability, and educational self-efficacy) and community reintegration. The investigators will conduct pilot usability testing with 15 Veterans who meet eligibility criteria, conducted in three waves. Each usability testing wave will include 5 participants, based on the finding that approximately 80% of usability problems can be detected using a sample of 4 to 5 participant per wave.

Recruitment. Participants will be 15 Veterans recruited from the Bedford VA medical center (VAMC), and local colleges. As noted in Phase 1, based on the rate of student Veterans seeking educational support services at the Bedford facility and the number of Veteran students in the community, the investigators should have no difficulty recruiting the proposed number of participants. Additionally, this research group has had no problem meeting prior, higher recruitment goals for the recent VITAL Supported Education (SEd) using similar recruitment strategies (n = 43).

Screening. Potential participants will be pre-screened by phone to determine if basic study eligibility criteria are met. Individuals who remain eligible after the pre-screen will be invited to attend a comprehensive in-person baseline session. At the beginning of the baseline session, the investigators will provide participants with detailed information about the study and obtain written informed consent. Following informed consent, participants will complete study assessments and, with staff assistance, and download the app to their personal device.

Usability testing procedures. Participants will be provided with the app and log-in information, to utilize at home over a four-week period. During that time, they will complete at least two in-app tasks within per week, including setting up an educational roadmap, tracking course assignments, learning about successful study strategies, completing a coping exercise, and finding information related to GI Bill benefits.

Participants will be evaluated after app use in the following areas: (a) app usefulness: whether users can successfully complete designated tasks on the app, (b) effectiveness: the ability to accomplish assigned educational app tasks quickly and easily, (c) learnability: the ability to meet pre-determined app navigation goals (e.g. accessing and inputting information into the education roadmap), (d) satisfaction: users' feelings and opinions about the app related to their educational functioning and alignment with mental-health needs, and (e) engagement: initial and sustained interaction between a user and a mobile application. Usability measures will include two quantitative system usability scales (SUS; AES), task completion (quantitative), app log-in and use information (engagement). Perceived usefulness, engagement, and overall satisfaction will also be assessed through a post-app-testing qualitative interview (1.5 hours).

After each round of testing, the investigative team will work with the programming team to improve the app. The following measures will be utilized: Mental Health Diagnostic Measure: The Structured Clinical Interview (SCID-5) is a reliable semi-structured instrument that is the gold standard for diagnosing mental health disorders.

System Usability Scale (SUS): The SUS a 10-item measure, scored on a 5-point Likert scale, that assesses human-computer interaction. A SUS score above 58 is regarded as above average, and a SUS score above 80 a score above which participants are likely to recommend the product to friends.

Acceptability E-Scale (AES): The AES is a 5-item validated measure of the acceptability of mobile interventions, which has a cut-off of 80% for acceptability covering overall and specific component satisfaction.

Academic Functioning Outcomes: Increased academic function is the primary non-usability related goal of the VetEd app. Consequently, preliminary educational functioning information related to academic self-efficacy, preparedness for course tasks, and perceived confidence in course completion/retention will also be collected and described. In line with educational research suggesting non-grade factors can be stronger predictors of educational success and graduation, the investigators will specifically assess academic self-efficacy (Academic Self-Efficacy Scale: ASES), ability to complete course tasks (The Coping With Education Barriers Subscale: CWBS), and confidence in completing coursework and academic retention (qualitative interview).

Community Reintegration: The Military to Civilian questionnaire (M2C-Q) is a 16-item intended to measure dimensions of community reintegration related to productivity (in education, work, and domestic life); social relations; community engagement; and perceived meaning in life, self-care, and leisure.

Data management. VetEd will be developed and hosted on a secure server by Geisel Software, Inc. The app will collect usability and content data, but will not collect personally identifiable information. Coded usernames and passwords will be used for app log-in. When data collection is complete, Geisel Software, Inc. will transfer the coded usability data to the Bedford VAMC research staff. Data will not be shared outside the VA. Paper research files will be kept at the Bedford VAMC in a locked cabinet and use deidentified numbers. Data will be checked for inconsistencies, omissions, and errors. The research assistant will be trained in data management and confidentiality procedures, and be responsible for data entry and management.

Data analysis plan. The main Phase 2 quantitative measures of acceptability and usability will be the Acceptability E-Scale and the System Usability Scale (SUS), respectively. The investigators expect that Acceptability E-Scale and SUS ratings will increase across the 3 waves of app revision, and that by the third wave the investigators will reach the benchmark of 80% on the Acceptability E-Scale and a 68 on the SUS (good acceptability). Additional data will be provided by the qualitative interviews, using rapid qualitative processes. The investigators will also assess feasibility and engagement by measuring the proportion of individuals who successfully complete the 4 week app testing, with at least 5 completed activities (of requested 8, 2 per week) or 63% of the program completed, in line with the mean number of completed sessions found in previous mobile interventions for chronic conditions. The investigators will also assess academic functioning (academic self-efficacy, course task completion, and perceived retention/course completion) and community reintegration (M2C-Q) pre- and post-app use for each student Veteran specifically to 1) characterize the sample in terms of baseline academic functioning and community reintegration, 2) describe changes in academic functioning and community reintegration, and 3) test the feasibility of these measures in anticipation of a larger, future trial.

• Study Type: Interventional
• Enrollment (Actual): 13
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • Massachusetts
    • Bedford, Massachusetts, United States, 01730-1114
      • VA Bedford HealthCare System, Bedford, MA

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

• Currently enrolled in a minimum of at least 2 courses in a higher education setting
• Current DSM-5 mental health disorder
• Competent to provide informed consent
• Owns a smart phone capable of supporting mobile apps
• Are willing and able to read in English
• Self-report struggling with current course activities

Exclusion Criteria:

• Not currently enrolled as a student
• Current or recent (within 1 month of study entry) moderate or severe DSM-5 alcohol or drug use disorder
• Cognitive impairment that would interfere with participation
• Severe suicidal ideation or a psychiatric hospitalization within the past month

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Other
• Allocation: N/A
• Interventional Model: Single Group Assignment
• Masking: None (Open Label)

Number of Arms

1

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: VetEd Mobile Application; Participants be provided with the VetEd mobile app, with instructions on how to utilize the app at home over a four-week period.	Behavioral: VetEd Mobile Application; Participants be provided with the VetEd mobile app, with instructions on how to utilize the app at home over a four-week period

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
System Usability Scale (SUS)	The System Usability Scale (SUS) a 10-item measure, scored on a 5-point Likert scale, that assesses human-computer interaction. Respondents are asked to rate 10 statements on a 5-point Likert scale (1=strongly disagree to 5=strongly agree). All SUS items focus on their experience using the mobile application, for example, "I would imagine that most people would learn to use this system very quickly." Each item is rescored, totaled, and multiplied by 2.5 to get a SUS total score ranging from 0 to 100. Higher scores indicate greater usability of a technology system, with above-average usability of a system suggested by a score >68, while a score >80 indicates high usability.	4-week time point

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Acceptability E-Scale (AES)	The Acceptability E-Scale (AES) is a 6-item validated measure of the acceptability of mobile apps and online interventions, which has a cut-off of 80% (i.e., a score of 24) for acceptability covering overall and specific component satisfaction. The six items are scored on a scale from 1 to 5, then summed for a total AES score. AES total scores range from 5 to 30, with higher scores showing greater satisfaction and acceptability of the VetEd mobile app.	4-week time point
The Academic Self-Efficacy Scale (ASES)	The Academic Self-Efficacy Scale (ASES), is an eight-item rapid assessment instrument to assess respondent's self-efficacy regarding several skills related to academic achievement, such as time management, taking notes, taking tests, and general academic ability. Each statement reflects the respondents academic abilities and sense of effectiveness, and is scored on a scale of 1 (Very Untrue) to 7 (Very True). Scores range from 8 to 56, with higher scores indicating greater reported academic self-efficacy.	Baseline
The Coping With Education Barriers Subscale (CWBS)	The Coping with Education Barriers Subscale (CWBS) is a subscale of the full 28 item scale that includes 2 subscales, the Coping with Career-Related Barriers (not administered for this study) and the Coping with Educational Barriers scale (21 items, administered for this study). This subscale assesses the participants' degree of confidence in overcoming internal and external educational barriers. Each statement is rated by the participants' degree of confidence that they could overcome the potential educational barrier, from not at all confident (1) to highly confident (5). Scores range from 21 to 105, and higher scores reflect greater perceived confidence in overcoming the common-listed educational barriers.	Baseline
Military to Civilian Questionnaire (M2C-Q)	The Military to Civilian questionnaire (M2C-Q) contains 16-items intended to measure dimensions of community reintegration related to productivity (in education, work, and domestic life); social relations; community engagement; and perceived meaning in life, self-care, and leisure. Items are rated on a scale from No difficulty (0) to Extreme difficulty (4), with higher scores indicating greater difficulties with community reintegration and ranging from 0 to 64.	Baseline

Collaborators and Investigators

• Sponsor: VA Office of Research and Development
• Investigators: Principal Investigator: Erin D. Reilly, PhD, VA Bedford HealthCare System, Bedford, MA

Study record dates

Study Major Dates

• Study Start (Actual): June 1, 2023
• Primary Completion (Actual): August 30, 2024
• Study Completion (Actual): September 30, 2024

Study Registration Dates

• First Submitted: April 15, 2022
• First Submitted That Met QC Criteria: April 18, 2022
• First Posted (Actual): April 25, 2022

Study Record Updates

• Last Update Posted (Estimated): October 31, 2025
• Last Update Submitted That Met QC Criteria: October 21, 2025
• Last Verified: October 1, 2025

More Information

Terms related to this study

• Keywords: Veteran; Education; Students; College; Mobile applications
• Additional Relevant MeSH Terms: Trauma and Stressor Related Disorders; Mental Disorders; Behavioral Symptoms; Chemically-Induced Disorders; Stress Disorders, Traumatic; Behavior; Personal Satisfaction; Anxiety Disorders; Depression; Substance-Related Disorders; Stress Disorders, Post-Traumatic; Psychological Well-Being
• Other Study ID Numbers: D3897-P; RX-21-003 (Other Grant/Funding Number: Rehabilitation Research & Development)

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No
• product manufactured in and exported from the U.S.: No