Examining the Feasibility of the Ask RoSE Mobile Mental Health Application

Examining the Feasibility of a Mobile Mental Health Application (Ask RoSE) for Augmentation of In-person Psychotherapy

Students are one of the most at-risk groups for mental illness and suicide. Up to 40% of persons with serious mental illnesses do not receive care or stop intervention programs due to negative stigma. Focusing on students, only 30% of universities provide mental health counseling as part of universities' student health services. There is an urgent need to better understand and manage the mental health burden among students. The investigators will use a mental health-focused application that utilizes artificial intelligence to identify symptoms and provides targeted, tailored in-person therapy solutions for its users. Once a patient is matched with a therapist, the application provides solutions to supplement ongoing in-person psychotherapy. This includes journaling and daily assessments that provide curated content and feedback. The current feasibility study aims to recruit 45 Johns Hopkins graduate students with at least mild depressive and/or anxiety symptoms who will be consented and enrolled in a five-week study. There are two study arms: (1) the intervention arm and (2) a waitlist control arm. During the course of the study, the participants in the intervention arm will use the application daily. Participants will receive weekly in-person psychotherapy for a total of four sessions over four weeks. Licensed therapists will provide the in-person psychotherapy. The participants in the waitlist arm will serve as controls unless there is attrition from the intervention group at which time waitlist participants will be offered a spot in the intervention arm. The primary objectives of the study are (1) To assess the usability of the application and (2) To evaluate the short-term impact on mood and anxiety of using the application to augment in-person psychotherapy.

Study Overview

• Status: Withdrawn
• Conditions: Anxiety; Mental Health Wellness 1
• Intervention / Treatment: Behavioral: Mobile Mental Health App

Detailed Description

Problem description Students are one of the most at-risk groups for mental illness and suicide. Eight out of ten students experience stress frequently or daily for months at a time. A four-year prospective study with college students revealed that 12% had suicidal ideation sometime during the students' years at college. Students must manage family life, academics, extracurricular activities, hormonal changes, relationships, identity development, and much more. All these factors combined add stress to students' daily lives and can lead to depression, anxiety or other mental health conditions.

While mental illness negatively affects young adults in significant ways, primary care doctors identify less than one third of all depression cases and up to 40% of persons with serious mental illnesses do not receive care or stop intervention programs due to negative stigma. On the university side, only 30% of respondent universities in one study provided mental health counseling within universities' student health services and after-hour care was only available at 38% of the 129 respondents.

Study model This study uses is a mental health-focused application that utilizes artificial intelligence to identify symptoms and provides targeted, tailored in-person therapy solutions for its users. The application is meant to augment in-person psychotherapy utilizing online journaling and daily patient assessments to provide curated content and feedback. In brief, the application utilizes machine learning and artificial intelligence to select content (e.g., self-help articles) and provide feedback (e.g., weekly summary score of mood and anxiety) based on user input (e.g., journal entries, daily mood rating)

Study Design

The proposed study is a 5-week feasibility study to assess the usability and potential short-term benefits of the app platform in a population of Johns Hopkins medical students.

Potential participants will be informed of the study using email listservs and flyers. The investigators also anticipate word-of-mouth communication among students. Word-of-mouth communication from students means that the investigators will tell students a broad description of the study, "The investigators are testing whether a mental health mobile app with personalized therapy sessions is beneficial to the maintenance of emotional and mental well-being in graduate students. All written materials are submitted for approval. Interested students will be directed to a secure REDcap survey, which will screen the students for eligibility. Students that meet all inclusion criteria and no exclusion criteria will be offered entrance into the study. Of note, no medical or academic records of the students will be accessed as part of the study.

After the initial screening survey, 45 eligible students will complete the consent process. The consented participants will be randomized to one of the two study arms: (1) the intervention arm or (2) the waitlist control arm in a 2:1 ratio. The intervention arm is described in detail below. The participants in the waitlist control arm will serve as controls for the study. These participants will complete the pre-pilot assessment and the post-pilot assessment. These participants will be given details on the Johns Hopkins Student Assistance Program and encouraged to pursue standard of care services. During the participant's time on the waitlist, participants may reach out to study personnel if participants need assistance obtaining psychiatric care. The investigators aim to have 30 participants complete the intervention arm and in the setting of participant attrition from the intervention arm, participants in the waitlist control arm will be offered entrance to the intervention arm. If recruited off the waitlist, the participant will re-complete consent, this time selecting "Intervention" as the study arm and will be entered into the five-week intervention study.

Participants in the intervention arm will be sent an individualized redemption code to download the application on to participant's personal mobile phones. The participants enter individual login details. During the first week of the study, each participant will be prompted by the application to complete key surveys and assessments at predetermined time intervals. The platform generates a summary health metric on-demand that is a combination of responses entered from the in-app assessments.

A summary health metric will be pushed in-app after the first week of use. Details for scheduling the first in-person therapy session will be provided at the same time as the summary health metric. Participants will be informed that the general health metrics will be shared with the therapist to help the therapist better plan the sessions. The therapy session will be scheduled in a private room on-campus. The session will be approximately 45 min long and therapist notes will be captured and stored in REDCap at the session. The therapists will offer standard-of-care psychotherapy (e.g., supportive psychotherapy, cognitive-behavioral therapy). The participants use the app for a total of 5 weeks and receive weekly in-person therapy for a total of 4 weeks (one week application only lead in, four weeks application plus in-person psychotherapy).

Two functions of the application that will be independently examined for usage and utility are its journaling function and curated insights. Participants will be asked to use the journaling function daily (more details below). The curated insights are selected by the application based on participant responses to the daily mood and anxiety surveys, as well as a built-in sentiment analysis of the journaling.

At the end of 5 weeks, all participants in both study arms will complete the post-pilot assessment using the same assessments as pre-pilot.

Participants who agree to be contacted post-pilot study will be given the option at 3, 6, and 12 months to take the same online assessments as at pre- and post-pilot. This evaluation will be used to test potential long-term impact of the app. In addition, post-pilot focus groups may be completed. There would be one focus group with therapists and separate focus groups with participants. These optional assessments will be pending study results, funding, and resources and would be submitted in the future as a change in research to the current Institutional Review Board (IRB).

Details of Study Assessments Scales / Questionnaires Mood / Anxiety Meter: The application will send timed notifications three times a day (morning, afternoon and evening) for the user to pick current mood and anxiety on a Likert scale General Anxiety Disorder-7 (GAD-7) anxiety scale. The app will prompt the user to fill in this assessment every 2 weeks. The GAD-7 parallels the 7 diagnostic symptom criteria that define DSM-IV GAD. The format and temporal framework of the items also correspond to the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) criteria and will facilitate the follow-up review of symptoms and diagnostic process. At only 7 items, the GAD-7 is substantially shorter than most anxiety screening measures. This instrument has also demonstrated acceptability among non-psychiatric patients and among busy primary care providers and is a valid and reliable screening tool for anxiety.

Patient Health Questionnaire-9 (PHQ-9) depression scale. The app will prompt the user to fill in this assessment every 2 weeks. The PHQ-9 parallels the 9 diagnostic symptom criteria that define DSM-IV. The format and temporal framework of the items also correspond to the DSM-IV criteria and will facilitate the follow-up review of symptoms and diagnostic process. At only 9 items, the PHQ-9 is substantially shorter than most depression screening measures. This instrument has also demonstrated acceptability among non-psychiatric patients and among busy primary care providers and is a valid and reliable screening tool for major depression.

Positive and Negative Emotions Scale (PANAS): The app will prompt the user to fill in this assessment every 2 weeks. The PANAS is a self-report questionnaire that consists of two 1-item scales to measure both positive and negative affect. Each item is rated on a 5-point scale of 1 (not at all) to 5 (very much).

Mobile Application Rating Scale (MARS): The MARS is a well-established framework for classifying and assessing the objective and subjective quality of apps, as well as the app's perceived impact. It is designed to score apps on the criteria of engagement, functionality, aesthetics, and information quality, as well as app subjective quality. Each MARS item used a 5-point scale (1-Inadequate, 2-Poor, 3-Acceptable, 4-Good, 5-Excellent). The MARS is calculated as mean scores of the engagement, functionality, aesthetics, and information quality objective sub-scales, and an overall mean app quality total score.

Other Online Journaling: online journaling feature allows the user to enter his/her feelings and thoughts, like in the case of a diary. The user is asked daily "How has your day been? (please provide at least four sentences to describe how you are doing and feeling today)". Every two weeks the user is asked "How have you been the last two weeks? (please provide at least four sentences describing your emotional well-being over the last two weeks)". This second prompt is meant to allow comparison with the PHQ-9 and GAD-7, both of which ask about a two-week time frame's neural network is built and calibrated to provide an emotional context through pre-labeled datasets that tag words, phrases, and sentences to a specific emotion (e.g., fear, sadness, anxiety). The neural network model then analyzes the input from the journaling features and assigns it an emotion weighting. User input is then analyzed and the output indicates what condition participants are likely currently experiencing based on participant's entry.

• Study Type: Interventional
• Phase: Not Applicable

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 21 years to 28 years (ADULT)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

Description

Inclusion Criteria:

• English speaking;
• 21-28 years of age;
• Full time Johns Hopkins University graduate student;
• Score >5 on PHQ-9 and/or >5 on GAD-7;
• Have accessible smartphone with data plan;
• Ability to dedicate one hour a week for therapy sessions;
• Ability to give informed consent and understand the tasks involved.

Exclusion Criteria:

• Not living in Baltimore or surrounding counties as this will make it difficult to schedule on-campus therapy sessions;
• Part-time students;
• Students currently receiving psychiatric care.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: SUPPORTIVE_CARE
• Allocation: RANDOMIZED
• Interventional Model: PARALLEL
• Masking: NONE

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
EXPERIMENTAL: the intervention arm; During the course of the study, the participants in the intervention arm will use the Ask RoSE application daily. Participants will receive weekly in-person psychotherapy for a total of four sessions over four weeks. Licensed therapists will provide the in-person psychotherapy	Behavioral: Mobile Mental Health App; The Mobile Mental Health App is an app that utilizes artificial intelligence to identify symptoms and provides targeted, tailored solutions for its users.
NO_INTERVENTION: a waitlist control arm; The participants in the waitlist arm will serve as controls unless there is attrition from the intervention group at which time waitlist participants will be offered a spot in the intervention arm

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Average session time when using the mobile app	Over the 5 weeks of the study, the mobile app will collect time (seconds) the user spends within each session. We will calculate the average time the user spent on mobile app in each session.	5 weeks
Time of day the mobile app is used	Over the 5 weeks of using the mobile app, we will capture the time of data the app was used in ( morning, afternoon and evenings). We will derive proportions for the app use at these times of day	5 weeks
Most commonly used feature within the app	For each session, the app keeps track of different in-app features that are used (mood meter, journal, insights, assessments). Over the 5 week period, we will rank the most commonly used features.	5 weeks
Change in Short term impact on mood as assessed by the Positive and Negative Emotions Scale (PANAS)	The PANAS is a self-report questionnaire that consists of two 1-item scales to measure both positive and negative affect. Each item is rated on a 5-point scale of 1 (not at all) to 5 (very much).	Baseline, 5 weeks
Change in Short term impact on anxiety as assessed by the General Anxiety Disorder-7 (GAD-7) scale	The GAD-7 is a self-administered 7 item instrument that uses some of the DSM-V criteria for anxiety to identify probable cases of GAD along with measuring anxiety symptom severity. Responders are asked to rate the frequency of anxiety symptoms in the last 2 weeks on a Likert scale ranging from 0-3.Severity can be determined by examining the total score. A meaningful change is 5 or more points.	Baseline, 5 weeks

Collaborators and Investigators

• Sponsor: Johns Hopkins University
• Investigators: Principal Investigator: Paul Kim, MD, PhD, Johns Hopkins University

Publications and helpful links

General Publications

• Wilcox HC, Arria AM, Caldeira KM, Vincent KB, Pinchevsky GM, O'Grady KE. Prevalence and predictors of persistent suicide ideation, plans, and attempts during college. J Affect Disord. 2010 Dec;127(1-3):287-94. doi: 10.1016/j.jad.2010.04.017. Epub 2010 May 14.
• Auerbach RP, Alonso J, Axinn WG, Cuijpers P, Ebert DD, Green JG, Hwang I, Kessler RC, Liu H, Mortier P, Nock MK, Pinder-Amaker S, Sampson NA, Aguilar-Gaxiola S, Al-Hamzawi A, Andrade LH, Benjet C, Caldas-de-Almeida JM, Demyttenaere K, Florescu S, de Girolamo G, Gureje O, Haro JM, Karam EG, Kiejna A, Kovess-Masfety V, Lee S, McGrath JJ, O'Neill S, Pennell BE, Scott K, Ten Have M, Torres Y, Zaslavsky AM, Zarkov Z, Bruffaerts R. Mental disorders among college students in the World Health Organization World Mental Health Surveys. Psychol Med. 2016 Oct;46(14):2955-2970. doi: 10.1017/S0033291716001665. Epub 2016 Aug 3. Erratum In: Psychol Med. 2017 Nov;47(15):2737.
• Visnjic A, Velickovic V, Sokolovic D, Stankovic M, Mijatovic K, Stojanovic M, Milosevic Z, Radulovic O. Relationship between the Manner of Mobile Phone Use and Depression, Anxiety, and Stress in University Students. Int J Environ Res Public Health. 2018 Apr 8;15(4):697. doi: 10.3390/ijerph15040697.
• Malinauskas R, Dumciene A, Sipaviciene S, Malinauskiene V. Relationship Between Emotional Intelligence and Health Behaviours among University Students: The Predictive and Moderating Role of Gender. Biomed Res Int. 2018 Jun 4;2018:7058105. doi: 10.1155/2018/7058105. eCollection 2018.
• Moreland JJ, Coxe KA, Yang J. Collegiate athletes' mental health services utilization: A systematic review of conceptualizations, operationalizations, facilitators, and barriers. J Sport Health Sci. 2018 Jan;7(1):58-69. doi: 10.1016/j.jshs.2017.04.009. Epub 2017 Apr 28.
• Kroenke K, Wu J, Yu Z, Bair MJ, Kean J, Stump T, Monahan PO. Patient Health Questionnaire Anxiety and Depression Scale: Initial Validation in Three Clinical Trials. Psychosom Med. 2016 Jul-Aug;78(6):716-27. doi: 10.1097/PSY.0000000000000322.
• Allan NP, Lonigan CJ, Phillips BM. Examining the Factor Structure and Structural Invariance of the PANAS Across Children, Adolescents, and Young Adults. J Pers Assess. 2015;97(6):616-25. doi: 10.1080/00223891.2015.1038388. Epub 2015 May 15.

Study record dates

Study Major Dates

• Study Start (ANTICIPATED): July 1, 2022
• Primary Completion (ANTICIPATED): December 1, 2022
• Study Completion (ANTICIPATED): December 1, 2022

Study Registration Dates

• First Submitted: April 8, 2019
• First Submitted That Met QC Criteria: April 9, 2019
• First Posted (ACTUAL): April 10, 2019

Study Record Updates

• Last Update Posted (ACTUAL): August 26, 2022
• Last Update Submitted That Met QC Criteria: August 23, 2022
• Last Verified: December 1, 2021

More Information

Terms related to this study

• Other Study ID Numbers: IRB00206859

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No