Postsecondary Student Engagement With a Mental Health App and Online Platform (Thought Spot): Qualitative Study of User Experience

Howard W Wong, Brian Lo, Jenny Shi, Elisa Hollenberg, Alexxa Abi-Jaoude, Andrew Johnson, Gloria Chaim, Kristin Cleverley, Joanna Henderson, Andrea Levinson, Janine Robb, Aristotle Voineskos, David Wiljer, Howard W Wong, Brian Lo, Jenny Shi, Elisa Hollenberg, Alexxa Abi-Jaoude, Andrew Johnson, Gloria Chaim, Kristin Cleverley, Joanna Henderson, Andrea Levinson, Janine Robb, Aristotle Voineskos, David Wiljer

Abstract

Background: There is growing interest in using mobile apps and online tools to support postsecondary student mental health, but most of these solutions have suboptimal user engagement in real-world settings. Poor engagement can limit long-term effectiveness and usefulness of these tools. Previous literature has proposed several theories that link factors such as low usability and poor user-centered design to app disengagement. However, few studies provide direct evidence showing what factors contribute to suboptimal user engagement in the context of mobile mental health apps for postsecondary students.

Objective: This study focuses on understanding postsecondary students' attitudes and behaviors when using Thought Spot, a co-designed mental health app and online platform, to understand factors related to engagement and user experience.

Methods: Students who were given access to Thought Spot for 6 months during a randomized trial of the intervention were invited to participate in one-on-one semistructured interviews. The interviews explored participants' overall experiences and perceptions of the app, along with factors that affected their usage of various features. All interviews were recorded, and template analysis was used to analyze transcripts.

Results: User satisfaction was mixed among users of Thought Spot. The degree of engagement with the app appeared to be affected by factors that can be grouped into 5 themes: (1) Students valued detailed, inclusive, and relevant content; (2) Technical glitches and a lack of integration with other apps affected the overall user experience and satisfaction with the app; (3) Using the app to support peers or family can increase engagement; (4) Crowdsourced information from peers about mental health resources drove user engagement, but was difficult to obtain; and (5) Users often turned to the app when they had an immediate need for mental health information, rather than using it to track mental health information over time.

Conclusions: Content, user experience, user-centeredness, and peer support are important determinants of user engagement with mobile mental health apps among postsecondary students. In this study, participants disengaged when the app did not meet their expectations on these determinants. Future studies on user engagement should further explore the effectiveness of different features and the relative importance of various criteria for high-quality apps. Further focus on these issues may inform the creation of interventions that increase student engagement and align with their mental health needs.

Trial registration: ClinicalTrials.gov NCT03412461; https://ichgcp.net/clinical-trials-registry/NCT03412461.

International registered report identifier (irrid): RR2-10.2196/resprot.6446.

Keywords: adolescent; help-seeking; mental health; mobile apps; postsecondary; qualitative study; transition-aged youth; user experience.

Conflict of interest statement

Conflicts of Interest: None declared.

©Howard W Wong, Brian Lo, Jenny Shi, Elisa Hollenberg, Alexxa Abi-Jaoude, Andrew Johnson, Gloria Chaim, Kristin Cleverley, Joanna Henderson, Andrea Levinson, Janine Robb, Aristotle Voineskos, David Wiljer. Originally published in JMIR Mental Health (http://mental.jmir.org), 02.04.2021.

References

    1. Auerbach RP, Alonso J, Axinn WG, Cuijpers P, Ebert DD, Green JG, Hwang I, Kessler RC, Liu H, Mortier P, Nock MK, Pinder-Amaker S, Sampson NA, Aguilar-Gaxiola S, Al-Hamzawi A, Andrade LH, Benjet C, Caldas-de-Almeida JM, Demyttenaere K, Florescu S, de GG, Gureje O, Haro JM, Karam EG, Kiejna A, Kovess-Masfety V, Lee S, McGrath JJ, O'Neill S, Pennell B, Scott K, Ten HM, Torres Y, Zaslavsky AM, Zarkov Z, Bruffaerts R. Mental disorders among college students in the World Health Organization World Mental Health Surveys. Psychol Med. 2016 Oct;46(14):2955–2970. doi: 10.1017/S0033291716001665.
    1. Anderson KK, Norman R, MacDougall A, Edwards J, Palaniyappan L, Lau C, Kurdyak P. Effectiveness of Early Psychosis Intervention: Comparison of Service Users and Nonusers in Population-Based Health Administrative Data. Am J Psychiatry. 2018 May 01;175(5):443–452. doi: 10.1176/appi.ajp.2017.17050480.
    1. Kessler RC, Avenevoli S, Ries Merikangas K. Mood disorders in children and adolescents: an epidemiologic perspective. Biological Psychiatry. 2001 Jun;49(12):1002–1014. doi: 10.1016/s0006-3223(01)01129-5.
    1. Seko Y, Kidd S, Wiljer D, McKenzie K. Youth mental health interventions via mobile phones: a scoping review. Cyberpsychol Behav Soc Netw. 2014 Sep;17(9):591–602. doi: 10.1089/cyber.2014.0078.
    1. Garrido S, Cheers D, Boydell K, Nguyen QV, Schubert E, Dunne L, Meade T. Young People's Response to Six Smartphone Apps for Anxiety and Depression: Focus Group Study. JMIR Ment Health. 2019 Oct 02;6(10):e14385. doi: 10.2196/14385.
    1. Hollis C, Falconer CJ, Martin JL, Whittington C, Stockton S, Glazebrook C, Davies EB. Annual Research Review: Digital health interventions for children and young people with mental health problems: a systematic and meta-review. J Child Psychol Psychiatry. 2016 Dec 10; doi: 10.1111/jcpp.12663.
    1. Harrer M, Adam SH, Baumeister H, Cuijpers P, Karyotaki E, Auerbach RP, Kessler RC, Bruffaerts R, Berking M, Ebert DD. Internet interventions for mental health in university students: A systematic review and meta-analysis. Int J Methods Psychiatr Res. 2019 Jun;28(2):e1759. doi: 10.1002/mpr.1759.
    1. Becker TD, Torous JB. Recent Developments in Digital Mental Health Interventions for College and University Students. Curr Treat Options Psych. 2019 Jun 14;6(3):210–220. doi: 10.1007/s40501-019-00178-8.
    1. Conley CS, Durlak JA, Shapiro JB, Kirsch AC, Zahniser E. A Meta-Analysis of the Impact of Universal and Indicated Preventive Technology-Delivered Interventions for Higher Education Students. Prev Sci. 2016 Dec;17(6):659–78. doi: 10.1007/s11121-016-0662-3.
    1. Sander L, Rausch L, Baumeister H. Effectiveness of Internet-Based Interventions for the Prevention of Mental Disorders: A Systematic Review and Meta-Analysis. JMIR Ment Health. 2016;3(3):e38. doi: 10.2196/mental.6061.
    1. Firth J, Torous J, Nicholas J, Carney R, Pratap A, Rosenbaum S, Sarris J. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry. 2017 Oct;16(3):287–298. doi: 10.1002/wps.20472. doi: 10.1002/wps.20472.
    1. Perski O, Blandford A, West R, Michie S. Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis. Transl Behav Med. 2016 Dec 13; doi: 10.1007/s13142-016-0453-1.
    1. Baumel A, Muench F, Edan S, Kane JM. Objective User Engagement With Mental Health Apps: Systematic Search and Panel-Based Usage Analysis. J Med Internet Res. 2019 Sep 25;21(9):e14567. doi: 10.2196/14567.
    1. Couper MP, Alexander GL, Zhang N, Little RJA, Maddy N, Nowak MA, McClure JB, Calvi JJ, Rolnick SJ, Stopponi MA, Cole JC. Engagement and retention: measuring breadth and depth of participant use of an online intervention. J Med Internet Res. 2010;12(4):e52. doi: 10.2196/jmir.1430.
    1. Torous J, Nicholas J, Larsen ME, Firth J, Christensen H. Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements. Evid Based Ment Health. 2018 Jun 05; doi: 10.1136/eb-2018-102891.
    1. Ng MM, Firth J, Minen M, Torous J. User Engagement in Mental Health Apps: A Review of Measurement, Reporting, and Validity. Psychiatr Serv. 2019 Mar 27;:appips201800519. doi: 10.1176/appi.ps.201800519.
    1. Dunbar MS, Sontag-Padilla L, Kase CA, Seelam R, Stein BD. Unmet Mental Health Treatment Need and Attitudes Toward Online Mental Health Services Among Community College Students. Psychiatr Serv. 2018 May 01;69(5):597–600. doi: 10.1176/appi.ps.201700402.
    1. Vaghefi I, Tulu B. The Continued Use of Mobile Health Apps: Insights From a Longitudinal Study. JMIR Mhealth Uhealth. 2019 Aug 29;7(8):e12983. doi: 10.2196/12983.
    1. Torous JB, Chan SR, Gipson SYT, Kim JW, Nguyen T, Luo J, Wang P. A Hierarchical Framework for Evaluation and Informed Decision Making Regarding Smartphone Apps for Clinical Care. Psychiatr Serv. 2018 May 01;69(5):498–500. doi: 10.1176/appi.ps.201700423.
    1. Wiljer D, Abi-Jaoude A, Johnson A, Ferguson G, Sanches M, Levinson A, Robb J, Heffernan O, Herzog T, Chaim G, Cleverley K, Eysenbach G, Henderson J, Hollenberg E, Jiang H, Isaranuwatchai W, Law M, Sharpe S, Tripp T, Voineskos A. Enhancing Self-Efficacy for Help-Seeking Among Transition-Aged Youth in Postsecondary Settings With Mental Health and/or Substance Use Concerns, Using Crowd-Sourced Online and Mobile Technologies: The Thought Spot Protocol. JMIR Res Protoc. 2016 Nov 04;5(4):e201. doi: 10.2196/resprot.6446.
    1. Hagen P, Collin P, Metcalf A, Nicholas M, Rahilly K, Swainston N. Participatory design of evidence-based online youth mental health promotion, intervention and treatment. 2012. [2021-03-06]. .
    1. Bandura A. Social cognitive theory of personality. In: Pervin LA, John OP, editors. Handbook of Personality (2nd ed) New York, NY: The Guilford Press; 1999. pp. 154–96.
    1. Rickwood D, Thomas K. Conceptual measurement framework for help-seeking for mental health problems. Psychol Res Behav Manag. 2012;5:173–83. doi: 10.2147/PRBM.S38707. doi: 10.2147/PRBM.S38707.
    1. Sennah S, Shi J, Hollenberg E, Johnson A, Ferguson G, Abi-Jaoudé A, Wiljer D. Thought Spot: Embedding Usability Testing into the Development Cycle. Stud Health Technol Inform. 2019;257:375–381.
    1. VanHeerwaarden N, Ferguson G, Abi-Jaoude A, Johnson A, Hollenberg E, Chaim G, Cleverley K, Eysenbach G, Henderson J, Levinson A, Robb J, Sharpe S, Voineskos A, Wiljer D. The Optimization of an eHealth Solution (Thought Spot) with Transition-Aged Youth in Postsecondary Settings: Participatory Design Research. J Med Internet Res. 2018 Mar 06;20(3):e79. doi: 10.2196/jmir.8102.
    1. Wiljer D, Johnson A, McDiarmid E, Abi-Jaoude A, Ferguson G, Hollenberg E, van Heerwaarden N, Tripp T, Law M. Thought Spot: Co-Creating Mental Health Solutions with Post-Secondary Students. Stud Health Technol Inform. 2017;234:370–375.
    1. Wiljer D, Shi J, Lo B, Sanches M, Hollenberg E, Johnson A, Abi-Jaoudé A, Chaim G, Cleverley K, Henderson J, Isaranuwatchai W, Levinson A, Robb J, Wong HW, Voineskos A. Effects of a Mobile and Web App (Thought Spot) on Mental Health Help-Seeking Among College and University Students: Randomized Controlled Trial. J Med Internet Res. 2020 Oct 30;22(10):e20790. doi: 10.2196/20790.
    1. Kim H, Sefcik JS, Bradway C. Characteristics of Qualitative Descriptive Studies: A Systematic Review. Res Nurs Health. 2017 Feb;40(1):23–42. doi: 10.1002/nur.21768.
    1. Gao M, Kortum P, Oswald F. Psychometric Evaluation of the USE (Usefulness, Satisfaction, and Ease of use) Questionnaire for Reliability and Validity. Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 2018 Sep 27;62(1):1414–1418. doi: 10.1177/1541931218621322.
    1. Smith J, Harré R, Van LL. Rethinking Methods in Psychology. Thousand Oaks, CA: Sage; 1995.
    1. Saunders B, Sim J, Kingstone T, Baker S, Waterfield J, Bartlam B, Burroughs H, Jinks C. Saturation in qualitative research: exploring its conceptualization and operationalization. Qual Quant. 2018;52(4):1893–1907. doi: 10.1007/s11135-017-0574-8.
    1. Brooks J, McCluskey S, Turley E, King N. The Utility of Template Analysis in Qualitative Psychology Research. Qual Res Psychol. 2015 Apr 03;12(2):202–222. doi: 10.1080/14780887.2014.955224.
    1. Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform. 2004 Feb;37(1):56–76. doi: 10.1016/j.jbi.2004.01.003.
    1. Assarroudi A, Heshmati Nabavi F, Armat MR, Ebadi A, Vaismoradi M. Directed qualitative content analysis: the description and elaboration of its underpinning methods and data analysis process. Journal of Research in Nursing. 2018 Jan 10;23(1):42–55. doi: 10.1177/1744987117741667.
    1. Brown A, Rice SM, Rickwood DJ, Parker AG. Systematic review of barriers and facilitators to accessing and engaging with mental health care among at-risk young people. Asia Pac Psychiatry. 2016 Mar;8(1):3–22. doi: 10.1111/appy.12199.
    1. Findlay LC, Sunderland A. Professional and informal mental health support reported by Canadians aged 15 to 24. Health Rep. 2014 Dec;25(12):3–11.
    1. Reavley NJ, Yap MBH, Wright A, Jorm AF. Actions taken by young people to deal with mental disorders: findings from an Australian national survey of youth. Early Interv Psychiatry. 2011 Nov;5(4):335–42. doi: 10.1111/j.1751-7893.2011.00292.x.
    1. Hetrick SE, Robinson J, Burge E, Blandon R, Mobilio B, Rice SM, Simmons MB, Alvarez-Jimenez M, Goodrich S, Davey CG. Youth Codesign of a Mobile Phone App to Facilitate Self-Monitoring and Management of Mood Symptoms in Young People With Major Depression, Suicidal Ideation, and Self-Harm. JMIR Ment Health. 2018 Jan 23;5(1):e9. doi: 10.2196/mental.9041.
    1. Thabrew H, Fleming T, Hetrick S, Merry S. Co-design of eHealth Interventions With Children and Young People. Front Psychiatry. 2018;9:481. doi: 10.3389/fpsyt.2018.00481. doi: 10.3389/fpsyt.2018.00481.
    1. Chan S, Torous J, Hinton L, Yellowlees P. Towards a Framework for Evaluating Mobile Mental Health Apps. Telemed J E Health. 2015 Jul 14; doi: 10.1089/tmj.2015.0002.
    1. Zelmer J, van HK, Notarianni M, van MT, Schellenberg M, Tannenbaum C. An Assessment Framework for e-Mental Health Apps in Canada: Results of a Modified Delphi Process. JMIR Mhealth Uhealth. 2018 Jul 09;6(7):e10016. doi: 10.2196/10016.
    1. Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth. 2015;3(1):e27. doi: 10.2196/mhealth.3422.

Source: PubMed

Patrocinadores y Colaboradores

Condiciones médicas

Intervenciones de drogas

3
Suscribir