Framework for the Design Engineering and Clinical Implementation and Evaluation of mHealth Apps for Sleep Disturbance: Systematic Review

Melissa Aji, Christopher Gordon, Elizabeth Stratton, Rafael A Calvo, Delwyn Bartlett, Ronald Grunstein, Nick Glozier, Melissa Aji, Christopher Gordon, Elizabeth Stratton, Rafael A Calvo, Delwyn Bartlett, Ronald Grunstein, Nick Glozier

Abstract

Background: Mobile health (mHealth) apps offer a scalable option for treating sleep disturbances at a population level. However, there is a lack of clarity about the development and evaluation of evidence-based mHealth apps.

Objective: The aim of this systematic review was to provide evidence for the design engineering and clinical implementation and evaluation of mHealth apps for sleep disturbance.

Methods: A systematic search of studies published from the inception of databases through February 2020 was conducted using 5 databases (MEDLINE, Embase, Cochrane Library, PsycINFO, and CINAHL).

Results: A total of 6015 papers were identified using the search strategy. After screening, 15 papers were identified that examined the design engineering and clinical implementation and evaluation of 8 different mHealth apps for sleep disturbance. Most of these apps delivered cognitive behavioral therapy for insomnia (CBT-I, n=4) or modified CBT-I (n=2). Half of the apps (n=4) identified adopting user-centered design or multidisciplinary teams in their design approach. Only 3 papers described user and data privacy. End-user acceptability and engagement were the most frequently assessed implementation metrics. Only 1 app had available evidence assessing all 4 implementation metrics (ie, acceptability, engagement, usability, and adherence). Most apps were prototype versions (n=5), with few matured apps. A total of 6 apps had supporting papers that provided a quantitative evaluation of clinical outcomes, but only 1 app had a supporting, adequately powered randomized controlled trial.

Conclusions: This is the first systematic review to synthesize and examine evidence for the design engineering and clinical implementation and evaluation of mHealth apps for sleep disturbance. The minimal number of apps with published evidence for design engineering and clinical implementation and evaluation contrasts starkly with the number of commercial sleep apps available. Moreover, there appears to be no standardization and consistency in the use of best practice design approaches and implementation assessments, along with very few rigorous efficacy evaluations. To facilitate the development of successful and evidence-based apps for sleep disturbance, we developed a high-level framework to guide researchers and app developers in the end-to-end process of app development and evaluation.

Keywords: insomnia; internet-based intervention; mHealth; mobile applications; mobile health; sleep; systematic review.

Conflict of interest statement

Conflicts of Interest: MA, CG, RC, RG, and NG are named on 2 provisional patents for the SleepFix app.

©Melissa Aji, Christopher Gordon, Elizabeth Stratton, Rafael A Calvo, Delwyn Bartlett, Ronald Grunstein, Nick Glozier. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 17.02.2021.

Figures

Figure 1
Figure 1
PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) flow diagram.
Figure 2
Figure 2
Framework for the design engineering and clinical implementation and evaluation of apps for sleep disturbance. CBT-I: cognitive behavioral therapy for insomnia.

References

    1. Adams RJ, Appleton SL, Taylor AW, Gill TK, Lang C, McEvoy RD, Antic NA. Sleep health of Australian adults in 2016: results of the 2016 Sleep Health Foundation national survey. Sleep Health. 2017 Feb;3(1):35–42. doi: 10.1016/j.sleh.2016.11.005.
    1. Cuijpers P, Sijbrandij M, Koole SL, Andersson G, Beekman AT, Reynolds CF. The efficacy of psychotherapy and pharmacotherapy in treating depressive and anxiety disorders: a meta-analysis of direct comparisons. World Psychiatry. 2013 Jun;12(2):137–48. doi: 10.1002/wps.20038. doi: 10.1002/wps.20038.
    1. Baglioni C, Battagliese G, Feige B, Spiegelhalder K, Nissen C, Voderholzer U, Lombardo C, Riemann D. Insomnia as a predictor of depression: a meta-analytic evaluation of longitudinal epidemiological studies. J Affect Disord. 2011 Dec;135(1-3):10–9. doi: 10.1016/j.jad.2011.01.011.
    1. Taylor DJ, Lichstein KL, Durrence HH, Reidel BW, Bush AJ. Epidemiology of insomnia, depression, and anxiety. Sleep. 2005 Nov;28(11):1457–64. doi: 10.1093/sleep/28.11.1457.
    1. Khan MS, Aouad R. The effects of insomnia and sleep loss on cardiovascular disease. Sleep Med Clin. 2017 Jun;12(2):167–77. doi: 10.1016/j.jsmc.2017.01.005.
    1. Vgontzas AN, Liao D, Bixler EO, Chrousos GP, Vela-Bueno A. Insomnia with objective short sleep duration is associated with a high risk for hypertension. Sleep. 2009 Apr;32(4):491–7. doi: 10.1093/sleep/32.4.491.
    1. Cappuccio FP, D'Elia L, Strazzullo P, Miller MA. Quantity and quality of sleep and incidence of type 2 diabetes: a systematic review and meta-analysis. Diabetes Care. 2010 Feb;33(2):414–20. doi: 10.2337/dc09-1124.
    1. van Straten A, van der Zweerde T, Kleiboer A, Cuijpers P, Morin CM, Lancee J. Cognitive and behavioral therapies in the treatment of insomnia: a meta-analysis. Sleep Med Rev. 2018 Apr;38:3–16. doi: 10.1016/j.smrv.2017.02.001.
    1. Jindal RD. Cognitive behavioral therapy alone and with medication for persistent insomnia. J Am Med Assoc. 2009 Sep 09;302(10):1053. doi: 10.1001/jama.2009.1282.
    1. Mitchell MD, Gehrman P, Perlis M, Umscheid CA. Comparative effectiveness of cognitive behavioral therapy for insomnia: a systematic review. BMC Fam Pract. 2012 May 25;13:40. doi: 10.1186/1471-2296-13-40.
    1. 325,000 mobile health apps available in 2017. Research2Guidance. 2017. [2020-07-07]. .
    1. Wu Y, Yao X, Vespasiani G, Nicolucci A, Dong Y, Kwong J, Li L, Sun X, Tian H, Li S. Mobile app-based interventions to support diabetes self-management: a systematic review of randomized controlled trials to identify functions associated with glycemic efficacy. JMIR Mhealth Uhealth. 2017 Mar 14;5(3):35. doi: 10.2196/mhealth.6522.
    1. Mateo GF, Font EG, Grau CF, Carreras XM. Mobile phone apps to promote weight loss and increase physical activity: a systematic review and meta-analysis. J Med Internet Res. 2015 Nov 10;17(11):253. doi: 10.2196/jmir.4836.
    1. Creber RMM, Maurer MS, Reading M, Hiraldo G, Hickey KT, Iribarren S. Review and analysis of existing mobile phone apps to support heart failure symptom monitoring and self-care management using the mobile application rating scale (MARS) JMIR Mhealth Uhealth. 2016 Jun 14;4(2):74. doi: 10.2196/mhealth.5882.
    1. Donker T, Petrie K, Proudfoot J, Clarke J, Birch M, Christensen H. Smartphones for smarter delivery of mental health programs: a systematic review. J Med Internet Res. 2013 Nov 15;15(11):247. doi: 10.2196/jmir.2791.
    1. Firth J, Torous J. Smartphone apps for schizophrenia: a systematic review. JMIR Mhealth Uhealth. 2015 Nov 06;3(4):102. doi: 10.2196/mhealth.4930.
    1. Grundy Q, Chiu K, Held F, Continella A, Bero L, Holz R. Data sharing practices of medicines related apps and the mobile ecosystem: traffic, content, and network analysis. BMJ. 2019 Mar 20;364:920. doi: 10.1136/bmj.l920.
    1. Armontrout J, Torous J, Fisher M, Drogin E, Gutheil T. Mobile mental health: navigating new rules and regulations for digital tools. Curr Psychiatry Rep. 2016 Oct;18(10):91. doi: 10.1007/s11920-016-0726-x.
    1. Nouri R, R Niakan Kalhori S, Ghazisaeedi M, Marchand G, Yasini M. Criteria for assessing the quality of mHealth apps: a systematic review. J Am Med Inform Assoc. 2018 Aug 01;25(8):1089–98. doi: 10.1093/jamia/ocy050.
    1. Buijink AWG, Visser BJ, Marshall L. Medical apps for smartphones: lack of evidence undermines quality and safety. Evid Based Med. 2013 Jun;18(3):90–2. doi: 10.1136/eb-2012-100885.
    1. Chan S, Torous J, Hinton L, Yellowlees P. Towards a framework for evaluating mobile mental health apps. Telemed J E Health. 2015 Dec;21(12):1038–41. doi: 10.1089/tmj.2015.0002.
    1. Schellong J, Lorenz P, Weidner K. Proposing a standardized, step-by-step model for creating post-traumatic stress disorder (PTSD) related mobile mental health apps in a framework based on technical and medical norms. Eur J Psychotraumatol. 2019;10(1):1611090. doi: 10.1080/20008198.2019.1611090.
    1. Bauer AM, Hodsdon S, Bechtel JM, Fortney JC. Applying the principles for digital development: case study of a smartphone app to support collaborative care for rural patients with posttraumatic stress disorder or bipolar disorder. J Med Internet Res. 2018 Jun 06;20(6):10048. doi: 10.2196/10048.
    1. Duan H, Wang Z, Ji Y, Ma L, Liu F, Chi M, Deng N, An J. Using goal-directed design to create a mobile health app to improve patient compliance with hypertension self-management: development and deployment. JMIR Mhealth Uhealth. 2020 Feb 25;8(2):14466. doi: 10.2196/14466.
    1. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009 Jul 21;6(7):1000097. doi: 10.1371/journal.pmed.1000097.
    1. Bowen DJ, Kreuter M, Spring B, Cofta-Woerpel L, Linnan L, Weiner D, Bakken S, Kaplan CP, Squiers L, Fabrizio C, Fernandez M. How we design feasibility studies. Am J Prev Med. 2009 May;36(5):452–7. doi: 10.1016/j.amepre.2009.02.002.
    1. Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res. 2011 Aug 05;13(3):52. doi: 10.2196/jmir.1772.
    1. WHO Monitoring and evaluating digital health interventions: a practical guide to conducting research and assessment. WHO. 2016:1–144. doi: 10.1017/cbo9780511712074.003.
    1. De Geest S, Sabaté E. Adherence to long-term therapies: evidence for action. Eur J Cardiovascular Nursing. 2016 Jun 22;2(4):323–3. doi: 10.1016/s1474-5151(03)00091-4.
    1. Pulantara IW, Parmanto B, Germain A. Clinical feasibility of a just-in-time adaptive intervention app (iREST) as a behavioral sleep treatment in a military population: feasibility comparative effectiveness study. J Med Internet Res. 2018 Dec 07;20(12):10124. doi: 10.2196/10124.
    1. Pulantara IW, Parmanto B, Germain A. Development of a just-in-time adaptive mhealth intervention for insomnia: usability study. JMIR Hum Factors. 2018 May 17;5(2):21. doi: 10.2196/humanfactors.8905.
    1. Reilly ED, Robinson SA, Petrakis BA, Kuhn E, Pigeon WR, Wiener RS, McInnes DK, Quigley KS. Mobile app use for insomnia self-management: pilot findings on sleep outcomes in veterans. Interact J Med Res. 2019 Jul 24;8(3):12408. doi: 10.2196/12408.
    1. Koffel E, Kuhn E, Petsoulis N, Erbes CR, Anders S, Hoffman JE, Ruzek JI, Polusny MA. A randomized controlled pilot study of CBT-I Coach: feasibility, acceptability, and potential impact of a mobile phone application for patients in cognitive behavioral therapy for insomnia. Health Informatics J. 2018 Mar;24(1):3–13. doi: 10.1177/1460458216656472.
    1. Babson KA, Ramo DE, Baldini L, Vandrey R, Bonn-Miller MO. Mobile app-delivered cognitive behavioral therapy for insomnia: feasibility and initial efficacy among veterans with cannabis use disorders. JMIR Res Protoc. 2015 Jul 17;4(3):87. doi: 10.2196/resprot.3852.
    1. Shirazi A, Clawson J, Hassanpour Y. Int J Human-Computer Stud. 2013. [2021-01-22].
    1. Bhat S, Ferraris A, Gupta D, Mozafarian M, DeBari VA, Gushway-Henry N, Gowda SP, Polos PG, Rubinstein M, Seidu H, Chokroverty S. Is there a clinical role for smartphone sleep apps? Comparison of sleep cycle detection by a smartphone application to polysomnography. J Clin Sleep Med. 2015 Jul 15;11(7):709–15. doi: 10.5664/jcsm.4840. doi: 10.5664/jcsm.4840.
    1. Fino E, Plazzi G, Filardi M, Marzocchi M, Pizza F, Vandi S, Mazzetti M. (Not so) Smart sleep tracking through the phone: findings from a polysomnography study testing the reliability of four sleep applications. J Sleep Res. 2020 Feb;29(1):12935. doi: 10.1111/jsr.12935.
    1. Scott H, Lack L, Lovato N. A pilot study of a novel smartphone application for the estimation of sleep onset. J Sleep Res. 2018 Feb;27(1):90–7. doi: 10.1111/jsr.12575. doi: 10.1111/jsr.12575.
    1. Baron KG, Duffecy J, Reid K, Begale M, Caccamo L. Technology-assisted behavioral intervention to extend sleep duration: development and design of the sleep bunny mobile app. JMIR Ment Health. 2018 Jan 10;5(1):3. doi: 10.2196/mental.8634.
    1. Bauer J, Consolvo S, Greenstein B. ShutEy: encouraging awareness of healthy sleep recommendations with a mobile, peripheral display. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; ; Austin, Texas, USA; May, 2012; Austin Texas USA. New York: Association for Computing Machinery; 2012. pp. 1401–10.
    1. Kang S, Kang JM, Cho S, Ko K, Lee YJ, Lee H, Kim L, Winkelman JW. Cognitive behavioral therapy using a mobile application synchronizable with wearable devices for insomnia treatment: a pilot study. J Clin Sleep Med. 2017 Apr 15;13(4):633–40. doi: 10.5664/jcsm.6564. doi: 10.5664/jcsm.6564.
    1. Aji M, Gordon C, Peters D, Bartlett D, Calvo RA, Naqshbandi K, Glozier N. Exploring user needs and preferences for mobile apps for sleep disturbance: mixed methods study. JMIR Ment Health. 2019 May 24;6(5):13895. doi: 10.2196/13895.
    1. Aji M, Glozier N, Bartlett D, Peters D, Calvo RA, Zheng Y, Grunstein R, Gordon C. A feasibility study of a mobile app to treat insomnia. Transl Behav Med. 2020 Mar 30;:-. doi: 10.1093/tbm/ibaa019.
    1. Horsch CH, Lancee J, Griffioen-Both F, Spruit S, Fitrianie S, Neerincx MA, Beun RJ, Brinkman W. Mobile phone-delivered cognitive behavioral therapy for insomnia: a randomized waitlist controlled trial. J Med Internet Res. 2017 Apr 11;19(4):70. doi: 10.2196/jmir.6524.
    1. Kuhn E, Weiss BJ, Taylor KL, Hoffman JE, Ramsey KM, Manber R, Gehrman P, Crowley JJ, Ruzek JI, Trockel M. CBT-I coach: a description and clinician perceptions of a mobile app for cognitive behavioral therapy for insomnia. J Clin Sleep Med. 2016 Apr 15;12(4):597–606. doi: 10.5664/jcsm.5700. doi: 10.5664/jcsm.5700.
    1. Miller KE, Kuhn E, Owen JE, Taylor K, Yu JS, Weiss BJ, Crowley JJ, Trockel M. Clinician perceptions related to the use of the CBT-I coach mobile app. Behav Sleep Med. 2019;17(4):481–91. doi: 10.1080/15402002.2017.1403326.
    1. Omeogu C, Shofer F, Gehrman P, Green-McKenzie J. Efficacy of a mobile behavioral intervention for workers with insomnia. J Occup Environ Med. 2020 Mar;62(3):246–50. doi: 10.1097/JOM.0000000000001819.
    1. Inal Y, Wake JD, Guribye F, Nordgreen T. Usability evaluations of mobile mental health technologies: systematic review. J Med Internet Res. 2020 Jan 06;22(1):15337. doi: 10.2196/15337.
    1. Morin CM, Belleville G, Bélanger L, Ivers H. The Insomnia Severity Index: psychometric indicators to detect insomnia cases and evaluate treatment response. Sleep. 2011 May 01;34(5):601–8. doi: 10.1093/sleep/34.5.601.
    1. Ko PT, Kientz JA, Choe EK, Kay M, Landis CA, Watson NF. Consumer sleep technologies: a review of the landscape. J Clin Sleep Med. 2015 Dec 15;11(12):1455–61. doi: 10.5664/jcsm.5288. doi: 10.5664/jcsm.5288.
    1. Maguire M. Methods to support human-centred design. Int J Human-Computer Stud. [2021-01-22].
    1. McCurdie T, Taneva S, Casselman M, Yeung M, McDaniel C, Ho W, Cafazzo J. mHealth consumer apps: the case for user-centered design. Biomed Instrum Technol. 2012;Suppl:49–56. doi: 10.2345/0899-8205-46.s2.49.
    1. Shah SGS, Robinson I, AlShawi S. Developing medical device technologies from users' perspectives: a theoretical framework for involving users in the development process. Int J Technol Assess Health Care. 2009 Oct;25(4):514–21. doi: 10.1017/S0266462309990328.
    1. Ben-Zeev D, Schueller SM, Begale M, Duffecy J, Kane JM, Mohr DC. Strategies for mHealth research: lessons from 3 mobile intervention studies. Adm Policy Ment Health. 2015 Mar;42(2):157–67. doi: 10.1007/s10488-014-0556-2.
    1. NHS . National Health Service. United Kingdom: NHS; 2014. Five year forward view. .
    1. Pfadenhauer LM, Mozygemba K, Gerhardus A, Hofmann B, Booth A, Lysdahl KB, Tummers M, Burns J, Rehfuess EA. Context and implementation: a concept analysis towards conceptual maturity. Z Evid Fortbild Qual Gesundhwes. 2015;109(2):103–14. doi: 10.1016/j.zefq.2015.01.004.
    1. Dubad M, Winsper C, Meyer C, Livanou M, Marwaha S. A systematic review of the psychometric properties, usability and clinical impacts of mobile mood-monitoring applications in young people. Psychol Med. 2018 Jan;48(2):208–28. doi: 10.1017/S0033291717001659.
    1. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011 Mar;38(2):65–76. doi: 10.1007/s10488-010-0319-7.
    1. Brooke J. Usability Evaluation In Industry. Florida: CRC Press; 1996. SUS: a 'Quick and Dirty' usability scale.
    1. Powell AC, Landman AB, Bates DW. In search of a few good apps. J Am Med Assoc. 2014 May 14;311(18):1851–2. doi: 10.1001/jama.2014.2564.
    1. Kargl F, van der Heijden RW, Erb B, Bösch C. Privacy in mobile sensing. Digital Phenotyping and Mobile Sensing. 2019:3–12. doi: 10.1007/978-3-030-31620-4_1.
    1. Parker L, Bero L, Gillies D, Raven M, Grundy Q. The "Hot Potato" of mental health app regulation: a critical case study of the australian policy arena. Int J Health Policy Manag. 2019 Mar 01;8(3):168–76. doi: 10.15171/ijhpm.2018.117.
    1. O'Loughlin K, Neary M, Adkins EC, Schueller SM. Reviewing the data security and privacy policies of mobile apps for depression. Internet Interv. 2019 Mar;15:110–5. doi: 10.1016/j.invent.2018.12.001.
    1. Aguilera A, Muench F. There's an app for that: information technology applications for cognitive behavioral practitioners. Behav Ther (N Y N Y) 2012 Apr;35(4):65–73.
    1. Zachariae R, Lyby MS, Ritterband LM, O'Toole MS. Efficacy of internet-delivered cognitive-behavioral therapy for insomnia - a systematic review and meta-analysis of randomized controlled trials. Sleep Med Rev. 2016 Dec;30:1–10. doi: 10.1016/j.smrv.2015.10.004.
    1. Shin JC, Kim J, Grigsby-Toussaint D. Mobile phone interventions for sleep disorders and sleep quality: systematic review. JMIR Mhealth Uhealth. 2017 Sep 07;5(9):131. doi: 10.2196/mhealth.7244.
    1. Aji M, Gordon C, Peters D, Bartlett D, Calvo RA, Naqshbandi K, Glozier N. Exploring user needs and preferences for mobile apps for sleep disturbance: mixed methods study. JMIR Ment Health. 2019 May 24;6(5):13895. doi: 10.2196/13895.
    1. Ong AA, Gillespie MB. Overview of smartphone applications for sleep analysis. World J Otorhinolaryngol Head Neck Surg. 2016 Mar;2(1):45–9. doi: 10.1016/j.wjorl.2016.02.001.
    1. Yu JS, Kuhn E, Miller KE, Taylor K. Smartphone apps for insomnia: examining existing apps' usability and adherence to evidence-based principles for insomnia management. Transl Behav Med. 2019 Jan 01;9(1):110–9. doi: 10.1093/tbm/iby014.
    1. Grigsby-Toussaint DS, Shin JC, Reeves DM, Beattie A, Auguste E, Jean-Louis G. Sleep apps and behavioral constructs: a content analysis. Prev Med Rep. 2017 Jun;6:126–9. doi: 10.1016/j.pmedr.2017.02.018.
    1. Choi YK, Demiris G, Lin S, Iribarren SJ, Landis CA, Thompson HJ, McCurry SM, Heitkemper MM, Ward TM. Smartphone applications to support sleep self-management: review and evaluation. J Clin Sleep Med. 2018 Oct 15;14(10):1783–90. doi: 10.5664/jcsm.7396. doi: 10.5664/jcsm.7396.
    1. Leigh S, Ouyang J, Mimnagh C. Effective? Engaging? Secure? Applying the ORCHA-24 framework to evaluate apps for chronic insomnia disorder. Evid Based Ment Health. 2017 Nov;20(4):20. doi: 10.1136/eb-2017-102751.
    1. Wykes T, Schueller S. Why reviewing apps is not enough: transparency for trust (T4T) principles of responsible health app marketplaces. J Med Internet Res. 2019 May 02;21(5):12390. doi: 10.2196/12390.

Source: PubMed

Sponsoren und Mitarbeiter

Krankheiten

Drogeninterventionen

3
Abonnieren