Mobile app rating scale: a new tool for assessing the quality of health mobile apps

Stoyan R Stoyanov, Leanne Hides, David J Kavanagh, Oksana Zelenko, Dian Tjondronegoro, Madhavan Mani, Stoyan R Stoyanov, Leanne Hides, David J Kavanagh, Oksana Zelenko, Dian Tjondronegoro, Madhavan Mani

Abstract

Background: The use of mobile apps for health and well being promotion has grown exponentially in recent years. Yet, there is currently no app-quality assessment tool beyond "star"-ratings.

Objective: The objective of this study was to develop a reliable, multidimensional measure for trialling, classifying, and rating the quality of mobile health apps.

Methods: A literature search was conducted to identify articles containing explicit Web or app quality rating criteria published between January 2000 and January 2013. Existing criteria for the assessment of app quality were categorized by an expert panel to develop the new Mobile App Rating Scale (MARS) subscales, items, descriptors, and anchors. There were sixty well being apps that were randomly selected using an iTunes search for MARS rating. There were ten that were used to pilot the rating procedure, and the remaining 50 provided data on interrater reliability.

Results: There were 372 explicit criteria for assessing Web or app quality that were extracted from 25 published papers, conference proceedings, and Internet resources. There were five broad categories of criteria that were identified including four objective quality scales: engagement, functionality, aesthetics, and information quality; and one subjective quality scale; which were refined into the 23-item MARS. The MARS demonstrated excellent internal consistency (alpha = .90) and interrater reliability intraclass correlation coefficient (ICC = .79).

Conclusions: The MARS is a simple, objective, and reliable tool for classifying and assessing the quality of mobile health apps. It can also be used to provide a checklist for the design and development of new high quality health apps.

Keywords: assessment; e-health; mental health; mobile application; mobile health (mhealth); rating; scale development; well being.

Conflict of interest statement

Conflicts of Interest: None declared.

Figures

Figure 1
Figure 1
Flow diagram of the process utilized to identify apps for piloting the Mobile App Rating Scale (MARS).

References

    1. Riley WT, Rivera DE, Atienza AA, Nilsen W, Allison SM, Mermelstein R. Health behavior models in the age of mobile interventions: Are our theories up to the task? Transl Behav Med. 2011 Mar;1(1):53–71. doi: 10.1007/s13142-011-0021-7.
    1. Cisco & Co. Cisco. Cisco visual networking index: Cisco & Co; 2014. [2014-02-26]. Cisco visual networking index: Global mobile data traffic forecast update, 2013–2018 .
    1. Daum A. Canalys. 2013. [2014-02-27]. 11% quarterly growth in downloads for leading app stores .
    1. Dredge S. The Guardian Online. 2013. [2014-02-27]. Mobile apps revenues tipped to reach $26bn in 2013 .
    1. Cummings E, Borycki E, Roehrer E. Issues and considerations for healthcare consumers using mobile applications. Stud Health Technol Inform. 2013;183:227–231.
    1. Kuehnhausen M, Frost VS. Trusting smartphone apps? To install or not to install, that is the question. Cognitive Methods in Situation Awareness and Decision Support; IEEE International Multi-Disciplinary Conference; 2013 Feb 25-28; San Diego, CA, USA. IEEE; 2013. pp. 30–37.
    1. Girardello A, Michahelles F. AppAware: Which mobile applications are hot?. 12th international conference on Human computer interaction with mobile devices and services; 2010 Sept; New York. NY: ACM; 2010. Sep, pp. 431–434.
    1. Seethamraju R. Measurement of user perceived web quality. 12th European Conference on Information Systems (ECIS); 2004 Jan; Turku, Finland. 2004. Jan,
    1. Olsina L, Rossi G. Measuring Web application quality with WebQEM. IEEE Multimedia. 2002 Oct;9(4):20–29. doi: 10.1109/MMUL.2002.1041945.
    1. Aladwani AM, Palvia PC. Developing and validating an instrument for measuring user-perceived web quality. Information & Management. 2002 May;39(6):467–476. doi: 10.1016/S0378-7206(01)00113-6.
    1. Moustakis V, Litos C, Dalivigas A, Tsironis L. Website quality assessment criteria. 9th international conference of information quality; 2004 Nov 5-7; Boston, MA, USA. 2004. Nov, pp. 59–73.
    1. Kim P, Eng TR, Deering MJ, Maxfield A. Published criteria for evaluating health related web sites: Review. BMJ. 1999 Mar 6;318(7184):647–649.
    1. Handel MJ. mHealth (mobile health)-using Apps for health and wellness. Explore (NY) 2011;7(4):256–261. doi: 10.1016/j.explore.2011.04.011.
    1. Khoja S, Durrani H, Scott RE, Sajwani A, Piryani U. Conceptual framework for development of comprehensive e-health evaluation tool. Telemed J E Health. 2013 Jan;19(1):48–53. doi: 10.1089/tmj.2012.0073.
    1. Health Care Information Management Systems Society . mHIMSS App Usability Work Group. mHIMSS; 2012. [2014-03-04]. Selecting a mobile app: Evaluating the usability of medical applications .
    1. Lewis Thomas Lorchan, Wyatt Jeremy C. mHealth and mobile medical Apps: A framework to assess risk and promote safer use. J Med Internet Res. 2014;16(9):e210. doi: 10.2196/jmir.3133.
    1. . [2015-02-26]. Usability sciences .
    1. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Int J Surg. 2010;8(5):336–341. doi: 10.1016/j.ijsu.2010.02.007.
    1. . [2014-03-03].
    1. Zou GY. Sample size formulas for estimating intraclass correlation coefficients with precision and assurance. Stat Med. 2012 Dec 20;31(29):3972–3981. doi: 10.1002/sim.5466.
    1. Shrout PE, Fleiss JL. Intraclass correlations: Uses in assessing rater reliability. Psychol Bull. 1979 Mar;86(2):420–428.
    1. Hallgren KA. Computing inter-rater reliability for observational data: An overview and tutorial. Tutor Quant Methods Psychol. 2012;8(1):23–34.
    1. Oppenheim AN. Questionnaire design, interviewing, and attitude measurement. London: Pinter; 1992.
    1. JMIR Publications JMIR mhealth and uhealth (JMU) 2013. Mar 12, [2014-11-12]. Apps peer-review launched .
    1. Su W. A preliminary survey of knowledge discovery on smartphone applications (apps): Principles, techniques and research directions for e-health. Proceedings of the International Conference on Complex Medical Engineering; Proceedings of the International Conference on Complex Medical Engineering; 2014; Taipei, Taiwan. ; 2014.

Source: PubMed

Sponsorer og samarbejdspartnere

Medicinske tilstande

Narkotikainterventioner

3
Abonner