The mHealth App Usability Questionnaire (MAUQ): Development and Validation Study

Leming Zhou, Jie Bao, I Made Agus Setiawan, Andi Saptono, Bambang Parmanto, Leming Zhou, Jie Bao, I Made Agus Setiawan, Andi Saptono, Bambang Parmanto

Abstract

Background: After a mobile health (mHealth) app is created, an important step is to evaluate the usability of the app before it is released to the public. There are multiple ways of conducting a usability study, one of which is collecting target users' feedback with a usability questionnaire. Different groups have used different questionnaires for mHealth app usability evaluation: The commonly used questionnaires are the System Usability Scale (SUS) and Post-Study System Usability Questionnaire (PSSUQ). However, the SUS and PSSUQ were not designed to evaluate the usability of mHealth apps. Self-written questionnaires are also commonly used for evaluation of mHealth app usability but they have not been validated.

Objective: The goal of this project was to develop and validate a new mHealth app usability questionnaire.

Methods: An mHealth app usability questionnaire (MAUQ) was designed by the research team based on a number of existing questionnaires used in previous mobile app usability studies, especially the well-validated questionnaires. MAUQ, SUS, and PSSUQ were then used to evaluate the usability of two mHealth apps: an interactive mHealth app and a standalone mHealth app. The reliability and validity of the new questionnaire were evaluated. The correlation coefficients among MAUQ, SUS, and PSSUQ were calculated.

Results: In this study, 128 study participants provided responses to the questionnaire statements. Psychometric analysis indicated that the MAUQ has three subscales and their internal consistency reliability is high. The relevant subscales correlated well with the subscales of the PSSUQ. The overall scale also strongly correlated with the PSSUQ and SUS. Four versions of the MAUQ were created in relation to the type of app (interactive or standalone) and target user of the app (patient or provider). A website has been created to make it convenient for mHealth app developers to use this new questionnaire in order to assess the usability of their mHealth apps.

Conclusions: The newly created mHealth app usability questionnaire-MAUQ-has the reliability and validity required to assess mHealth app usability.

Keywords: mobile apps; questionnaire design; reliability and validity.

Conflict of interest statement

Conflicts of Interest: None declared.

©Leming Zhou, Jie Bao, I Made Agus Setiawan, Andi Saptono, Bambang Parmanto. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 11.04.2019.

Figures

Figure 1
Figure 1
A flow chart of the new usability questionnaire development and validation. mHealth: mobile health; s: number of statements; s1: number of statements in the questionnaire for interactive mobile apps; s2: number of statements in the questionnaire for standalone mobile apps.

References

    1. Kao C, Liebovitz DM. Consumer Mobile Health Apps: Current State, Barriers, and Future Directions. PM R. 2017 May;9(5S):S106–S115. doi: 10.1016/j.pmrj.2017.02.018.
    1. Roess A. The Promise, Growth, and Reality of Mobile Health - Another Data-free Zone. N Engl J Med. 2017 Nov 23;377(21):2010–2011. doi: 10.1056/NEJMp1713180.
    1. Parmanto B, Pramana G, Yu DX, Fairman AD, Dicianno BE, McCue MP. iMHere: A Novel mHealth System for Supporting Self-Care in Management of Complex and Chronic Conditions. JMIR Mhealth Uhealth. 2013 Jul 11;1(2):e10. doi: 10.2196/mhealth.2391.
    1. Fairman AD, Dicianno BE, Datt N, Garver A, Parmanto B, McCue M. Outcomes of Clinicians, Caregivers, Family Members and Adults with Spina Bifida Regarding Receptivity to use of the iMHere mHealth Solution to Promote Wellness. Int J Telerehabil. 2013 Jun;5(1):3–16. doi: 10.5195/ijt.2013.6116.
    1. Seto E, Leonard KJ, Cafazzo JA, Barnsley J, Masino C, Ross HJ. Perceptions and experiences of heart failure patients and clinicians on the use of mobile phone-based telemonitoring. J Med Internet Res. 2012;14(1):e25. doi: 10.2196/jmir.1912.
    1. Seto E, Leonard KJ, Cafazzo JA, Barnsley J, Masino C, Ross HJ. Mobile phone-based telemonitoring for heart failure management: a randomized controlled trial. J Med Internet Res. 2012;14(1):e31. doi: 10.2196/jmir.1909.
    1. Krebs P, Duncan DT. Health App Use Among US Mobile Phone Owners: A National Survey. JMIR Mhealth Uhealth. 2015 Nov;3(4):e101. doi: 10.2196/mhealth.4924.
    1. Research2Guidance. 2017. [2019-04-08]. mHealth App Economics 2017 - Current Status and Future Trends in Mobile Health
    1. Harrison R, Flood D, Duce D. Usability of mobile applications: literature review and rationale for a new usability model. J Interact Sci. 2013;1(1):1. doi: 10.1186/2194-0827-1-1.
    1. Ntawanga F, Calitz A, Barnard L. A Context-Aware Model to Improve Usability of Information Display on Smartphone Apps for Emerging Users. Afr J Inf Syst. 2015;7(4):30–55.
    1. Moumane K, Idri A, Abran A. Usability evaluation of mobile applications using ISO 9241 and ISO 25062 standards. Springerplus. 2016;5:548. doi: 10.1186/s40064-016-2171-z.
    1. Schnall R, Cho H, Liu J. Health Information Technology Usability Evaluation Scale (Health-ITUES) for Usability Assessment of Mobile Health Technology: Validation Study. JMIR Mhealth Uhealth. 2018 Jan 05;6(1):e4. doi: 10.2196/mhealth.8851.
    1. Brooke J. SUS: A quick and dirty usability scale. In: Jordan PW, Thomas B, Weerdmeester BA, McClelland AL, editors. Usability Evaluation in Industry. London: Taylor and Francis; 1996. pp. 189–194.
    1. Lewis JR. Psychometric Evaluation of the PSSUQ Using Data from Five Years of Usability Studies. International Journal of Human-Computer Interaction. 2002 Sep;14(3-4):463–488. doi: 10.1080/10447318.2002.9669130.
    1. Zhou L, Bao J, Parmanto B. Systematic Review Protocol to Assess the Effectiveness of Usability Questionnaires in mHealth App Studies. JMIR Res Protoc. 2017 Aug 01;6(8):e151. doi: 10.2196/resprot.7826.
    1. Lewis J. IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction. 1995 Jan;7(1):57–78. doi: 10.1080/10447319509526110.
    1. Davis F. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly. 1989 Sep;13(3):319–340. doi: 10.2307/249008.
    1. Lund A. Measuring Usability with the USE Questionnaire. Usability Interface. 2001;8(2):3–6.
    1. Coleman N. SUMI (Software Usability Measurement Inventory) as a knowledge elicitation tool for improving usability. Ireland: Department of Applied Psychology, University College Cork; 1993.
    1. Chin J, Diehl V, Norman K. Development of an instrument measuring user satisfaction of the human-computer interface. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; CHI'88; May 15 - 19; Washington DC. New York: ACM; 1988. pp. 213–218.
    1. Brown W, Yen P, Rojas M, Schnall R. Assessment of the Health IT Usability Evaluation Model (Health-ITUEM) for evaluating mobile health (mHealth) technology. J Biomed Inform. 2013 Dec;46(6):1080–7. doi: 10.1016/j.jbi.2013.08.001.
    1. Yen P, Sousa KH, Bakken S. Examining construct and predictive validity of the Health-IT Usability Evaluation Scale: confirmatory factor analysis and structural equation modeling results. J Am Med Inform Assoc. 2014 Oct;21(e2):e241–8. doi: 10.1136/amiajnl-2013-001811.
    1. Hart S, Staveland L. Development of NASA-TLX (Task Load Index): Results of empiricaltheoretical research. In: Hancock PA, Meshkati N, editors. Human Mental Workload. Amsterdam: North Holland Press; 1988.
    1. Parmanto B, Lewis, Jr. AN, Graham KM, Bertolet MH. Development of the Telehealth Usability Questionnaire (TUQ) Int J Telerehab. 2016 Jul 01;8(1):3–10. doi: 10.5195/ijt.2016.6196.
    1. Nielsen J. Usability Engineering. Boston: Academic Press Inc; 1993.
    1. Wang C, Huang C. A study of usability principles and interface design for mobile e-books. Ergonomics. 2015;58(8):1253–65. doi: 10.1080/00140139.2015.1013577.
    1. Bendixen RM, Fairman AD, Karavolis M, Sullivan C, Parmanto B. A User-Centered Approach: Understanding Client and Caregiver Needs and Preferences in the Development of mHealth Apps for Self-Management. JMIR Mhealth Uhealth. 2017 Sep 26;5(9):e141. doi: 10.2196/mhealth.7136.
    1. Parmanto B, Pramana G, Yu DX, Fairman AD, Dicianno BE. Development of mHealth system for supporting self-management and remote consultation of skincare. BMC Med Inform Decis Mak. 2015 Dec 30;15:114. doi: 10.1186/s12911-015-0237-4.
    1. Pitt + Me. [2019-04-08].
    1. Pew Research Center Pew Research Center. 2018. [2018-02-12]. Mobile Fact Sheet
    1. Sauro J, Lewis J. Quantifying the User Experience: Practical Statistics for User Research. Burlington, MA: Morgan Kaufmann; 2019.
    1. Costello AB, Osborn JW. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment Research & Evaluation. 2005;10(7):1–9.
    1. Tabachnick B, Fidell L. Using Multivariate Statistics. Boston: Allyn and Bacon; 2001.
    1. Landauer T. Behavioral research methods in human-computer interaction. In: Helander M, Landauer TK, Prabhu P, editors. Handbook of Human-Computer Interaction. 2nd Edition. Elsevier, Inc: Amsterdam, Netherlands; 1997. pp. 203–227.
    1. PITT Usability Questionnaire For Telehealth Systems (TUQ) and Mobile Health Apps (MAUQ) [2019-04-08]. .
    1. Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth. 2015 Mar;3(1):e27. doi: 10.2196/mhealth.3422.
    1. Stoyanov SR, Hides L, Kavanagh DJ, Wilson H. Development and Validation of the User Version of the Mobile Application Rating Scale (uMARS) JMIR Mhealth Uhealth. 2016 Jun 10;4(2):e72. doi: 10.2196/mhealth.5849.

Source: PubMed

Sponsors en medewerkers

Medische omstandigheden

Geneesmiddelinterventies

3
Abonneren