Establishing a usability cut-point for the health information technology usability evaluation scale (Health-ITUES)

Kah Poh Loh, Jianfang Liu, Sarah Ganzhorn, Gabriella Sanabria, Rebecca Schnall, Kah Poh Loh, Jianfang Liu, Sarah Ganzhorn, Gabriella Sanabria, Rebecca Schnall

Abstract

Objective: The Health Information Technology Usability Evaluation Scale (Health-ITUES) is a validated and reliable instrument to evaluate usability of information technology (IT) tools. In this study, we aimed to establish the optimal cut-point of the Health-ITUES to identify usability of IT tools.

Methods: Adult participants were recruited to a trial evaluating a mobile app for self-managing HIV. Participants completed the Health-ITUES at the 3- and 6-month follow-up. Health-ITUES is a 20-item questionnaire that assesses four subscales: impact, perceived usefulness, perceived ease of use, and user control. The total score ranged from 1 to 5 and a higher score indicates greater usability. App use was defined as the proportion of activities completed by participants in both study arms. The selection of an optimal cut-point involved a series of multiple linear regression models with 500 bootstrap replications to examine the relationship between the Health-ITUES total score and app use, controlling for potential covariates.

Results: We included 158 participants; mean age was 49.7 years (SD 10.3), 71% were African American/Black, and 72% were non-Hispanic. Mean Health-ITUES total scores at 3 and 6 months were 4.39 (SD 0.75) and 4.43 (SD 0.75), respectively. App use completedby participants from baseline to the 3-month follow-up visits was 0.61 (SD 0.36, range 0-1.72) and from 3-month to the 6-month follow-up visits was 0.51 (SD 0.37). Participants who reported greater Health-ITUES total score completed more activities [β = 0.18, 95% Confidence Interval (CI) 0.10-0.27]. The optimal cut-point of 4.32 (95% CI: 4.25-4.56) yielded the lowest p-value to identify usability of IT tools.

Conclusions: In this study of adults with HIV, we identified an optimal cut-point of 4.32 on the Health-ITUES total score to define usability. Further studies are needed to validate this cut-point.

Keywords: Health-ITUES; Information technology; Mobile health; Usability.

Conflict of interest statement

Conflicts of interest: Dr. Loh has served as a consultant to Pfizer and Seattle Genetics, and has received honoraria from Pfizer. All other authors have no relevant conflicts of interest to report.

Copyright © 2022 Elsevier B.V. All rights reserved.

Figures

Figure 1:
Figure 1:
WiseApp utilizes a self-management mobile app that contains real-time medication monitoring linked to an electronic pill bottle
Figure 2:
Figure 2:
Distribution of optimal cut-points of the Health-ITUES total score associated with app use from the 500 bootstrapping replicates

References

    1. Mobile Fact Sheet. Pew Research Center: Internet, Science & Tech April 7, 2021. .
    1. Statista Research Department, & 26, M. Healthcare apps available Google Play 2021. Statista. May 26, 2021. ..
    1. Franklin R (2019, March 19). 11 surprising mobile health statistics. Mobius MD. .
    1. Schnall R, Porras T, Cho H, Jia H, Siegel K, Mohr DC, Bakken S, Olender S, Efficacy, Use, and Usability of the VIP-HANA App for Symptom Self-management in PLWH with HANA Conditions, AIDS Behav, 25 (2021) 1699–1710.
    1. Schnall R, Kuhns L, Pearson C, Bruce J, Batey DS, Radix A, Belkind U, Hidalgo MA, Hirshfield S, Ganzhorn S, Garofalo R, Preliminary Results from a Pragmatic Clinical Trial of MyPEEPS Mobile to Improve HIV Prevention Behaviors in Young Men, Stud Health Technol Inform, 270 (2020) 1365–1366.
    1. Loh KP, Sanapala C, Di Giovanni G, Klepin HD, Janelsins M, Schnall R, Culakova E, Vertino P, Susiarjo M, Mendler JH, Liesveld JL, Lin PJ, Dunne RF, Kleckner I, Mustian K, Mohile SG, Developing and adapting a mobile health exercise intervention for older patients with myeloid neoplasms: A qualitative study, J Geriatr Oncol, 12 (2021) 909–914.
    1. Loh KP, Ramsdale E, Culakova E, Mendler JH, Liesveld JL, O'Dwyer KM, McHugh C, Gilles M, Lloyd T, Goodman M, Klepin HD, Mustian KM, Schnall R, Mohile SG, Novel mHealth App to Deliver Geriatric Assessment-Driven Interventions for Older Adults With Cancer: Pilot Feasibility and Usability Study, JMIR Cancer, 4 (2018) e10296.
    1. Akdur G, Aydin MN, Akdur G, Adoption of Mobile Health Apps in Dietetic Practice: Case Study of Diyetkolik, JMIR Mhealth Uhealth, 8 (2020) e16911.
    1. Wang C, Qi H, Influencing Factors of Acceptance and Use Behavior of Mobile Health Application Users: Systematic Review, Healthcare (Basel), 9 (2021).
    1. Scolari C, The sense of the interface: Applying semiotics to HCI research, 2009 (2009) 1–27.
    1. Crane D, Garnett C, Brown J, West R, Michie S, Factors Influencing Usability of a Smartphone App to Reduce Excessive Alcohol Consumption: Think Aloud and Interview Studies, Front Public Health, 5 (2017) 39.
    1. Furlow B mHealth apps may make chronic disease management easier. Clinical Advisor. November 2, 2012. .
    1. Davis R, Gardner J, Schnall R, A Review of Usability Evaluation Methods and Their Use for Testing eHealth HIV Interventions, Curr HIV/AIDS Rep, 17 (2020) 203–218.
    1. Jake-Schoffman DE, Silfee VJ, Waring ME, Boudreaux ED, Sadasivam RS, Mullen SP, Carey JL, Hayes RB, Ding EY, Bennett GG, Pagoto SL, Methods for Evaluating the Content, Usability, and Efficacy of Commercial Mobile Health Apps, JMIR Mhealth Uhealth, 5 (2017) e190.
    1. Thyvalikakath TP, Monaco V, Thambuganipalle H, Schleyer T, Comparative study of heuristic evaluation and usability testing methods, Stud Health Technol Inform, 143 (2009) 322–327.
    1. Bailey RW, Wolfson CA, Nall J, Koyani S, Performance-Based Usability Testing: Metrics That Have the Greatest Impact for Improving a System's Usability, HCI, 2009.
    1. Brooke J SUS-A quick and dirty usability scale. Usability Eval Ind. 1996;189:4–7.
    1. Lewis JR, IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use, International Journal of Human–Computer Interaction, 7 (1995) 57–78.
    1. Schnall R, Cho H, Liu J, Health Information Technology Usability Evaluation Scale (Health-ITUES) for Usability Assessment of Mobile Health Technology: Validation Study, JMIR Mhealth Uhealth, 6 (2018) e4.
    1. Zhou L, Bao J, Setiawan IMA, Saptono A, Parmanto B, The mHealth App Usability Questionnaire (MAUQ): Development and Validation Study, JMIR Mhealth Uhealth, 7 (2019) e11500.
    1. Sauro J, Lewis JR, Chapter 2 - Quantifying user research, in: Sauro J, Lewis JR (Eds.) Quantifying the User Experience (Second Edition), Morgan Kaufmann, Boston, 2016, pp. 9–18.
    1. Yen PY, Wantland D, Bakken S, Development of a Customizable Health IT Usability Evaluation Scale, AMIA Annu Symp Proc, 2010 (2010) 917–921.
    1. Davis FD, Perceived usefulness, perceived ease of use, and user acceptance of information technology, MIS Quarterly, 13 (1989) 319–340.
    1. Yen P-Y, Health information technology usability evaluation: methods, models, and measures, 2010.
    1. Yen PY, Sousa KH, Bakken S, Examining construct and predictive validity of the Health-IT Usability Evaluation Scale: confirmatory factor analysis and structural equation modeling results, J Am Med Inform Assoc, 21 (2014) e241–248.
    1. Lee J, Schnall R. Validity and Reliability of the Korean Version of the Health-ITUES Scale for Usability Assessments of Mobile Health Technology. [In Review].
    1. Flynn G, Jia H, Reynolds NR, Mohr DC, Schnall R, Protocol of the randomized control trial: the WiseApp trial for improving health outcomes in PLWH (WiseApp), BMC Public Health, 20 (2020) 1775.
    1. Weiss BD, Mays MZ, Martz W, Castro KM, DeWalt DA, Pignone MP, Mockbee J, Hale FA, Quick assessment of literacy in primary care: the newest vital sign, Ann Fam Med, 3 (2005) 514–522.
    1. Parker RM, Baker DW, Williams MV, Nurss JR, The test of functional health literacy in adults: a new instrument for measuring patients' literacy skills, J Gen Intern Med, 10 (1995) 537–541.
    1. Model-Building Strategies and Methods for Logistic Regression, Applied Logistic Regression 2000, pp. 91–142.
    1. Faraggi D, Simon R, A simulation study of cross-validation for selecting an optimal cutpoint in univariate survival analysis, Stat Med, 15 (1996) 2203–2213.
    1. Faraggi D, Simon R, A SIMULATION STUDY OF CROSS-VALIDATION FOR SELECTING AN OPTIMAL CUTPOINT IN UNIVARIATE SURVIVAL ANALYSIS, Statistics in Medicine, 15 (1996) 2203–2213.
    1. SAS. Statistical Analysis Software. Users’ Guide Statistics Version 9.4. SAS Institute Inc., Cary. 2013.
    1. Kuhns LM, Hereth J, Garofalo R, Hidalgo M, Johnson AK, Schnall R, Reisner SL, Belzer M, Mimiaga MJ, A Uniquely Targeted, Mobile App-Based HIV Prevention Intervention for Young Transgender Women: Adaptation and Usability Study, J Med Internet Res, 23 (2021) e21839.
    1. Stonbraker S, Cho H, Hermosi G, Pichon A, Schnall R, Usability Testing of a mHealth App to Support Self-Management of HIV-Associated Non-AIDS Related Symptoms, Stud Health Technol Inform, 250 (2018) 106–110.
    1. Salz T, Schnall RB, McCabe MS, Oeffinger KC, Corcoran S, Vickers AJ, Salner AL, Dornelas E, Raghunathan NJ, Fortier E, McKiernan J, Finitsis DJ, Chimonas S, Baxi S, Incorporating Multiple Perspectives Into the Development of an Electronic Survivorship Platform for Head and Neck Cancer, JCO Clin Cancer Inform, 2 (2018) 1–15.
    1. Schnall R, Rojas M, Bakken S, Brown W, Carballo-Dieguez A, Carry M, Gelaude D, Mosley JP, Travers J, A user-centered model for designing consumer mobile health (mHealth) applications (apps), J Biomed Inform, 60 (2016) 243–251.
    1. Cho H, Powell D, Pichon A, Kuhns LM, Garofalo R, Schnall R, Eye-tracking retrospective think-aloud as a novel approach for a usability evaluation, Int J Med Inform, 129 (2019) 366–373.
    1. Schnall R, John RM, Carballo-Dieguez A, Do High-Risk Young Adults Use the HIV Self-Test Appropriately? Observations from a Think-Aloud Study, AIDS Behav, 20 (2016) 939–948.
    1. Cho H, Yen PY, Dowding D, Merrill JA, Schnall R, A multi-level usability evaluation of mobile health applications: A case study, J Biomed Inform, 86 (2018) 79–89.
    1. Beauchemin M, Gradilla M, Baik D, Cho H, Schnall R, A Multi-step Usability Evaluation of a Self-Management App to Support Medication Adherence in Persons Living with HIV, Int J Med Inform, 122 (2019) 37–44.
    1. Sauro J Measuring Usability with the System Usability Scale (SUS). February 3, 2011. Available at: Accessed on July 27, 2021.

Source: PubMed

Спонсоры и соавторы

Заболевания

Медикаментозные вмешательства

Подписаться