Usability study of a computer-based self-management system for older adults with chronic diseases

Calvin Or, Da Tao, Calvin Or, Da Tao

Abstract

Background: Usability can influence patients' acceptance and adoption of a health information technology. However, little research has been conducted to study the usability of a self-management health care system, especially one geared toward elderly patients.

Objective: This usability study evaluated a new computer-based self-management system interface for older adults with chronic diseases, using a paper prototype approach.

Methods: Fifty older adults with different chronic diseases participated. Two usability evaluation methods were involved: (1) a heuristics evaluation and (2) end-user testing with a think-aloud testing method, audio recording, videotaping, and interviewing. A set of usability metrics was employed to determine the overall system usability, including task incompletion rate, task completion time, frequency of error, frequency of help, satisfaction, perceived usefulness, and perceived ease of use. Interviews were used to elicit participants' comments on the system design. The quantitative data were analyzed using descriptive statistics and the qualitative data were analyzed for content.

Results: The participants were able to perform the predesigned self-management tasks with the current system design and they expressed mostly positive responses about the perceived usability measures regarding the system interface. However, the heuristics evaluation, performance measures, and interviews revealed a number of usability problems related to system navigation, information search and interpretation, information presentation, and readability. Design recommendations for further system interface modifications were discussed.

Conclusions: This study verified the usability of the self-management system developed for older adults with chronic diseases. Also, we demonstrated that our usability evaluation approach could be used to quickly and effectively identify usability problems in a health care information system at an early stage of the system development process using a paper prototype. Conducting a usability evaluation is an essential step in system development to ensure that the system features match the users' true needs, expectations, and characteristics, and also to minimize the likelihood of the users committing user errors and having difficulties using the system.

Keywords: Usability evaluation; chronic disease; patient participation; self-management.

Conflict of interest statement

Conflicts of Interest: None declared.

Figures

Figure 1
Figure 1
The blood pressure measurement page of the self-management system.
Figure 2
Figure 2
The blood pressure history data page presents the past blood pressure values on a two-dimensional line chart.
Figure 3
Figure 3
The blood glucose module menu includes the six buttons for the selection of the six test times of blood glucose.

References

    1. Jimison H, Gorman P, Woods S, Nygren P, Walker M, Norris S, Hersh W. Barriers and drivers of health information technology use for the elderly, chronically ill, and underserved. Evid Rep Technol Assess (Full Rep) 2008 Nov;(175):1–1422.
    1. Or CK, Karsh BT. A systematic review of patient acceptance of consumer health information technology. J Am Med Inform Assoc. 2009;16(4):550–60. doi: 10.1197/jamia.M2888.
    1. Kaufman D, Roberts WD, Merrill J, Lai TY, Bakken S. Applying an evaluation framework for health information system design, development, and implementation. Nurs Res. 2006;55(2 Suppl):S37–42.
    1. Nielsen J. Usability Engineering. San Francisco, CA: Morgan Kaufmann; 1993.
    1. Gosbee J. Human factors engineering and patient safety. Qual Saf Health Care. 2002 Dec;11(4):352–4.
    1. Kushniruk A. Evaluation in the design of health information systems: application of approaches emerging from usability engineering. Comput Biol Med. 2002 May;32(3):141–9.
    1. Karsh BT. Beyond usability: designing effective technology implementation systems to promote patient safety. Qual Saf Health Care. 2004 Oct;13(5):388–94. doi: 10.1136/qhc.13.5.388.
    1. Edwards PJ, Moloney KP, Jacko JA, Sainfort F. Evaluating usability of a commercial electronic health record: a case study. Int J Hum-Comput St. 2008;66:718–728. doi: 10.1016/j.ijhcs.2008.06.002.
    1. Linder JA, Rose AF, Palchuk MB, Chang F, Schnipper JL, Chan JC, Middleton B. Decision support for acute problems: the role of the standardized patient in usability testing. J Biomed Inform. 2006 Dec;39(6):648–55. doi: 10.1016/j.jbi.2005.12.002.
    1. McDaniel AM, Hutchison S, Casper GR, Ford RT, Stratton R, Rembusch M. Usability testing and outcomes of an interactive computer program to promote smoking cessation in low income women. Proc AMIA Symp. 2002:509–13.
    1. Rose AF, Schnipper JL, Park ER, Poon EG, Li Q, Middleton B. Using qualitative studies to improve the usability of an EMR. J Biomed Inform. 2005 Feb;38(1):51–60. doi: 10.1016/j.jbi.2004.11.006.
    1. Tang Z, Johnson TR, Tindall RD, Zhang J. Applying heuristic evaluation to improve the usability of a telemedicine system. Telemed J E Health. 2006 Feb;12(1):24–34. doi: 10.1089/tmj.2006.12.24.
    1. Yen PY, Bakken S. A comparison of usability evaluation methods: heuristic evaluation versus end-user think-aloud protocol - an example from a web-based communication tool for nurse scheduling. AMIA Annu Symp Proc. 2009;2009:714–8.
    1. Yen PY, Gorman P. Usability testing of digital pen and paper system in nursing documentation. AMIA Annu Symp Proc. 2005:844–8.
    1. Aguero-Torres H, Hilleras PK, Winblad B. Disability in activities of daily living among the elderly. Current Opinion in Psychiatry. 2001;14:355–9.
    1. Atkinson HH, Cesari M, Kritchevsky SB, Penninx BW, Fried LP, Guralnik JM, Williamson JD. Predictors of combined cognitive and physical decline. J Am Geriatr Soc. 2005 Jul;53(7):1197–202. doi: 10.1111/j.1532-5415.2005.53362.x.
    1. Fisk AD, Rogers WA, Charness N, Czaja SJ, Sharit J. Designing for Older Adults: Principles and Creative Human Factors Approaches. Hoboken, NJ: CRC Press; 2009.
    1. Vanderheiden GC. Salvendy G. editor. Handbook of Human Factors and Ergonomics. New York, NY: John Wiley & Sons; 2006. Design for people with functional limitations; pp. 1387–417.
    1. Czaja SJ, Lee CC. Sears A, Jacko J. editors. The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications. New York, NY: Lawrence Erlbaum and Associates; 2007. Information technology and older adults; pp. 777–92.
    1. Strothotte T, Fritz S, Michel R, Raab A, Petrie H, Johnson V. Development of dialogue systems for a mobility aid for blind people: initial design and usability testing. the Second Annual ACM Conference on Assistive Technologies; April 11-12, 1996; Vancouver, BC, Canada. New York, NY: ACM; 1996.
    1. Holzinger A, Searle G, Kleinberger T, Seffah A, Javahery H. Investigating usability metrics for the design and development of applications for the elderly. Lecture Notes in Computer Science. 2008;5105:98–105. doi: 10.1007/978-3-540-70540-6_13.
    1. Mikkonen M, Vayrynen S, Ikonen V, Heikkila MO. User and concept studies as tools in developing mobile communication services for the elderly. Personal and Ubiquitous Computing. 2002;6:113–124. doi: 10.1007/s007790200010.
    1. Ballinger C, Pickering RM, Bannister S, Gore S, McLellan DL. Evaluating equipment for people with disabilities: user and technical perspectives on basic commodes. Clinical Rehabilitation. 1995;9:157–166. doi: 10.1177/026921559500900211.
    1. Demiris G, Oliver DP, Dickey G, Skubic M, Rantz M. Findings from a participatory evaluation of a smart home application for older adults. Technol Health Care. 2008;16(2):111–8.
    1. Good A, Stokes S, Jerrams-Smith J. Elderly, novice users and health information web sites: issues of accessibility and usability. J Healthc Inf Manag. 2007;21(3):72–9.
    1. Hung WW, Ross JS, Boockvar KS, Siu AL. Recent trends in chronic disease, impairment and disability among older adults in the United States. BMC Geriatr. 2011 Aug;11(47) doi: 10.1186/1471-2318-11-47.
    1. Jackson CL, Bolen S, Brancati FL, Batts-Turner ML, Gary TL. A systematic review of interactive computer-assisted technology in diabetes care: interactive information technology in diabetes care. J Gen Intern Med. 2006 Feb;21(2):105–10. doi: 10.1111/j.1525-1497.2005.00310.x.
    1. Goldberg LR, Piette JD, Walsh MN, Frank TA, Jaski BE, Smith AL, Rodriguez R, Mancini DM, Hopton LA, Orav EJ, Loh E, WHARF Investigators Randomized trial of a daily electronic home monitoring system in patients with advanced heart failure: the Weight Monitoring in Heart Failure (WHARF) trial. Am Heart J. 2003 Oct;146(4):705–12. doi: 10.1016/S0002-8703(03)00393-4.
    1. Rubin J, Chisnell D, Spool JM. Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. Indianapolis, IN: Wiley Pub; 2008.
    1. Tan WS, Liu D, Bishu R. Web evaluation: Heuristic evaluation vs. user testing. Int J Ind Ergon. 2009;39(4):621–7. doi: 10.1016/j.eururo.2012.06.052.
    1. Gosbee J, Gosbee LL. Carayon P, ditor. Handbook of Human Factors and Ergonomics in Health Care and Patient Safety. Boca Raton, FL: CRC Press; 2011. Usability evaluation in health care; pp. 543–556.
    1. Nielsen J, Molich R. Heuristic evaluation of user interfaces. CHI Conference on Human Factors in Computing; April 01-05, 1990; Seattle, WA. New York, NY: ACM Press; 1990. pp. 249–256.
    1. Jaspers MW. A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence. Int J Med Inform. 2009 May;78(5):340–53. doi: 10.1016/j.ijmedinf.2008.10.002.
    1. Snyder C. Paper Prototyping: The Fast and Easy Way to Design and Refine User Interfaces. San Francisco, CA: Morgan Kaufmann; 2003.
    1. Rudd J, Stern K, Isensee S. Low vs. high-fidelity prototyping debate. interactions. 1996;3(1):76–85. doi: 10.1145/223500.223514.
    1. Walker M, Takayama L, Landay JA. High-fidelity or low-fidelity, paper or computer? Choosing attributes when testing web prototypes. the Human Factors and Ergonomics Society 46th Annual Meeting; Sep 30-Oct 4, 2002; Baltimore, MD. 2002. pp. 661–5.
    1. Virzi RA, Sokolov JL, Karis D. Usability problem identification using both low- and high-fidelity prototypes. the SIGCHI Conference on Human Factors in Computing Systems; April 13-18, 1996; Vancouver, BC, Canada. 1996. pp. 236–43.
    1. Olmsted-Hawala EL, Romano JC, Murphy ED. The use of paper-prototyping in a low-fidelity usability study. IEEE International Professional Communication Conference; July 19-22, 2009; Waikiki, HI. 2009. pp. 1–11.
    1. Shneiderman B, Plaisant C. Designing the User Interface: Strategies for Effective Human-Computer Interaction. Boston, MA: Addison Wesley; 2005.
    1. Sanders MS, McCormick EJ. Human Factors in Engineering and Design. New York, NY: McGraw-Hill; 1993.
    1. Demiris G, Finkelstein SM, Speedie SM. Considerations for the design of a Web-based clinical monitoring and educational system for elderly patients. J Am Med Inform Assoc. 2001;8(5):468–72.
    1. Nielsen J, Mack RL. Usability Inspection Methods. New York, NY: Wiley; 1994.
    1. Grenier AS, Carayon P, Casper GR, Or CKL, Burke LJ, Brennan PF. Usability evaluation of an Internet-based health information/communication system for CHF patients. the 16th Triennial World Congress of the International Ergonomics Association; July 10-14, 2006; Maastricht, the Netherlands. 2006. pp. 10–14.
    1. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly. 1989;13(3):319–340. doi: 10.2307/249008.
    1. Krippendorff K. Content Analysis: An Introduction to Its Methodology. Thousand Oaks, CA: Sage Publications, Inc; 2012.
    1. Hick WE. On the rate of gain of information. Q J Exp Psychol. 1952;4(1):11–26. doi: 10.1080/17470215208416600.
    1. Hyman R. Stimulus information as a determinant of reaction time. J Exp Psychol. 1953 Mar;45(3):188–96.
    1. Catani MB, Biers DW. Usability evaluation and prototype fidelity: users and usability professionals. Human Factors and Ergonomics Society 42nd Annual Meeting; October 5-9, 1998; Chicago, IL. 1998. pp. 1331–5.
    1. Sefelin R, Tscheligi M, Giller V. Paper prototyping - what is it good for? A comparison of paper- and computer-based low-fidelity prototyping. CHI Conference on Human Factors in Computing Systems; April 5-10, 2003; Fort Lauderdale, FL. 2003. pp. 778–9.
    1. Yen PY, Bakken S. Review of health information technology usability study methodologies. J Am Med Inform Assoc. 2012;19(3):413–22. doi: 10.1136/amiajnl-2010-000020.
    1. Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, Morton SC, Shekelle PG. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006 May;144(10):742–52.
    1. Jamal A, McKenzie K, Clark M. The impact of health information technology on the quality of medical and health care: a systematic review. HIM J. 2009;38(3):26–37.

Source: PubMed

Sponsoren und Mitarbeiter

Krankheiten

Drogeninterventionen

3
Abonnieren