Evaluation framework for selecting wearable activity monitors for research

Abstract

Background: Wearable devices that support activity tracking and other measurements hold great potential to increase awareness of health behaviors and support the management of chronic health conditions. There is a scarcity of guidance for researchers of all disciplines when planning new studies to evaluate and select technologies appropriate for study purpose, population, and overall context. The aim of this study was to develop and test an evaluation framework to rapidly and systematically evaluate and select consumer-grade wearable devices that serve individual study needs in preparation for evaluations with target populations.

Methods: The wearable evaluation framework was defined based on published literature and past research experiences of the research team. We tested the framework with example case studies to select devices for two different research projects focused on aging-in-place and gestational diabetes. We show how knowledge of target population and research goals help prioritize application of the criteria to inform device selection and how project requirements inform sequence of criteria application.

Results: The framework for wearable device evaluation includes 27 distinct evaluation criteria: 12 for everyday use by users, 6 on device functionality, and 9 on infrastructure for developing the research infrastructure required to obtain the data. We evaluated 10 devices from four vendors. After prioritizing the framework criteria based on the two example case studies, we selected the Withings Steele HR, Garmin Vivosmart HR+ and Garmin Forerunner 35 for further evaluation through user studies with the target populations.

Conclusions: The aim of this paper was to develop and test a framework for researchers to rapidly evaluate suitability of consumer grade wearable devices for specific research projects. The use of this evaluation framework is not intended to identify a definitive single best device, but to systematically narrow the field of potential device candidates for testing with target study populations. Future work will include application of the framework within different research projects for further refinement.

Keywords: Wearable electronic devices; chronic disease; fitness trackers; mobile devices; research.

Conflict of interest statement

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/mhealth-19-253). The series “Real-Time Detection and Management of Chronic Illnesses” was commissioned by the editorial office without any funding or sponsorship. KC discloses National Science Foundation grant award 1405723 during the conduct of the study. KS reports grants from National Science Foundation, grants from National Science Foundation, grants from Indiana University, during the conduct of the study. The authors have no other conflicts of interest to declare.

2021 mHealth. All rights reserved.

Figures

Figure 1

Process for selecting wearable device for research study. This paper describes the evaluation framework, used in the early stages of devices selection to move from many devices to a few devices.