Accuracy of Vital Signs Measurements by a Smartwatch and a Portable Health Device: Validation Study

Abstract

Background: New consumer health devices are being developed to easily monitor multiple physiological parameters on a regular basis. Many of these vital sign measurement devices have yet to be formally studied in a clinical setting but have already spread widely throughout the consumer market.

Objective: The aim of this study was to investigate the accuracy and precision of heart rate (HR), systolic blood pressure (SBP), diastolic blood pressure (DBP), and oxygen saturation (SpO2) measurements of 2 novel all-in-one monitoring devices, the BodiMetrics Performance Monitor and the Everlast smartwatch.

Methods: We enrolled 127 patients (>18 years) from the Thomas Jefferson University Hospital Preadmission Testing Center. SBP and HR were measured by both investigational devices. In addition, the Everlast watch was utilized to measure DBP, and the BodiMetrics Performance Monitor was utilized to measure SpO2. After 5 min of quiet sitting, four hospital-grade standard and three investigational vital sign measurements were taken, with 60 seconds in between each measurement. The reference vital sign measurements were calculated by determining the average of the two standard measurements that bounded each investigational measurement. Using this method, we determined three comparison pairs for each investigational device in each subject. After excluding data from 42 individuals because of excessive variation in sequential standard measurements per prespecified dropping rules, data from 85 subjects were used for final analysis.

Results: Of 85 participants, 36 (42%) were women, and the mean age was 53 (SD 21) years. The accuracy guidelines were only met for the HR measurements in both devices. SBP measurements deviated 16.9 (SD 13.5) mm Hg and 5.3 (SD 4.7) mm Hg from the reference values for the Everlast and BodiMetrics devices, respectively. The mean absolute difference in DBP measurements for the Everlast smartwatch was 8.3 (SD 6.1) mm Hg. The mean absolute difference between BodiMetrics and reference SpO2 measurements was 3.02%.

Conclusions: Both devices we investigated met accuracy guidelines for HR measurements, but they failed to meet the predefined accuracy guidelines for other vital sign measurements. Continued sale of consumer physiological monitors without prior validation and approval procedures is a public health concern.

Keywords: mHealth; measurements validity; medical devices; vital signs.

Conflict of interest statement

Conflicts of Interest: JIJ is a founder and equity owner and has received research support from RTM Vital Signs LLC, a company developing a noninvasive and long-term implantable vital sign monitoring system with real-time diagnostic algorithms and that can transfer data via cell phone to a central monitoring system. JIJ and NVH have pending patent applications related to vital sign measurement. The other authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript.

©Christina Hahnen, Cecilia G Freeman, Nilanjan Haldar, Jacquelyn N Hamati, Dylan M Bard, Vignesh Murali, Geno J Merli, Jeffrey I Joseph, Noud van Helmond. Originally published in JMIR mHealth and uHealth (http://mhealth.jmir.org), 12.02.2020.

Figures

Figure 1

BodiMetrics Performance Monitor tricorder. Vital sign measurements are performed by placing the right index finger on the plethysmography sensor in the right upper corner under the flap. In addition, contact has to be made with the electrocardiogram electrodes at the front, left lateral side, and back using both hands.

Figure 2

Everlast smartwatch. To enable a physiological measurement, the watch must be worn on the bare wrist making contact with the skin. Measurements are initiated by pressing the button on the right side of the watch.

Figure 3

Study timeline. B: BodiMetrics Performance Monitor measurement; E: Everlast smartwatch measurement; S: Standard measurement (light grey color). The dark grey color indicates the investigational devices.

Figure 4

Systolic blood pressure (A and B), heart rate (C and D), and oxygen saturation (E and F) measurements by BodiMetrics tricorder and reference values. SBP: systolic blood pressure, HR: heart rate, SpO2: oxygen saturation.

Figure 5

Systolic blood pressure (A and B), diastolic blood pressure (C and D), and heart rate (E and F) measurements by Everlast smartwatch and reference values. BP: systolic blood pressure, DBP: diastolic blood pressure, HR: heart rate.

Figure 6

Difference between BodiMetrics tricorder systolic blood pressure measurements and calibration measurement versus difference between reference systolic blood pressure measurements and calibration measurement. The solid line in the scatterplot represents the line of identity. Data shown were not normally distributed (Shapiro-Wilk test).