Field-based Measurement of Sleep: Agreement between Six Commercial Activity Monitors and a Validated Accelerometer

Abstract

Objective: To examine agreement between multiple commercial activity monitors (CAMs) and a validated actigraph to measure sleep.

Methods: Thirty adults without sleep disorders wore an Actiwatch Spectrum (AW) and alternated wearing 6 CAMs for one 24-h period each (Fitbit Alta, Jawbone Up3, Misfit Shine 2, Polar A360, Samsung Gear Fit2, Xiaomi Mi Band 2). Total sleep time (TST) and wake after sleep onset (WASO) were compared between edited AW and unedited CAM outputs. Comparisons between AW and CAM data were made via paired t-tests, mean absolute percent error (MAPE) calculations, and intra-class correlations (ICC). Intra-model reliability was performed in 10 participants who wore a pair of each AW and CAM model.

Results: Fitbit, Jawbone, Misfit, and Xiaomi overestimated TST relative to AW (53.7-80.4 min, P ≤ .001). WASO was underestimated by Fitbit, Misfit, Samsung and Xiaomi devices (15.0-27.9 min; P ≤ .004) and overestimated by Polar (27.7 min, P ≤ .001). MAPEs ranged from 5.1% (Samsung) to 25.4% (Misfit) for TST and from 36.6% (Fitbit) to 165.1% (Polar) for WASO. TST ICCs ranged from .00 (Polar) to .92 (Samsung), while WASO ICCs ranged from .38 (Misfit) to .69 (Samsung). Differences were similar between poor sleepers (Pittsburgh Sleep Quality Index global score >5; n = 10) and good sleepers. Intra-model reliability analyses revealed minimal between-pair differences and high ICCs.

Conclusions: Agreement between CAMs and AW varied by device, with greater agreement observed for TST than WASO. While reliable, variability in agreement across CAMs with traditional actigraphy may complicate the interpretation of CAM data obtained for clinical or research purposes.

Figures

Figure 1.. Mean Absolute Percent Error for comparisons between unedited commercial activity monitor data relative to edited Actiwatch data.

Panel A: total sleep time; panel B: wake after sleep onset. Data are shown as mean±standard deviation. Abbreviations: SD=standard deviation; TST=total sleep time; WASO=wake after sleep onset.

Figure 2.. Bland-Altman plots for unedited commercial activity monitor vs. edited Actiwatch total sleep time.

Data are shown in minutes. X-axis indicates the mean total sleep time of the commercial activity monitor and Actiwatch (CAM & AW). Y-axis indicates the difference in total sleep time between the commercial activity monitor and Actiwatch (CAM-AW). Closed circles=good sleepers (i.e., Pittsburgh Sleep Quality Index [PSQI] global score ≤ 5); open circles=poor sleepers (i.e., PSQI global score > 5). Abbreviations: AW=Actiwatch; CAM=commercial activity monitor; TST=total sleep time.

Figure 3.. Bland-Altman plots for unedited commercial activity monitor vs. edited Actiwatch wake after sleep onset.

Data are shown in minutes. X-axis indicates the mean wake after sleep onset of the commercial activity monitor and Actiwatch (CAM & AW). Y-axis indicates the difference in wake after sleep onset between the commercial activity monitor and actiwatch (CAM-AW). Closed circles=good sleepers (i.e., Pittsburgh Sleep Quality Index [PSQI] global score ≤ 5); open circles=poor sleepers (i.e., PSQI global score > 5). Abbreviations: AW=Actiwatch; CAM=commercial activity monitor; WASO=wake after sleep onset.