Agreement of Sleep Measures-A Comparison between a Sleep Diary and Three Consumer Wearable Devices

Kristina Klier, Matthias Wagner, Kristina Klier, Matthias Wagner

Abstract

Nowadays, self-tracking and optimization are widely spread. As sleep is essential for well-being, health, and peak performance, the number of available consumer technologies to assess individual sleep behavior is increasing rapidly. However, little is known about the consumer wearables' usability and reliability for sleep tracking. Therefore, the aim of the present study was to compare the sleep measures of wearable devices with a standardized sleep diary in young healthy adults in free-living conditions. We tracked night sleep from 30 participants (19 females, 11 males; 24.3 ± 4.2 years old). Each wore three wearables and simultaneously assessed individual sleep patterns for four consecutive nights. Wearables and diaries correlated substantially regarding time in bed (Range CCCLin: 0.74-0.84) and total sleep time (Range CCCLin: 0.76-0.85). There was no sufficient agreement regarding the measures of sleep efficiency (Range CCCLin: 0.05-0.34) and sleep interruptions (Range CCCLin: -0.02-0.10). Finally, these results show wearables to be an easy-to-handle, time- and cost-efficient alternative to tracking sleep in healthy populations. Future research should develop and empirically test the usability of such consumer sleep technologies.

Keywords: concordance; self-tracking; sleep; sleep assessment; sleep diaries; wearable devices.

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Flow Chart of the examined study protocol.
Figure 2
Figure 2
Overview of the analyzed sleep variables presenting time parameters in minutes and sleep efficiency in percentage.
Figure 3
Figure 3
Bland-Altman plot for TIB Fitbit® vs. diary.
Figure 4
Figure 4
Bland-Altman plot for TIB Garmin® vs. diary.
Figure 5
Figure 5
Bland-Altman plot for TIB Polar® vs. diary.
Figure 6
Figure 6
Bland-Altman plot for TST Fitbit® vs. diary.
Figure 7
Figure 7
Bland-Altman Plot for TST Garmin® vs. diary.
Figure 8
Figure 8
Bland-Altman plot for TST Polar® vs. diary.
Figure 9
Figure 9
Bland-Altman plot for SE Fitbit® vs. diary.
Figure 10
Figure 10
Bland-Altman plot for SE Garmin® vs. diary.
Figure 11
Figure 11
Bland-Altman plot for SE Polar® vs. diary.
Figure 12
Figure 12
Bland-Altman Plot for WASO Fitbit® vs. diary.
Figure 13
Figure 13
Bland-Altman Plot for WASO Garmin® vs. diary.
Figure 14
Figure 14
Bland-Altman Plot for WASO Polar® vs. diary.

References

    1. Halson S.L., Juliff L.E. Sleep, sport, and the brain. Prog. Brain Res. 2017;234:13–31. doi: 10.1016/bs.pbr.2017.06.006.
    1. Goel N., Basner M., Rao H., Dinges D.F. Circadian rhythms, sleep deprivation, and human performance. Prog. Mol. Biol. Transl. Sci. 2013;119:155–190. doi: 10.1016/B978-0-12-396971-2.00007-5.
    1. Grandner M.A. Sleep, Health, and Society. Sleep Med. Clin. 2017;12:1–22. doi: 10.1016/j.jsmc.2016.10.012.
    1. Buysse D.J. Sleep health: Can we define it? Does it matter? Sleep. 2014;37:9–17. doi: 10.5665/sleep.3298.
    1. Mackert M., Mabry-Flynn A., Champlin S., Donovan E.E., Pounders K. Health Literacy and Health Information Technology Adoption: The Potential for a New Digital Divide. J. Med. Internet Res. 2016;18:e264. doi: 10.2196/jmir.6349.
    1. Kitson A., Prpa M., Riecke B.E. Immersive Interactive Technologies for Positive Change: A Scoping Review and Design Considerations. Front. Psychol. 2018;9:1354. doi: 10.3389/fpsyg.2018.01354.
    1. Mansukhani M.P., Kolla B.P. Apps and fitness trackers that measure sleep: Are they useful? Clevel. Clin. J. Med. 2017;84:451–456. doi: 10.3949/ccjm.84a.15173.
    1. Peake J.M., Kerr G., Sullivan J.P. A Critical Review of Consumer Wearables, Mobile Applications, and Equipment for Providing Biofeedback, Monitoring Stress, and Sleep in Physically Active Populations. Front. Physiol. 2018;9:743. doi: 10.3389/fphys.2018.00743.
    1. Evenson K.R., Goto M.M., Furberg R.D. Systematic review of the validity and reliability of consumer-wearable activity trackers. Int. J. Behav. Nutr. Phys. Act. 2015;12:159. doi: 10.1186/s12966-015-0314-1.
    1. Fuller D., Colwell E., Low J., Orychock K., Tobin M.A., Simango B., Buote R., van Heerden D., Luan H., Cullen K., et al. Reliability and Validity of Commercially Available Wearable Devices for Measuring Steps, Energy Expenditure, and Heart Rate: Systematic Review. JMIR MHealth UHealth. 2020;8:e18694. doi: 10.2196/18694.
    1. Grifantini K. How’s my sleep? Personal sleep trackers are gaining in popularity, but their accuracy is still open to debate. IEEE Pulse. 2014;5:14–18. doi: 10.1109/MPUL.2014.2339252.
    1. Shin G., Jarrahi M.H., Fei Y., Karami A., Gafinowitz N., Byun A., Lu X. Wearable activity trackers, accuracy, adoption, acceptance and health impact: A systematic literature review. J. Biomed. Inform. 2019;93:103153. doi: 10.1016/j.jbi.2019.103153.
    1. Grandner M.A., Rosenberger M.E. Actigraphic sleep tracking and wearables: Historical context, scientific applications and guidelines, limitations, and considerations for commercial sleep devices. In: Grandner M.A., Allen S.F., editors. Sleep and Health. Academic Press Elsevier; Amsterdam, The Netherlands: 2019. pp. 147–157.
    1. Lu L., Zhang J., Xie Y., Gao F., Xu S., Wu X., Ye Z. Wearable Health Devices in Health Care: Narrative Systematic Review. JMIR MHealth UHealth. 2020;8:e18907. doi: 10.2196/18907.
    1. Lyons E., Lang M., Ridgers N.D., Timperio A., Brown H., Ball K., Macfarlane S., Lai S.K., Richards K., Mackintosh K.A., et al. Wearable Activity Tracker Use Among Australian Adolescents: Usability and Acceptability Study. JMIR MHealth UHealth. 2018;6:e86. doi: 10.2196/mhealth.9199.
    1. De Arriba-Pérez F., Caeiro-Rodríguez M., Santos-Gago J.M. How do you sleep? Using off the shelf wrist wearables to estimate sleep quality, sleepiness level, chronotype and sleep regularity indicators. J. Ambient Intell. Humaniz. Comput. 2018;9:897–917. doi: 10.1007/s12652-017-0477-5.
    1. Chinoy E.D., Huwa K.E., Snider M.N., Bessman S.C., Cuellar J.A., Cooper A.D., Hirsch D.A., Nardulli P.J., Jameson J.T., Drummond S.P.A., et al. Examination of Wearable and Non-Wearable Consumer Sleep-Tracking Devices Versus Polysomnography. Sleep. 2019;42:A403–A404. doi: 10.1093/sleep/zsz067.999.
    1. Crivello A., Barsocchi P., Girolami M., Palumbo F. The Meaning of Sleep Quality: A Survey of Available Technologies. IEEE Access. 2019;7:167374–167390. doi: 10.1109/ACCESS.2019.2953835.
    1. Kolla B.P., Mansukhani S., Mansukhani M.P. Consumer sleep tracking devices: A review of mechanisms, validity and utility. Expert Rev. Med. Devices. 2016;13:497–506. doi: 10.1586/17434440.2016.1171708.
    1. Liang Z., Chapa Martell M.A. Validity of Consumer Activity Wristbands and Wearable EEG for Measuring Overall Sleep Parameters and Sleep Structure in Free-Living Conditions. J. Healthc. Inform. Res. 2018;2:152–178. doi: 10.1007/s41666-018-0013-1.
    1. Manuta J., Gravel N., Spencer R.M.C. Reliability of Sleep Measures from Four Personal Health Monitoring Devices Compared to Research-Based Actigraphy and Polysomnography. Sensors. 2016;16:646. doi: 10.3390/s16050646.
    1. Menghini L., Cellini N., Goldstone A., Baker F.C., de Zambotti M. A standardized framework for testing the performance of sleep-tracking technology: Step-by-step guidelines and open-source code. Sleep. 2020;44:zsaa170. doi: 10.1093/sleep/zsaa170.
    1. Muzet A., Werner S., Fuchs G., Roth T., Saoud J.B., Viola A.U., Schaffhauser J.Y., Luthringer R. Assessing sleep architecture and continuity measures through the analysis of heart rate and wrist movement recordings in healthy subjects: Comparison with results based on polysomnography. Sleep Med. 2016;21:47–56. doi: 10.1016/j.sleep.2016.01.015.
    1. Roomkham S., Lovell D., Cheung J., Perrin D. Promises and Challenges in the Use of Consumer-Grade Devices for Sleep Monitoring. IEEE Rev. Biomed. Eng. 2018;11:53–67. doi: 10.1109/RBME.2018.2811735.
    1. Williamson J., Liu Q., Lu F., Mohrman W., Li K., Dick R., Shang L. Data Sensing and Analysis: Challenges for Wearables; Proceedings of the 20th Asia and South Pacific Design Automation Conference; Chiba, Japan. 19–22 January 2015; pp. 136–141.
    1. Baron K.G., Duffecy J., Berendsen M.A., Cheung Mason I., Lattie E.G., Manalo N.C. Feeling validated yet? A scoping review of the use of consumer-targeted wearable and mobile technology to measure and improve sleep. Sleep Med. Rev. 2018;40:151–159. doi: 10.1016/j.smrv.2017.12.002.
    1. De Zambotti M., Cellini N., Goldstone A., Colrain I.M., Baker F.C. Wearable Sleep Technology in Clinical and Research Settings. Med. Sci. Sports Exerc. 2019;51:1538–1557. doi: 10.1249/MSS.0000000000001947.
    1. Perez-Pozuelo I., Zhai B., Palotti J., Mall R., Aupetit M., Garcia-Gomez J.M., Taheri S., Guan Y., Fernandez-Luque L. The future of sleep health: A data-driven revolution in sleep science and medicine. NPJ Digit. Med. 2020;3:42. doi: 10.1038/s41746-020-0244-4.
    1. Wiesing U., Ehni H.J. Die Deklaration von Helsinki des Weltärztebundes—Ethische Grundsätze für die Forschung am Menschen. In: Lenk C., Duttge G., Fangerau H., editors. Handbuch Ethik und Recht der Forschung am Menschen. Springer; Berlin/Heidelberg, Germany: 2014. pp. 517–524.
    1. Lee J.M., Byun W., Keill A., Dinkel D., Seo Y. Comparison of Wearable Trackers’ Ability to Estimate Sleep. Int. J. Environ. Res. Public Health. 2018;15:1265. doi: 10.3390/ijerph15061265.
    1. Faul F., Erdfelder E., Buchner A., Lang A.G. Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behav. Res. Methods. 2009;41:1149–1160. doi: 10.3758/BRM.41.4.1149.
    1. Düking P., Giessing L., Frenkel M.O., Koehler K., Holmberg H.C., Sperlich B. Wrist-Worn Wearables for Monitoring Heart Rate and Energy Expenditure While Sitting or Performing Light-to-Vigorous Physical Activity: Validation Study. JMIR Mhealth Uhealth. 2020;8:e16716. doi: 10.2196/16716.
    1. Hoffmann R.M., Müller T., Hajak G., Cassel W. Abend-Morgenprotokolle in Schlafforschung und Schlafmedizin: Ein Standardinstrument für den deutschsprachigen Raum. Somnologie. 1997;1:103–109. doi: 10.1007/s11818-997-0019-z.
    1. Bland J.M., Altman D.G. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1:307–310. doi: 10.1016/S0140-6736(86)90837-8.
    1. Lin L.I.K. A Concordance Correlation Coefficient to Evaluate Reproducibility. Biometrics. 1989;45:255. doi: 10.2307/2532051.
    1. Shapiro S.S., Wilk M.B. An analysis of variance test for normality (complete samples) Biometrika. 1965;52:591. doi: 10.1093/biomet/52.3-4.591.
    1. Breusch T.S., Pagan A.R. A simple test for heteroscedasticity and random coefficient variation. Econometrica. 1979;47:1287. doi: 10.2307/1911963.
    1. Bland J.M., Altman D.G. Measuring agreement in method comparison studies. Stat. Methods Med. Res. 1999;8:135–160. doi: 10.1177/096228029900800204.
    1. Ohayon M., Wickwire E.M., Hirshkowitz M., Albert S.M., Avidan A., Daly F.J., Dauvilliers Y., Ferri R., Fung C., Gozal D., et al. National Sleep Foundation’s sleep quality recommendations: First report. Sleep Health. 2017;3:6–19. doi: 10.1016/j.sleh.2016.11.006.
    1. Landis J.R., Koch G.G. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159. doi: 10.2307/2529310.
    1. Johnson N.L. Systems of frequency curves generated by methods of translation. Biometrika. 1949;36:149–176. doi: 10.1093/biomet/36.1-2.149.
    1. Degroote L., Hamerlinck G., Poels K., Maher C., Crombez G., de Bourdeaudhuij I., Vandendriessche A., Curtis R.G., DeSmet A. Low-cost consumer-based trackers to measure physical activity and sleep duration among adults in free-living conditions: Validation study. JMIR MHealth UHealth. 2020;8:e16674. doi: 10.2196/16674.
    1. Jungquist C.R., Pender J.J., Klingman K.J., Mund J. Validation of Capturing Sleep Diary Data via a Wrist-Worn Device. Sleep Disord. 2015;2015:758937. doi: 10.1155/2015/758937.
    1. Campanini M.Z., Lopez-Garcia E., Rodríguez-Artalejo F., González A.D., Andrade S.M., Mesas A.E. Agreement between sleep diary and actigraphy in a highly educated Brazilian population. Sleep Med. 2017;35:27–34. doi: 10.1016/j.sleep.2017.04.004.
    1. Thurman S.M., Wasylyshyn N., Roy H., Lieberman G., Garcia J.O., Asturias A., Okafor G.N., Elliott J.C., Giesbrecht B., Grafton S.T., et al. Individual differences in compliance and agreement for sleep logs and wrist actigraphy: A longitudinal study of naturalistic sleep in healthy adults. PLoS ONE. 2018;13:e0191883. doi: 10.1371/journal.pone.0191883.
    1. Stone J.D., Rentz L.E., Forsey J., Ramadan J., Markwald R.R., Finomore V.S., Galster S.M., Rezai A., Hagen J.A. Evaluations of Commercial Sleep Technologies for Objective Monitoring During Routine Sleeping Conditions. Nat. Sci. Sleep. 2020;12:821–842. doi: 10.2147/NSS.S270705.
    1. Khosla S., Deak M.C., Gault D., Goldstein C.A., Hwang D., Kwon Y., O’Hearn D., Schutte-Rodin S., Yurcheshen M., Rosen I.M., et al. Consumer Sleep Technology: An American Academy of Sleep Medicine Position Statement. J. Clin. Sleep Med. Off. Publ. Am. Acad. Sleep Med. 2018;14:877–880. doi: 10.5664/jcsm.7128.
    1. Ibáñez V., Silva J., Navarro E., Cauli O. Sleep assessment devices: Types, market analysis, and a critical view on accuracy and validation. Expert Rev. Med. Devices. 2019;16:1041–1052. doi: 10.1080/17434440.2019.1693890.
    1. Berryhill S., Morton C.J., Dean A., Berryhill A., Provencio-Dean N., Patel S.I., Estep L., Combs D., Mashaqi S., Gerald L.B., et al. Effect of wearables on sleep in healthy individuals: A randomized crossover trial and validation study. J. Clin. Sleep Med. Off. Publ. Am. Acad. Sleep Med. 2020;16:775–783. doi: 10.5664/jcsm.8356.
    1. Blume C., Schmidt M.H., Cajochen C. Effects of the COVID-19 lockdown on human sleep and rest-activity rhythms. Curr. Biol. 2020;30:R795–R797. doi: 10.1016/j.cub.2020.06.021.
    1. Ancoli-Israel S., Martin J.L., Blackwell T., Buenaver L., Liu L., Meltzer L.J., Sadeh A., Spira A.P., Taylor D.J. The SBSM Guide to Actigraphy Monitoring: Clinical and Research Applications. Behav. Sleep Med. 2015;13((Suppl. 1)):S4–S38. doi: 10.1080/15402002.2015.1046356.

Source: PubMed

스폰서 및 공동 작업자

건강 상태

약물 개입

3
구독하다