Validity, potential clinical utility, and comparison of consumer and research-grade activity trackers in Insomnia Disorder I: In-lab validation against polysomnography

Piyumi Kahawage, Ria Jumabhoy, Kellie Hamill, Massimiliano de Zambotti, Sean P A Drummond, Piyumi Kahawage, Ria Jumabhoy, Kellie Hamill, Massimiliano de Zambotti, Sean P A Drummond

Abstract

Consumer activity trackers claiming to measure sleep/wake patterns are ubiquitous within clinical and consumer settings. However, validation of these devices in sleep disorder populations are lacking. We examined 1 night of sleep in 42 individuals with insomnia (mean = 49.14 ± 17.54 years) using polysomnography, a wrist actigraph (Actiwatch Spectrum Pro: AWS) and a consumer activity tracker (Fitbit Alta HR: FBA). Epoch-by-epoch analysis and Bland-Altman methods evaluated each device against polysomnography for sleep/wake detection, total sleep time, sleep efficiency, wake after sleep onset and sleep latency. FBA sleep stage classification of light sleep (N1 + N2), deep sleep (N3) and rapid eye movement was also compared with polysomnography. Compared with polysomnography, both activity trackers displayed high accuracy (81.12% versus 82.80%, AWS and FBA respectively; ns) and sensitivity (sleep detection; 96.66% versus 96.04%, respectively; ns) but low specificity (wake detection; 39.09% versus 44.76%, respectively; p = .037). Both trackers overestimated total sleep time and sleep efficiency, and underestimated sleep latency and wake after sleep onset. FBA demonstrated sleep stage sensitivity and specificity, respectively, of 79.39% and 58.77% (light), 49.04% and 95.54% (deep), 65.97% and 91.53% (rapid eye movement). Both devices were more accurate in detecting sleep than wake, with equivalent sensitivity, but statistically different specificity. FBA provided equivalent estimates as AWS for all traditional actigraphy sleep parameters. FBA also showed high specificity when identifying N3, and rapid eye movement, though sensitivity was modest. Thus, it underestimates these sleep stages and overestimates light sleep, demonstrating more shallow sleep than actually obtained. Whether FBA could serve as a low-cost substitute for actigraphy in insomnia requires further investigation.

Keywords: actigraphy; activity-monitor; consumer sleep tracker; wearables.

© 2019 European Sleep Research Society.

References

REFERENCES

    1. American Academy of Sleep Medicine (2014). The AASM manual for the scoring of sleep and associated events: Rules, terminology, and technical specifications (vol. 2.1). Darien, IL: American Academy of Sleep Medicine.
    1. Baron, K. G., Duffecy, J., Berendsen, M. A., Cheung, M. I., Lattie, E. G., & Manalo, N. C. (2018). Feeling validated yet? A scoping review of the use of consumer-targeted wearable and mobile technology to measure and improve sleep. Sleep Medicine Reviews, 40, 151-159.
    1. Beattie, Z., Oyang, Y., Statan, A., Ghoreyshi, A., Pantelopoulos, A., Russell, A., & Heneghan, C. (2017). Estimation of sleep stages in a healthy adult population from optical plethysmography and accelerometer signals. Physiological Measurement, 38(11), 1968-1979.
    1. Bei, B., Wiley, J. F., Trinder, J., & Manber, R. (2016). Beyond the mean: A systematic review on the correlates of daily intraindividual variability of sleep/wake patterns. Sleep Medicine Reviews, 28, 108-124.
    1. Bergman, H., & Källmén, H. (2002). Alcohol use among Swedes and a psychometric evaluation of the alcohol use disorders identification test. Alcohol and Alcoholism, 37, 245-251.
    1. Bin, Y. S., Marshall, N. S., & Glozier, N. (2012). The burden of insomnia on individual function and healthcare consumption in Australia. Australian and New Zealand Journal of Public Health, 36(5), 462-468.
    1. Bland, J. M., & Altman, D. G. (1986). Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet, 1, 307-310.
    1. Byrt, T., Bishop, J., & Carlin, J. B. (1993). Bias, prevalence and kappa. Journal of Clinical Epidemiology, 46, 423-429.
    1. Carney, C. E., Buysse, D. J., Ancoli-Israel, S., Edinger, J. D., Krystal, A. D., Lichstein, K. L., & Morin, C. M. (2012). The consensus sleep diary: Standardizing prospective sleep self-monitoring. Sleep, 35(2), 287-302.
    1. Chung, F., Yegneswaran, B., Liao, P. U., Chung, S. A., Vairavanathan, S., Islam, S., …, Shapiro, C. M. (2008). STOP questionnaire: A tool to screen patients for obstructive sleep apnea. Anesthesiology, 108, 812-821.
    1. Cohen, J. (1988). Statistical power analysis for the behavioural sciences (2nd edn). New York, NY: Academic Press.
    1. Cook, J. D., Eftekari, S. C., Dallmann, E., Sippy, M., & Plante, D. T. (2018). Ability of the Fitbit Alta HR to quantify and classify sleep in patients with suspected central disorders of hypersomnolence: A comparison against polysomnography. Journal of Sleep Research, 1(1), e12789.
    1. Cook, J. D., Prairie, M. L., & Plante, D. T. (2017). Utility of the Fitbit Flex to evaluate sleep in major depressive disorder: A comparison against polysomnography and wrist-worn actigraphy. Journal of Affective Disorders, 217, 299-305.
    1. de Zambotti, M., Cellini, N., Goldstone, A., Colrain, I. M., & Baker, F. C. (2018). Wearable sleep technology in clinical and research settings. Medicine & Science in Sports & Exercise, 51(7), 1538-1557.
    1. de Zambotti, M., Goldstone, A., Claudatos, S., Colrain, I. M., & Baker, F. C. (2018). A validation study of Fitbit Charge 2 compared with polysomnography in adults. Chronobiology International, 35, 465-476.
    1. Edinger, J. D., Wyatt, J. K., Olsen, M. K., Stechuchak, K. M., Carney, C. E., Chiang, A., …, Radtke, R. A. (2006). Duke structured interview schedule for DSM-IV-TR and international classification of sleep disorders (2nd edn). Durham, NC: Veterans Affairs and Duke University Medical Center.
    1. Field, A. (2018). Discovering statistics using IBM SPSS statistics. London, UK: SAGE Publications.
    1. First, M. B., Williams, J. B., Karg, R. S., & Spitzer, R. L. (2016). Structured clinical interview for DSM-5 disorders-clinical version (SCID-5-CV). Arlington, VA: American Psychiatric Association.
    1. Hamill, K., Jumabhoy, R., Kahawage, P., de Zambotti, M., Walters, E. M., & Drummond, S. P. (2019). Validity, potential clinical utility, and comparison of consumer and research-grade activity trackers in insomnia disorder. II: Outside the laboratory. Journal of Sleep Research, e12944.
    1. Kang, S. G., Kang, J. M., Ko, K. P., Park, S. C., Mariani, S., & Weng, J. (2017). Validity of a commercial wearable sleep tracker in adult insomnia disorder patients and good sleepers. Journal of Psychosomatic Research, 97, 38-44.
    1. Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159-174.
    1. Marino, M., Li, Y. I., Rueschman, M. N., Winkelman, J. W., Ellenbogen, J. M., Solet, J. M., …, Buxton, O. M. (2013). Measuring sleep: Accuracy, sensitivity, and specificity of wrist actigraphy compared to polysomnography. Sleep, 36(11), 1747-1755.
    1. Markwald, R. R., Bessman, S. C., Reini, S. A., & Drummond, S. P. (2016). Performance of a portable sleep monitoring device in individuals with high versus low sleep efficiency. Journal of Clinical Sleep Medicine, 12(1), 95-103.
    1. Mellor, A., Hamill, K., Jenkins, M. M., Baucom, D. H., Norton, P. J., & Drummond, S. P. (2019). Partner-assisted cogntive behavoiural therapy for insomnia versus cognitive behavioural therapy for insomnoa: A radomised controlled trial. Trials, 20(1), 262.
    1. Meltzer, L. J., Walsh, C. M., Traylor, J., & Westin, A. M. (2012). Direct comparison of two new actigraphs and polysomnography in children and adolescents. Sleep, 35, 159-166.
    1. Montgomery-Downs, H. E., Insana, S. P., & Bond, J. A. (2012). Movement toward a novel activity monitoring device. Sleep Breath, 16(3), 913-917.
    1. Morin, C. M., Belleville, G., Bélanger, L., & Ivers, H. (2011). The insomnia severity index: Psychometric indicators to detect insomnia cases and evaluate treatment response. Sleep, 34, 601-608.
    1. Qaseem, A., Kansagara, D., Forciea, M. A., Cooke, M., & Denberg, T. D. (2016). Management of chronic insomnia disorder in adults: A clinical practice guideline from the American college of physicians. Annals of Internal Medicine, 165(2), 125-133.
    1. Sadeh, A. (2015). III: Sleep assessment methods. Monographs of the Society for Research in Child Development, 80, 33-48.
    1. Shelgikar, A. V., Anderson, P. F., & Stephens, M. R. (2016). Sleep tracking, wearable technology, and opportunities for research and clinical care. Chest, 150(3), 732-743.
    1. Sivertsen, B., Omvik, S., Havik, O. E., Pallesen, S., Bjorvatn, B., Nielsen, G. H., …, Nordhus, I. H. (2006). A comparison of actigraphy and polysomnography in older adults treated for chronic primary insomnia. Sleep, 29(10), 1353-1358.
    1. Straus, L. D., Drummond, S. P., Nappi, C. M., Jenkins, M. M., & Norman, S. B. (2015). Sleep variability in military-related PTSD: A comparison to primary insomnia and healthy controls. Journal of Traumatic Stress, 28, 8-16.
    1. Taibi, D. M., Landis, C. A., & Vitiello, M. V. (2013). Concordance of polysomnographic and actigraphic measurement of sleep and wake in older women with insomnia. Journal of Clinical Sleep Medicine, 9(3), 217-225.
    1. Tilmanne, J., Urbain, J., Kothare, M. V., Wouwer, A. V., & Kothare, S. V. (2009). Algorithms for sleep-wake identification using actigraphy: A comparative study and new results. Journal of Sleep Research, 18(1), 85-98.
    1. Toon, E., Davey, M. J., Hollis, S. L., Nixon, G. M., Horne, R. S., & Biggs, S. N. (2016). Comparison of commercial wrist-based and smartphone accelerometers, actigraphy, and PSG in a clinical cohort of children and adolescents. Journal of Clinical Sleep Medicine, 12(3), 343-350.
    1. Watson, N. F., Badr, M. S., Belenky, G., Bliwise, D. L., Buxton, O. M., Buysse, D., …, Tasali, E. (2015). Recommended amount of sleep for a healthy adult: A joint consensus statement of the American Academy of Sleep Medicine and Sleep Research Society. Sleep, 38(6), 843-844.
    1. World Health Organization (2001). AUDIT: The alcohol use disorders identification test: Guidelines for use in primary health care (2nd edn). Geneva, Switzerland: World Health Organization.

Source: PubMed

Patrocinadores e Colaboradores

Condições médicas

Intervenções de drogas

3
Se inscrever