Systematic review of the validity and reliability of consumer-wearable activity trackers

Kelly R Evenson, Michelle M Goto, Robert D Furberg, Kelly R Evenson, Michelle M Goto, Robert D Furberg

Abstract

Background: Consumer-wearable activity trackers are electronic devices used for monitoring fitness- and other health-related metrics. The purpose of this systematic review was to summarize the evidence for validity and reliability of popular consumer-wearable activity trackers (Fitbit and Jawbone) and their ability to estimate steps, distance, physical activity, energy expenditure, and sleep.

Methods: Searches included only full-length English language studies published in PubMed, Embase, SPORTDiscus, and Google Scholar through July 31, 2015. Two people reviewed and abstracted each included study.

Results: In total, 22 studies were included in the review (20 on adults, 2 on youth). For laboratory-based studies using step counting or accelerometer steps, the correlation with tracker-assessed steps was high for both Fitbit and Jawbone (Pearson or intraclass correlation coefficients (CC) > =0.80). Only one study assessed distance for the Fitbit, finding an over-estimate at slower speeds and under-estimate at faster speeds. Two field-based studies compared accelerometry-assessed physical activity to the trackers, with one study finding higher correlation (Spearman CC 0.86, Fitbit) while another study found a wide range in correlation (intraclass CC 0.36-0.70, Fitbit and Jawbone). Using several different comparison measures (indirect and direct calorimetry, accelerometry, self-report), energy expenditure was more often under-estimated by either tracker. Total sleep time and sleep efficiency were over-estimated and wake after sleep onset was under-estimated comparing metrics from polysomnography to either tracker using a normal mode setting. No studies of intradevice reliability were found. Interdevice reliability was reported on seven studies using the Fitbit, but none for the Jawbone. Walking- and running-based Fitbit trials indicated consistently high interdevice reliability for steps (Pearson and intraclass CC 0.76-1.00), distance (intraclass CC 0.90-0.99), and energy expenditure (Pearson and intraclass CC 0.71-0.97). When wearing two Fitbits while sleeping, consistency between the devices was high.

Conclusion: This systematic review indicated higher validity of steps, few studies on distance and physical activity, and lower validity for energy expenditure and sleep. The evidence reviewed indicated high interdevice reliability for steps, distance, energy expenditure, and sleep for certain Fitbit models. As new activity trackers and features are introduced to the market, documentation of the measurement properties can guide their use in research settings.

References

    1. Almalki M, Gray K, Sanchez FM. The use of self-quantification systems for personal health information: big data management activities and prospects. Health Information Science Systems. 2015;3(Suppl 1 HISA Big Data in Biomedicine and Healthcare 2013 Con):S1. doi: 10.1186/2047-2501-3-S1-S1.
    1. Danova T. Just 3.3 million fitness trackers were sold in the US in the past year. Business Insider 2014. . Accessed March 2, 2015.
    1. Lyons EJ, Lewis ZH, Mayrsohn BG, Rowland JL. Behavior change techniques implemented in electronic lifestyle activity monitors: A systematic content analysis. J Med Internet Res. 2014;16(8):e192. doi: 10.2196/jmir.3469.
    1. Cadmus-Bertram LA, Marcus BH, Patterson RE, Parker BA, Morey BL. Randomized trial of a Fitbit-Based physical activity intervention for women. Am J Prev Med. 2015;49(3):414–8. doi: 10.1016/j.amepre.2015.01.020.
    1. Michie S, Ashford S, Sniehotta FF, Dombrowski SU, Bishop A, French DP. A refined taxonomy of behaviour change techniques to help people change their physical activity and healthy eating behaviours: The CALO-RE taxonomy. Psych Health. 2011;26(11):1479–98. doi: 10.1080/08870446.2010.540664.
    1. Fox S, Duggan M. Tracking for Health. Pew Research Center, Pew Internet and American Life Project. 2013. . Accessed October 9, 2015.
    1. Bentley F, Tollmar K, Stephenson P, Levy L, Jones B, Robertson S, et al. Health mashups: Presenting statistical patterns between well-being data and context in natural language to promote behavior change. ACM Trans Comput-Hum Interact. 2013;20(5):1–25. doi: 10.1145/2503823.
    1. Kurti AN, Dallery J. Internet-based contingency management increases walking in sedentary adults. J Appl Behav Anal. 2013;46(3):568–81. doi: 10.1002/jaba.58.
    1. Washington WD, Banna KM, Gibson AL. Preliminary efficacy of prize-based contingency management to increase activity levels in healthy adults. J Appl Behav Anal. 2014;47(2):231–45. doi: 10.1002/jaba.119.
    1. Thompson WG, Kuhle CL, Koepp GA, McCrady-Spitzer SK, Levine JA. “Go4Life” exercise counseling, accelerometer feedback, and activity levels in older people. Arch Gerontol Geriatr. 2014;58(3):314–9. doi: 10.1016/j.archger.2014.01.004.
    1. Wang JB, Cadmus-Bertram LA, Natarajan L, White MM, Madanat H, Nichols JF, et al. Wearable sensor/device (Fitbit One) and SMS text-messaging prompts to increase physical activity in overweight and obese adults: A randomized controlled trial. Telemed J E-Health. 2015;21(10):782–92. doi: 10.1089/tmj.2014.0176.
    1. Hayes LB, Van Camp CM. Increasing physical activity of children during school recess. J Appl Behav Anal. 2015;48(3):690–5. doi: 10.1002/jaba.222.
    1. Fitbit Inc. How accurate are Fitbit trackers? 2015. . Accessed June 16, 2015.
    1. Jawbone. Jawbone UP: Activity Data Issues. 2015. . Accessed June 16, 2015.
    1. Diaz KM, Krupka DJ, Chang MJ, Peacock J, Ma Y, Goldsmith J, et al. Fitbit: An accurate and reliable device for wireless physical activity tracking. Intl J Cardiol. 2015;185:138–40. doi: 10.1016/j.ijcard.2015.03.038.
    1. Klassen TD, Eng JJ, Chan C, Hassall Z, Lim S, Louie R, et al. Step count monitor for individuals post-stroke: Accuracy of the Fitbit One. Stroke. 2014;45(12):e261.
    1. Perez-Macias JM, Jimison H, Korhonen I, Pavel M. Comparative assessment of sleep quality estimates using home monitoring technology. Conference proceedings: Annual International Conference of the IEEE Engineering in Medicine and Biology Society IEEE Engineering in Medicine and Biology Society Annual Conference. 2014; 2014:4979–82. doi: 10.1109/embc.2014.6944742.
    1. Fulk GD, Combs SA, Danks KA, Nirider CD, Raja B, Reisman DS. Accuracy of 2 activity monitors in detecting steps in people with stroke and traumatic brain injury. Phys Ther. 2014;94(2):222–9. doi: 10.2522/ptj.20120525.
    1. Vooijs M, Alpay LL, Snoeck-Stroband JB, Beerthuizen T, Siemonsma PC, Abbink JJ, et al. Validity and usability of low-cost accelerometers for internet-based self-monitoring of physical activity in patients with chronic obstructive pulmonary disease. Interactive J Med Res. 2014;3(4):e14. doi: 10.2196/ijmr.3056.
    1. Albert MV, Deeny S, McCarthy C, Valentin J, Jayaraman A. Monitoring daily function in persons with transfemoral amputations using a commercial activity monitor: A feasibility study. PM & R: J Inj Funct Rehabil. 2014;6(12):1120–7. doi: 10.1016/j.pmrj.2014.06.006.
    1. Naslund JA, Aschbrenner KA, Barre LK, Bartels SJ. Feasibility of popular m-health technologies for activity tracking among individuals with serious mental illness. Telemed J E-Health. 2015;21(3):213–6. doi: 10.1089/tmj.2014.0105.
    1. Phillips LJ, Petroski GF, Markis NE. A comparison of accelerometer accuracy in older adults. Res Gerontol Nursing. 2015: 1–7. doi:10.3928/19404921-20150429-03.
    1. Lauritzen J, Munoz A, Luis Sevillano J, Civit A. The usefulness of activity trackers in elderly with reduced mobility: A case study. Stud Health Technol Inform. 2013;192:759–62.
    1. De Vries SI, Van Hirtum HW, Bakker I, Hopman-Rock M, Hirasing RA, Van Mechelen W. Validity and reproducibility of motion sensors in youth: A systematic update. Med Sci Sports Exerc. 2009;41(4):818–27. doi: 10.1249/MSS.0b013e31818e5819.
    1. Higgins PA, Straub AJ. Understanding the error of our ways: Mapping the concepts of validity and reliability. Nurs Outlook. 2006;54(1):23–9. doi: 10.1016/j.outlook.2004.12.004.
    1. Landis J, Koch G. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74. doi: 10.2307/2529310.
    1. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. PLoS Med. 2009;6(7):e1000100. doi: 10.1371/journal.pmed.1000100.
    1. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med. 2009;6(7):e1000097. doi: 10.1371/journal.pmed.1000097.
    1. Adam Noah J, Spierer DK, Gu J, Bronner S. Comparison of steps and energy expenditure assessment in adults of Fitbit Tracker and Ultra to the Actical and indirect calorimetry. J Med Eng Tech. 2013;37(7):456–62. doi: 10.3109/03091902.2013.831135.
    1. Case MA, Burwick HA, Volpp KG, Patel MS. Accuracy of smartphone applications and wearable devices for tracking physical activity data. JAMA. 2015;313(6):625–6. doi: 10.1001/jama.2014.17841.
    1. Dannecker KL, Sazonova NA, Melanson EL, Sazonov ES, Browning RC. A comparison of energy expenditure estimation of several physical activity monitors. Med Sci Sports Exerc. 2013;45(11):2105–12. doi: 10.1249/MSS.0b013e318299d2eb.
    1. Dontje ML, de Groot M, Lengton RR, van der Schans CP, Krijnen WP. Measuring steps with the Fitbit activity tracker: An inter-device reliability study. J Med Eng Tech. 2015;39(5):286–90. doi: 10.3109/03091902.2015.1050125.
    1. Ferguson T, Rowlands AV, Olds T, Maher C. The validity of consumer-level, activity monitors in healthy adults worn in free-living conditions: A cross-sectional study. Intl J Behav Nutr Phys Act. 2015;12:42. doi: 10.1186/s12966-015-0201-9.
    1. Gusmer R, Bosch T, Watkins A, Ostrem J, Dengel D. Comparison of Fitbit Ultra to ActiGraph GT1M for assessment of physical activity in young adults during treadmill walking. Open Sports Med J. 2014;8:11–5. doi: 10.2174/1874387001408010011.
    1. Lee JM, Kim Y, Welk GJ. Validity of consumer-based physical activity monitors. Med Sci Sports Exerc. 2014;46(9):1840–8. doi: 10.1249/MSS.0000000000000287.
    1. Mammen G, Gardiner S, Senthinathan A, McClemont L, Stone M, Faulkner G. Is this bit fit? Measuring the quality of the FitBit step-counter. Health Fit J Can. 2012;5(4):30–9.
    1. Meltzer LJ, Hiruma LS, Avis K, Montgomery-Downs H, Valentin J. Comparison of a commercial accelerometer with polysomnography and actigraphy in children and adolescents. Sleep. 2015;38(8):1323–30.
    1. Montgomery-Downs HE, Insana SP, Bond JA. Movement toward a novel activity monitoring device. Sleep Breath. 2012;16(3):913–7. doi: 10.1007/s11325-011-0585-y.
    1. Sasaki JE, Hickey A, Mavilia M, Tedesco J, John D, Kozey Keadle S, et al. Validation of the Fitbit wireless activity tracker for prediction of energy expenditure. J Phys Act Health. 2015;12:149–54. doi: 10.1123/jpah.2012-0495.
    1. Stackpool CM, Porcari JP, Mikat RP, Gillette C, Foster C. The accuracy of various activity trackers in estimating steps taken and energy expenditure. J Fit Res. 2014;3(3):32–48.
    1. Stahl ST, Insana SP. Caloric expenditure assessment among older adults: Criterion validity of a novel accelerometry device. J Health Psych. 2014;19(11):1382–7. doi: 10.1177/1359105313490771.
    1. Storm FA, Heller BW, Mazza C. Step detection and activity recognition accuracy of seven physical activity monitors. PLoS ONE. 2015;10(3):e0118723. doi: 10.1371/journal.pone.0118723.
    1. Takacs J, Pollock CL, Guenther JR, Bahar M, Napier C, Hunt MA. Validation of the Fitbit One activity monitor device during treadmill walking. J Sci Med Sport. 2014;17(5):496–500. doi: 10.1016/j.jsams.2013.10.241.
    1. Tully MA, McBride C, Heron L, Hunter RF. The validation of Fibit Zip physical activity monitor as a measure of free-living physical activity. BMC Res Notes. 2014;7:952. doi: 10.1186/1756-0500-7-952.
    1. Bai Y, Welk GJ, Nam YH, Lee JA, Lee JM, Kim Y et al. Comparison of consumer and research monitors under semistructured settings. Med Sci Sports Exercise. 2015, in press. doi:10.1249/MSS.0000000000000727.
    1. Simpson LA, Eng JJ, Klassen TD, Lim SB, Louie DR, Parappilly B, et al. Capturing step counts at slow walking speeds in older adults: Comparison of ankle and waist placement of measuring device. J Rehabil Med. 2015;47(9):830–5. doi: 10.2340/16501977-1993.
    1. de Zambotti M, Claudatos S, Inkelis S, Colrain IM, Baker FC. Evaluation of a consumer fitness-tracking device to assess sleep in adults. Chronobiol Intl. 2015;32(7):1024–8. doi: 10.3109/07420528.2015.1054395.
    1. de Zambotti M, Baker FC, Colrain IM. Validation of sleep-tracking technology compared with polysomnography in adolescents. Sleep. 2015;38(9):1461–8.
    1. Freedson PS, Melanson E, Sirard J. Calibration of the computer science and applications, Inc. accelerometer. Med Sci Sports Exerc. 1998;30(5):777–81. doi: 10.1097/00005768-199805000-00021.
    1. Sasaki JE, John D, Freedson PS. Validation and comparison of ActiGraph activity monitors. J Sci Med Sport. 2011;14(5):411–6. doi: 10.1016/j.jsams.2011.04.003.
    1. Crouter SE, Schneider PL, Karabulut M, Bassett DR., Jr Validity of 10 electronic pedometers for measuring steps, distance, and energy cost. Med Sci Sports Exerc. 2003;35(8):1455–60. doi: 10.1249/01.MSS.0000078932.61440.A2.
    1. Meltzer LJ, Montgomery-Downs HE, Insana SP, Walsh CM. Use of actigraphy for assessment in pediatric sleep research. Sleep Med Rev. 2012;16(5):463–75. doi: 10.1016/j.smrv.2011.10.002.
    1. Shih P, Han K, Poole E, Rosson M, Carroll J. Use and adoption challenges of wearable activity trackers. 2015. iConference 2015 Proceedings. . Accessed June 16, 2015.
    1. Fitbit I. A brief look into how the Fitbit algorithms work. 2009.
    1. John D, Freedson P. ActiGraph and Actical physical activity monitors: A peek under the hood. Med Sci Sport Exerc. 2012;44(1 Suppl 1):S86–9. doi: 10.1249/MSS.0b013e3182399f5e.
    1. John D, Sasaki J, Hickey A, Mavilia M, Freedson PS. ActiGraph activity monitors: “The firmware effect”. Med Sci Sport Exerc. 2014;46(4):834–9. doi: 10.1249/MSS.0000000000000145.
    1. Crouter SE, Schneider PL, Bassett DR., Jr Spring-levered versus piezo-electric pedometer accuracy in overweight and obese adults. Med Sci Sport Exerc. 2005;37(10):1673–9. doi: 10.1249/01.mss.0000181677.36658.a8.
    1. Welk GJ, McClain J, Ainsworth BE. Protocols for evaluating equivalency of accelerometry-based activity monitors. Med Sci Sport Exerc. 2012;44(1 Suppl 1):S39–49. doi: 10.1249/MSS.0b013e3182399d8f.
    1. King AC, Glanz K, Patrick K. Technologies to measure and modify physical activity and eating environments. Am J Prev Med. 2015;48(5):630–8. doi: 10.1016/j.amepre.2014.10.005.
    1. de Montjoye YA, Hidalgo CA, Verleysen M, Blondel VD. Unique in the Crowd: The privacy bounds of human mobility. Sci Rep. 2013;3:1376. doi: 10.1038/srep01376.
    1. Health Data Exploration Project. Personal Data for the Public Good: New Opportunities to Enrish Understanding of Individual and Population Health. 2014. . Accessed October 9, 2015. Calit2, UC Irvine and UC San Diego.
    1. Fitbit Inc. Weathering the weather. 2015. . Accessed October 9, 2015.
    1. Mohan S. The Jawbone Blog: What makes people happy? We have the data. 2015. . Accessed October 9, 2015.
    1. IDC. Wearable Market Remained Strong in the First Quarter Despite the Pending Debut of the Apple Watch, Says IDC. Press release from IDC on June 3, 2015. Based on the “IDC Worldwide Quarterly Wearable Tracker, June 2, 2015”. 2015. . Accessed October 9, 2015.

Source: PubMed

Sponsoren und Mitarbeiter

Krankheiten

Drogeninterventionen

3
Abonnieren