Reliability and validity of ten consumer activity trackers

Thea J M Kooiman, Manon L Dontje, Siska R Sprenger, Wim P Krijnen, Cees P van der Schans, Martijn de Groot, Thea J M Kooiman, Manon L Dontje, Siska R Sprenger, Wim P Krijnen, Cees P van der Schans, Martijn de Groot

Abstract

Background: Activity trackers can potentially stimulate users to increase their physical activity behavior. The aim of this study was to examine the reliability and validity of ten consumer activity trackers for measuring step count in both laboratory and free-living conditions.

Method: Healthy adult volunteers (n = 33) walked twice on a treadmill (4.8 km/h) for 30 min while wearing ten different activity trackers (i.e. Lumoback, Fitbit Flex, Jawbone Up, Nike+ Fuelband SE, Misfit Shine, Withings Pulse, Fitbit Zip, Omron HJ-203, Yamax Digiwalker SW-200 and Moves mobile application). In free-living conditions, 56 volunteers wore the same activity trackers for one working day. Test-retest reliability was analyzed with the Intraclass Correlation Coefficient (ICC). Validity was evaluated by comparing each tracker with the gold standard (Optogait system for laboratory and ActivPAL for free-living conditions), using paired samples t-tests, mean absolute percentage errors, correlations and Bland-Altman plots.

Results: Test-retest analysis revealed high reliability for most trackers except for the Omron (ICC .14), Moves app (ICC .37) and Nike+ Fuelband (ICC .53). The mean absolute percentage errors of the trackers in laboratory and free-living conditions respectively, were: Lumoback (-0.2, -0.4), Fibit Flex (-5.7, 3.7), Jawbone Up (-1.0, 1.4), Nike+ Fuelband (-18, -24), Misfit Shine (0.2, 1.1), Withings Pulse (-0.5, -7.9), Fitbit Zip (-0.3, 1.2), Omron (2.5, -0.4), Digiwalker (-1.2, -5.9), and Moves app (9.6, -37.6). Bland-Altman plots demonstrated that the limits of agreement varied from 46 steps (Fitbit Zip) to 2422 steps (Nike+ Fuelband) in the laboratory condition, and 866 steps (Fitbit Zip) to 5150 steps (Moves app) in the free-living condition.

Conclusion: The reliability and validity of most trackers for measuring step count is good. The Fitbit Zip is the most valid whereas the reliability and validity of the Nike+ Fuelband is low.

Keywords: Accelerometry; Activity trackers; Free-living; Reliability; Validation study.

Figures

Fig. 1
Fig. 1
Descriptive Statistics (mean number of steps, 95 % CI) as measured by the gold standards (horizontal lines) and the ten activity trackers in the laboratory and free-living condition
Fig. 2
Fig. 2
Bland-Altman plots of the top three activity trackers in the laboratory condition (Optogait vs. Fitbit Zip, Lumoback, and Withings Pulse, figure a-c) and free-living condition (ActivPal vs. Fitbit Zip, Misfit Shine, and Lumoback, figure d-f). The middle line shows the mean difference between the tracker and the gold standard, and the dashed lines indicate the limits of agreement (±1.96 * SD of the difference scores)

References

    1. Lee IM, Shiroma EJ, Lobelo F, Puska P, Blair SN, Katzmarzyk PT, Lancet Physical Activity Series Working Group Effect of physical inactivity on major non-communicable diseases worldwide: an analysis of burden of disease and life expectancy. Lancet. 2012;380(9838):219–229. doi: 10.1016/S0140-6736(12)61031-9.
    1. Warburton DE, Nicol CW, Bredin SS. Health benefits of physical activity: the evidence. CMAJ. 2006;174(6):801–809. doi: 10.1503/cmaj.051351.
    1. Haskell WL, Lee IM, Pate RR, Blair SN, Franklin BA, Macera CA, Heath GW, Thompson PD, Bauman A. Physical activity and public health: updated recommendation for adults from the American College of Sports Medicine and the American Heart Association. Med Sci Sports Exerc. 2007;39(8):1423–1434. doi: 10.1249/mss.0b013e3180616b27.
    1. Tudor-Locke C, Craig CL, Brown WJ, Clemes SA, De Cocker K, Giles-Corti B, Hatano Y, Inoue S, Matsudo SM, Mutrie N, Oppert J, Rowe DA, Schmidt MD, Schofield GM, Spence JC, Teixeira PJ, Tully MA, Blair SN. How many steps/day are enough? For adults. Int J Behav Nutr Phys Act. 2011;8:79-5868-8-79.
    1. Godino JG, Watkinson C, Corder K, Sutton S, Griffin SJ, Van Sluijs EM. Awareness of physical activity in healthy middle-aged adults: a cross-sectional study of associations with sociodemographic, biological, behavioural, and psychological factors. BMC Public Health. 2014;14(1):421. doi: 10.1186/1471-2458-14-421.
    1. Vooijs M, Alpay LL, Snoeck-Stroband JB, Beerthuizen T, Siemonsma PC, Abbink JJ, Sont JK, Rövekamp TA. Validity and usability of low-cost accelerometers for internet-based self-monitoring of physical activity in patients with chronic obstructive pulmonary disease. Interact J Med Res. 2014;3(4):e14. doi: 10.2196/ijmr.3056.
    1. Bravata DM, Smith-Spangler C, Sundaram V, Gienger AL, Lin N, Lewis R, Stave CD, Olkin I, Sirard JR. Using pedometers to increase physical activity and improve health: a systematic review. JAMA. 2007;298(19):2296–304. doi: 10.1001/jama.298.19.2296.
    1. El-Gayar O, Timsina P, Nawar N, Eid W. A systematic review of IT for diabetes self-management: are we there yet? Int J Med Inform. 2013;82(8):637–652. doi: 10.1016/j.ijmedinf.2013.05.006.
    1. Adam Noah J, Spierer DK, Gu J, Bronner S. Comparison of steps and energy expenditure assessment in adults of Fitbit Tracker and Ultra to the Actical and indirect calorimetry. J Med Eng Technol. 2013;37(7):456–462. doi: 10.3109/03091902.2013.831135.
    1. Dannecker KL, Sazonova NA, Melanson EL, Sazonov ES, Browning RC. A comparison of energy expenditure estimation of several physical activity monitors. Med Sci Sports Exerc. 2013;45(11):2105–2112. doi: 10.1249/MSS.0b013e318299d2eb.
    1. Fortune E, Lugade V, Morrow M, Kaufman K. Validity of using tri-axial accelerometers to measure human movement - Part II: Step counts at a wide range of gait velocities. Med Eng Phys. 2014;36(6):659–669. doi: 10.1016/j.medengphy.2014.02.006.
    1. Lee JM, Kim Y, Welk GJ. Validity of consumer-based physical activity monitors. Med Sci Sports Exerc. 2014;46(9):1840–1848. doi: 10.1249/MSS.0000000000000287.
    1. Stahl ST, Insana SP. Caloric expenditure assessment among older adults: criterion validity of a novel accelerometry device. J Health Psychol. 2014;19(11):1382–1387. doi: 10.1177/1359105313490771.
    1. Takacs J, Pollock CL, Guenther JR, Bahar M, Napier C, Hunt MA. Validation of the Fitbit One activity monitor device during treadmill walking. Interact J Med Res. 2014;17(5):496–500.
    1. Tully MA, McBride C, Heron L, Allen W, Hunter RF. The validation of Fibit ZipTM physical activity monitor as a measure of free-living physical activity. BMC Res Notes. 2014;7(1):952. doi: 10.1186/1756-0500-7-952.
    1. Case MA, Burwick HA, Volpp KG, Patel MS. Accuracy of smartphone applications and wearable devices for tracking physical activity data. JAMA. 2015;313(6):625–626. doi: 10.1001/jama.2014.17841.
    1. Ferguson T, Rowlands AV, Olds T, Maher C. The validity of consumer-level, activity monitors in healthy adults worn in free-living conditions: a cross-sectional study. Int J Behav Nutr Phys Act. 2015;12:42-015-0201-9. doi: 10.1186/s12966-015-0201-9.
    1. Dontje ML, de Groot M, Lengton RR, van der Schans CP, Krijnen WP. Measuring steps with the Fitbit activity tracker: an inter-device reliability study. J Med Eng Technol. 2015;39(5):286–290. doi: 10.3109/03091902.2015.1050125.
    1. Lee M, Song C, Lee K, Shin D, Shin S. Agreement between the spatio-temporal gait parameters from treadmill-based photoelectric cell and the instrumented treadmill system in healthy young adults and stroke patients. Med Sci Monit. 2014;20:1210–1219. doi: 10.12659/MSM.890658.
    1. Dahlgren G, Carlsson D, Moorhead A, Hager-Ross C, McDonough SM. Test-retest reliability of step counts with the ActivPAL device in common daily activities. Gait Posture. 2010;32(3):386–390. doi: 10.1016/j.gaitpost.2010.06.022.
    1. Dowd KP, Harrington DM, Donnelly AE. Criterion and concurrent validity of the activPAL professional physical activity monitor in adolescent females. PLoS One. 2012;7(10):e47633. doi: 10.1371/journal.pone.0047633.
    1. Ryan CG, Grant PM, Tigbe WW, Granat MH. The validity and reliability of a novel activity monitor as a measure of walking. Br J Sports Med. 2006;40(9):779–784. doi: 10.1136/bjsm.2006.027276.
    1. Tudor-Locke C, Sisson SB, Lee SM, Craig CL, Plotnikoff RC, Bauman A. Evaluation of quality of commercial pedometers. Can J Public Health. 2006;97:S10–S15.
    1. Portney L, Watkins M. Foundations of clinical research: applications to practice. Upper Saddle River, NJ: Pearson/Prentice Hall; 2009.
    1. Cohen J. A power primer. Psychol Bull. 1992;112(1):155. doi: 10.1037/0033-2909.112.1.155.
    1. Melanson EL, Knoll JR, Bell ML, Hill JO, Nysse LJ, Lanningham-Foster L, Peters JC, Levine JA. Commercially available pedometers: considerations for accurate step counting. Prev Med. 2004;39(2):361–368. doi: 10.1016/j.ypmed.2004.01.032.
    1. De Cocker KA, De Meyer J, De Bourdeaudhuij IM, Cardon GM. Non-traditional wearing positions of pedometers: validity and reliability of the Omron HJ-203-ED pedometer under controlled and free-living conditions. J Sci Med Sport. 2012;15(5):418–424. doi: 10.1016/j.jsams.2012.02.002.
    1. Beets MW, Patton MM, Edwards S. The accuracy of pedometer steps and time during walking in children. Med Sci Sports Exerc. 2005;37(3):513–520. doi: 10.1249/01.MSS.0000155395.49960.31.
    1. Martin Bland J, Altman D. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;327(8476):307–310. doi: 10.1016/S0140-6736(86)90837-8.
    1. Atallah L, Lo B, King R, Yang G. Sensor positioning for activity recognition using wearable accelerometers. IEEE Trans Biomed Circuits Syst. 2011;5(4):320–329. doi: 10.1109/TBCAS.2011.2160540.
    1. Mannini A, Intille SS, Rosenberger M, Sabatini AM, Haskell W. Activity recognition using a single accelerometer placed at the wrist or ankle. Med Sci Sports Exerc. 2013;45(11):2193–2203. doi: 10.1249/MSS.0b013e31829736d6.

Source: PubMed

Спонсоры и соавторы

Заболевания

Медикаментозные вмешательства

Подписаться