The validity of activity trackers is affected by walking speed: the criterion validity of Garmin Vivosmart® HR and StepWatch™ 3 for measuring steps at various walking speeds under controlled conditions

Frederik Rose Svarre, Mads Møller Jensen, Josephine Nielsen, Morten Villumsen, Frederik Rose Svarre, Mads Møller Jensen, Josephine Nielsen, Morten Villumsen

Abstract

Introduction: The use of activity trackers has increased both among private consumers and in healthcare. It is therefore relevant to consider whether a consumer-graded activity tracker is comparable to or may substitute a research-graded activity tracker, which could further increase the use of activity trackers in healthcare and rehabilitation. Such use will require knowledge of their accuracy as the clinical implications may be significant. Studies have indicated that activity trackers are not sufficiently accurate, especially at lower walking speeds. The present study seeks to inform decision makers and healthcare personnel considering implementing physical activity trackers in clinical practice. This study investigates the criterion validity of the consumer-graded Garmin Vivosmart® HR and the research-graded StepWatch™ 3 compared with manual step count (gold standard) at different walking speeds under controlled conditions.

Methods: Thirty participants, wearing Garmin Vivosmart® HR at the wrist and StepWatch™ 3 at the ankle, completed six trials on a treadmill at different walking speeds: 1.6 km/h, 2.4 km/h, 3.2 km/h, 4.0 km/h, 4.8 km/h, and 5.6 km/h. The participants were video recorded, and steps were registered by manual step count. Medians and inter-quartile ranges (IQR) were calculated for steps and differences in steps between manually counted steps and the two devices. In order to assess the clinical relevance of the tested devices, the mean absolute percentage error (MAPE) was determined at each speed. A MAPE ≤3% was considered to be clinically irrelevant. Furthermore, differences between manually counted steps and steps recorded by the two devices were presented in Bland-Altman style plots.

Results: The median of differences in steps between Garmin Vivosmart® HR and manual step count ranged from -49.5 (IQR = 101) at 1.6 km/h to -1 (IQR = 4) at 4.0 km/h. The median of differences in steps between StepWatch™ 3 and manual step count were 4 (IQR = 14) at 1.6 km/h and 0 (IQR = 1) at all other walking speeds. The results of the MAPE showed that differences in steps counted by Garmin Vivosmart® HR were clinically irrelevant at walking speeds 3.2-4.8 km/h (MAPE: 0.61-1.27%) as the values were below 3%. Differences in steps counted by StepWatch™ 3 were clinically irrelevant at walking speeds 2.4-5.6 km/h (MAPE: 0.08-0.35%).

Conclusion: Garmin Vivosmart® HR tended to undercount steps compared with the manual step count, and StepWatch™ 3 slightly overcounted steps compared with the manual step count. Both the consumer-graded activity tracker (Garmin Vivosmart® HR) and the research-graded (StepWatch™ 3) are valid in detecting steps at selected walking speeds in healthy adults under controlled conditions. However, both activity trackers miscount steps at slow walking speeds, and the consumer graded activity tracker also miscounts steps at fast walking speeds.

Keywords: Activity monitor; Activity tracker; Adults; Physical activity; Steps; Validity.

Conflict of interest statement

The authors declare that they have no competing interests.

© 2020 Svarre et al.

Figures

Figure 1. Illustration of the placement of…
Figure 1. Illustration of the placement of Garmin Vivosmart® HR.
Figure 2. Illustration of the placement of…
Figure 2. Illustration of the placement of StepWatch™ 3 monitors.
(A) Frontal view, (B) Sagittal view.
Figure 3. Bland Altman style plot illustrating…
Figure 3. Bland Altman style plot illustrating the differences between Garmin Vivosmart® HR and manual step count at walking speeds from 1.6 km/h to 5.6 km/h.
(A) 1.6 km/h, (B) 2.4 km/h, (C) 3.2 km/h, (D) 4.0 km/h, (E) 4.8 km/h, (F) 5.6 km/h.
Figure 4. Bland Altman style plot illustrating…
Figure 4. Bland Altman style plot illustrating the differences between StepWatch™ 3 and manual step count at walking speeds from 1.6 km/h to 5.6 km/h.
(A) 1.6 km/h, (B) 2.4 km/h, (C) 3.2 km/h, (D) 4.0 km/h, (E) 4.8 km/h, (F) 5.6 km/h.

References

    1. Aminian S, Hinckson EA. Examining the validity of the ActivPAL monitor in measuring posture and ambulatory movement in children. International Journal of Behavioral Nutrition and Physical Activity. 2012;9(1):119. doi: 10.1186/1479-5868-9-119.
    1. Anderson JL, Yoward LS, Green AJ. A study investigating the validity of an accelerometer in quantification of step count in adult hospital inpatients recovering from critical illness. Clinical Rehabilitation. 2019;33(5):936–942. doi: 10.1177/0269215519829893.
    1. Bergman RJ, Spellman JW, Hall ME, Bergman SM. Is there a valid app for that? Validity of a free pedometer iPhone application. Journal of Physical Activity & Health. 2012;9(5):670–676. doi: 10.1123/jpah.9.5.670.
    1. Bunn JA, Navalta JW, Fountaine CJ, Reece JD. Current state of commercial wearable technology in physical activity monitoring 2015–2017. International Journal of Exercise Science. 2018;11:503–515.
    1. Burton E, Hill KD, Lautenschlager NT, Thøgersen-Ntoumani C, Lewin G, Boyle E, Howie E. Reliability and validity of two fitness tracker devices in the laboratory and home environment for older community-dwelling people. BMC Geriatrics. 2018;18(1):103. doi: 10.1186/s12877-018-0793-4.
    1. Chigateri NG, Kerse N, Wheeler L, MacDonald B, Klenk J. Validation of an accelerometer for measurement of activity in frail older people. Gait & Posture. 2018;66:114–117. doi: 10.1016/j.gaitpost.2018.08.024.
    1. Colley RC, Barnes JD, Leblanc AG, Borghese M, Boyer C, Tremblay MS. Validity of the SC-StepMX pedometer during treadmill walking and running. Applied Physiology, Nutrition, and Metabolism. 2013;38(5):520–524. doi: 10.1139/apnm-2012-0321.
    1. Crouter SE, Schneider PL, Karabulut M, Bassett DR. Validity of 10 electronic pedometers for measuring steps, distance, and energy cost. Medicine and Science in Sports and Exercise. 2003;35(8):1455–1460. doi: 10.1249/01.MSS.0000078932.61440.A2.
    1. De Cocker KA, De Meyer J, De Bourdeaudhuij IM, Cardon GM. Non-traditional wearing positions of pedometers: validity and reliability of the Omron HJ-203-ED pedometer under controlled and free-living conditions. Journal of Science and Medicine in Sport/Sports Medicine Australia. 2012;15(5):418–424. doi: 10.1016/j.jsams.2012.02.002.
    1. Edbrooke L, Lythgo N, Goldsworthy U, Denehy L. Can an accelerometer-based monitor be used to accurately assess physical activity in a population of survivors of critical illness? Global Journal of Health Science. 2012;4(3):98–107. doi: 10.5539/gjhs.v4n3p98.
    1. Evenson KR, Goto MM, Furberg RD. Systematic review of the validity and reliability of consumer-wearable activity trackers. International Journal of Behavioral Nutrition and Physical Activity. 2015;12(1):159. doi: 10.1186/s12966-015-0314-1.
    1. Ferguson T, Rowlands AV, Olds T, Maher C. The validity of consumer-level, activity monitors in healthy adults worn in free-living conditions: a cross-sectional study. International Journal of Behavioral Nutrition and Physical Activity. 2015;12(1):42. doi: 10.1186/s12966-015-0201-9.
    1. Fokkema T, Kooiman TJM, Krijnen WP, Van Der Schans CP, De Groot M. Reliability and validity of ten consumer activity trackers depend on walking speed. Medicine and Science in Sports and Exercise. 2017;49(4):793–800. doi: 10.1249/MSS.0000000000001146.
    1. Fortune E, Lugade VA, Amin S, Kaufman KR. Step detection using multi-versus single tri-axial accelerometer-based systems. Physiological Measurement. 2015;36(12):2519–2535. doi: 10.1088/0967-3334/36/12/2519.
    1. Fortune E, Lugade V, Morrow M, Kaufman K. Validity of using tri-axial accelerometers to measure human movement—Part II: step counts at a wide range of gait velocities. Medical Engineering & Physics. 2014;36(6):659–669. doi: 10.1016/j.medengphy.2014.02.006.
    1. Garmin vivosmart HR | Garmin | Fitness Tracker. 2016a. . [16 May 2016].
    1. Garmin Sports Features | Garmin Connect. 2016b. [7 May 2016].
    1. Garmin . Garmin owner’s: Vivosmart® HR manual. Olathe: Garmin Ltd; 2016c.
    1. Garmin Previous Models Fitness Trackers | Garmin. 2020. . [12 April 2020].
    1. Gaz DV, Rieck TM, Peterson NW, Ferguson JA, Schroeder DR, Dunfee HA, Henderzahs-Mason JM, Hagen PT. Determining the validity and accuracy of multiple activity-tracking devices in controlled and free-walking conditions. American Journal of Health Promotion. 2018;32(8):1671–1678. doi: 10.1177/0890117118763273.
    1. Gialamas A, Laurence CO, Yelland LN, Tideman P, Worley P, Shephard MD, Tirimacco R, Willson KJ, Ryan P, Gill J, Thomas DW, Beilby JJ, Members of the PoCT Trial Management Committee Assessing agreement between point of care and laboratory results for lipid testing from a clinical perspective. Clinical Biochemistry. 2010;43(4–5):515–518. doi: 10.1016/j.clinbiochem.2009.11.014.
    1. Giannakidou DM, Kambas A, Ageloussis N, Fatouros I, Christoforidis C, Venetsanou F, Douroudos I, Taxildaris K. The validity of two Omron pedometers during treadmill walking is speed dependent. European Journal of Applied Physiology. 2012;112(1):49–57. doi: 10.1007/s00421-011-1951-y.
    1. GoPro GoPro Desktop App - Easily offload and enjoy your GoPro photos and videos, make quick edits and share your favorite shots. 2016. . [7 May 2016].
    1. Graham JE, Fisher SR, Bergés I-M, Kuo Y-F, Ostir GV. Walking speed threshold for classifying walking independence in hospitalized older adults. Physical Therapy. 2010;90(11):1591–1597. doi: 10.2522/ptj.20100018.
    1. Haskell WL, Lee I-M, Pate RR, Powell KE, Blair SN, Franklin BA, Macera CA, Heath GW, Thompson PD, Bauman A. Physical activity and public health: updated recommendation for adults from the American College of Sports Medicine and the American Heart Association. Medicine and Science in Sports and Exercise. 2007;39(8):1423–1434. doi: 10.1249/mss.0b013e3180616b27.
    1. Hendrick P, Boyd T, Low O, Takarangi K, Paterson M, Claydon L, Milosavljevic S. Construct validity of RT3 accelerometer: a comparison of level-ground and treadmill walking at self-selected speeds. Journal of Rehabilitation Research and Development. 2010;47(2):157–168. doi: 10.1682/JRRD.2009.04.0047.
    1. Hickey A, John D, Sasaki JE, Mavilia M, Freedson P. Validity of activity monitor step detection is related to movement patterns. Journal of Physical Activity & Health. 2015;13(2):145–153. doi: 10.1123/jpah.2015-0203.
    1. Holbrook EA, Barreira TV, Kang M. Validity and reliability of Omron pedometers for prescribed and self-paced walking. Medicine and Science in Sports and Exercise. 2009;41(3):670–674. doi: 10.1249/MSS.0b013e3181886095.
    1. Huang Y, Xu J, Yu B, Shull PB. Validity of FitBit, Jawbone UP, Nike+ and other wearable devices for level and stair walking. Gait & Posture. 2016;48:36–41. doi: 10.1016/j.gaitpost.2016.04.025.
    1. Johnson M, Meltz K, Hart K, Schmudlach M, Clarkson L, Borman K. Validity of the actical activity monitor for assessing steps and energy expenditure during walking. Journal of Sports Sciences. 2015;33(8):769–776. doi: 10.1080/02640414.2014.964747.
    1. Kooiman TJM, Dontje ML, Sprenger SR, Krijnen WP, Van der Schans CP, De Groot M. Reliability and validity of ten consumer activity trackers. BMC Sports Science, Medicine and Rehabilitation. 2015;7(1):24. doi: 10.1186/s13102-015-0018-5.
    1. Kottner J, Audigé L, Brorson S, Donner A, Gajewski BJ, Hróbjartsson A, Roberts C, Shoukri M, Streiner DL. Guidelines for reporting reliability and agreement studies (GRRAS) were proposed. Journal of Clinical Epidemiology. 2011;64(1):96–106. doi: 10.1016/j.jclinepi.2010.03.002.
    1. Kumahara H, Ayabe M, Ichibakase M, Tashima A, Chiwata M, Takashi T. Validity of activity monitors worn at multiple nontraditional locations under controlled and free-living conditions in young adult women. Applied Physiology, Nutrition, and Metabolism. 2015;40(5):448–456. doi: 10.1139/apnm-2014-0183.
    1. Le Masurier GC, Tudor-Locke C. Comparison of pedometer and accelerometer accuracy under controlled conditions. Medicine and Science in Sports and Exercise. 2003;35(5):867–871. doi: 10.1249/01.MSS.0000064996.63632.10.
    1. Lee I-M, Shiroma EJ, Kamada M, Bassett DR, Matthews CE, Buring JE. Association of step volume and intensity with all-cause mortality in older women. JAMA Internal Medicine. 2019;179(8):1105. doi: 10.1001/jamainternmed.2019.0899.
    1. Lee JA, Williams SM, Brown DD, Laurson KR. Concurrent validation of the Actigraph gt3x+, Polar active accelerometer, Omron HJ-720 and Yamax Digiwalker SW-701 pedometer step counts in lab-based and free-living settings. Journal of Sports Sciences. 2015;33(10):991–1000. doi: 10.1080/02640414.2014.981848.
    1. Lim SER, Ibrahim K, Sayer AA, Roberts HC. Assessment of physical activity of hospitalised older adults: a systematic review. Journal of Nutrition, Health & Aging. 2018;22(3):377–386. doi: 10.1007/s12603-017-0931-2.
    1. Liu S, Brooks D, Thomas S, Eysenbach G, Nolan RP. Lifesource XL-18 pedometer for measuring steps under controlled and free-living conditions. Journal of Sports Sciences. 2015;33(10):1001–1006. doi: 10.1080/02640414.2014.981847.
    1. Modus Health . StepWatch™ 3 activity monitor & USB docking station—Version 3.4 user manual. Washington, D.C.: Modus Health; 2014.
    1. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, Bouter LM, De Vet HCW. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Quality of Life Research. 2010a;19(4):539–549. doi: 10.1007/s11136-010-9606-8.
    1. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, Bouter LM, De Vet HCW. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. Journal of Clinical Epidemiology. 2010b;63(7):737–745. doi: 10.1016/j.jclinepi.2010.02.006.
    1. Riel H, Rathleff CR, Kalstrup PM, Madsen NK, Pedersen ES, Pape-Haugaard LB, Villumsen M. Comparison between mother, ActiGraph wGT3X-BT, and a hand tally for measuring steps at various walking speeds under controlled conditions. PeerJ. 2016;4(1):e2799. doi: 10.7717/peerj.2799.
    1. Rosenkranz RR, Rosenkranz SK, Weber C. Validity of the Actical accelerometer step-count function in children. Pediatric Exercise Science. 2011;23(3):355–365. doi: 10.1123/pes.23.3.355.
    1. Ryan CG, Grant PM, Tigbe WW, Granat MH. The validity and reliability of a novel activity monitor as a measure of walking. British Journal of Sports Medicine. 2006;40(9):779–784. doi: 10.1136/bjsm.2006.027276.
    1. Sellers C, Dall P, Grant M, Stansfield B. Validity and reliability of the activPAL3 for measuring posture and stepping in adults and young people. Gait & Posture. 2016;43:42–47. doi: 10.1016/j.gaitpost.2015.10.020.
    1. Shephard RJ. PAR-Q, Canadian home fitness test and exercise screening alternatives. Sports Medicine. 1988;5(3):185–195. doi: 10.2165/00007256-198805030-00005.
    1. Stansfield B, Hajarnis M, Sudarshan R. Characteristics of very slow stepping in healthy adults and validity of the activPAL3™ activity monitor in detecting these steps. Medical Engineering & Physics. 2015;37(1):42–47. doi: 10.1016/j.medengphy.2014.10.003.
    1. Steeves JA, Tyo BM, Connolly CP, Gregory DA, Stark NA, Bassett DR. Validity and reliability of the Omron HJ-303 tri-axial accelerometer-based pedometer. Journal of Physical Activity & Health. 2011;8(7):1014–1020. doi: 10.1123/jpah.8.7.1014.
    1. Takacs J, Pollock CL, Guenther JR, Bahar M, Napier C, Hunt MA. Validation of the Fitbit One activity monitor device during treadmill walking. Journal of Science and Medicine in Sport/Sports Medicine Australia. 2014;17(5):496–500. doi: 10.1016/j.jsams.2013.10.241.
    1. Taraldsen K, Askim T, Sletvold O, Einarsen E, Bjåstad K, Indredavik B, Helbostad J. Evaluation of a body-worn sensor system to measure physical activity in older people with impaired function. Physical Therapy. 2011;91(2):277–285. doi: 10.2522/ptj.20100159.
    1. Tedesco S, Sica M, Ancillao A, Timmons S, Barton J, O’Flynn B. Accuracy of consumer-level and research-grade activity trackers in ambulatory settings in older adults. PLOS ONE. 2019;14(5):e0216891. doi: 10.1371/journal.pone.0216891.
    1. Thompson WR. Worldwide survey of fitness trends for 2016: 10th Anniversary Edition. ACSM’s Health & Fitness Journal. 2015;19(6):9–18. doi: 10.1249/FIT.0000000000000164.
    1. Thompson WR. Worldwide survey of fitness trends for 2017. ACSM’s Health & Fitness Journal. 2016;20(6):8–17. doi: 10.1249/FIT.0000000000000252.
    1. Thompson WR. Worldwide survey of fitness trends for 2019. ACSM’s Health & Fitness Journal. 2018;22(6):10–17. doi: 10.1249/FIT.0000000000000438.
    1. Thompson WR. Worldwide survey of fitness trends for 2020. ACSM’s Health & Fitness Journal. 2019;23(6):10–18. doi: 10.1249/FIT.0000000000000526.
    1. Treacy D, Hassett L, Schurr K, Chagpar S, Paul SS, Sherrington C. Validity of different activity monitors to count steps in an inpatient rehabilitation setting. Physical Therapy. 2017;97(5):581–588. doi: 10.1093/ptj/pzx010.
    1. Tudor-Locke C, Bassett DR, Shipe MF, McClain JJ. Pedometry methods for assessing free-living adults. Journal of Physical Activity & Health. 2011a;8(3):445–453. doi: 10.1123/jpah.8.3.445.
    1. Tudor-Locke C, Craig CL, Brown WJ, Clemes SA, De Cocker K, Giles-Corti B, Hatano Y, Inoue S, Matsudo SM, Mutrie N, Oppert J-M, Rowe DA, Schmidt MD, Schofield GM, Spence JC, Teixeira PJ, Tully MA, Blair SN. How many steps/day are enough? For adults. International Journal of Behavioral Nutrition and Physical Activity. 2011b;8(1):79. doi: 10.1186/1479-5868-8-79.
    1. Tudor-Locke C, Williams JE, Reis JP, Pluto D. Utility of pedometers for assessing physical activity: convergent validity. Sports Medicine. 2002;32(12):795–808. doi: 10.2165/00007256-200232120-00004.
    1. Van Remoortel H, Giavedoni S, Raste Y, Burtin C, Louvaris Z, Gimeno-Santos E, Langer D, Glendenning A, Hopkinson NS, Vogiatzis I, Peterson BT, Wilson F, Mann B, Rabinovich R, Puhan MA, Troosters T, PROactive consortium Validity of activity monitors in health and chronic disease: a systematic review. International Journal of Behavioral Nutrition and Physical Activity. 2012;9(1):84. doi: 10.1186/1479-5868-9-84.
    1. zebris Medical GmbH . FDM-T technical specifications and user manual. Isny im Allgä: zebris Medical GmbH; 2015.

Source: PubMed

Sponsors en medewerkers

Medische omstandigheden

Geneesmiddelinterventies

3
Abonneren