Measuring Free-Living Physical Activity With Three Commercially Available Activity Monitors for Telemonitoring Purposes: Validation Study

Martine Jm Breteler, Joris H Janssen, Wilko Spiering, Cor J Kalkman, Wouter W van Solinge, Daan Aj Dohmen, Martine Jm Breteler, Joris H Janssen, Wilko Spiering, Cor J Kalkman, Wouter W van Solinge, Daan Aj Dohmen

Abstract

Background: Remote monitoring of physical activity in patients with chronic conditions could be useful to offer care professionals real-time assessment of their patient's daily activity pattern to adjust appropriate treatment. However, the validity of commercially available activity trackers that can be used for telemonitoring purposes is limited.

Objective: The purpose of this study was to test usability and determine the validity of 3 consumer-level activity trackers as a measure of free-living activity.

Methods: A usability evaluation (study 1) and validation study (study 2) were conducted. In study 1, 10 individuals wore one activity tracker for a period of 30 days and filled in a questionnaire on ease of use and wearability. In study 2, we validated three selected activity trackers (Apple Watch, Misfit Shine, and iHealth Edge) and a fourth pedometer (Yamax Digiwalker) against the reference standard (Actigraph GT3X) in 30 healthy participants for 72 hours. Outcome measures were 95% limits of agreement (LoA) and bias (Bland-Altman analysis). Furthermore, median absolute differences (MAD) were calculated. Correction for bias was estimated and validated using leave-one-out cross validation.

Results: Usability evaluation of study 1 showed that iHealth Edge and Apple Watch were more comfortable to wear as compared with the Misfit Flash. Therefore, the Misfit Flash was replaced by Misfit Shine in study 2. During study 2, the total number of steps of the reference standard was 21,527 (interquartile range, IQR 17,475-24,809). Bias and LoA for number of steps from the Apple Watch and iHealth Edge were 968 (IQR -5478 to 7414) and 2021 (IQR -4994 to 9036) steps. For Misfit Shine and Yamax Digiwalker, bias was -1874 and 2004, both with wide LoA of (13,869 to 10,121) and (-10,932 to 14,940) steps, respectively. The Apple Watch noted the smallest MAD of 7.7% with the Actigraph, whereas the Yamax Digiwalker noted the highest MAD (20.3%). After leave-one-out cross validation, accuracy estimates of MAD of the iHealth Edge and Misfit Shine were within acceptable limits with 10.7% and 11.3%, respectively.

Conclusions: Overall, the Apple Watch and iHealth Edge were positively evaluated after wearing. Validity varied widely between devices, with the Apple Watch being the most accurate and Yamax Digiwalker the least accurate for step count in free-living conditions. The iHealth Edge underestimates number of steps but can be considered reliable for activity monitoring after correction for bias. Misfit Shine overestimated number of steps and cannot be considered suitable for step count because of the low agreement. Future studies should focus on the added value of remotely monitoring activity patterns over time in chronic patients.

Keywords: activity trackers; exercise; telemedicine.

Conflict of interest statement

Conflicts of Interest: At the time of the study, MB and JJ were employees of FocusCura (Health IT company, Driebergen-Zeist, The Netherlands), and DD is founder and CEO of FocusCura.

©Martine JM Breteler, Joris H Janssen, Wilko Spiering, Cor J Kalkman, Wouter W van Solinge, Daan AJ Dohmen. Originally published in JMIR Formative Research (http://formative.jmir.org), 24.04.2019.

Figures

Figure 1
Figure 1
Bland-Altman plot for Apple Watch versus Actigraph step counts over a 72 hour period (n=30).
Figure 2
Figure 2
Bland-Altman plot for iHealth Edge versus Actigraph steps over a 72 hour period (n=30).
Figure 3
Figure 3
Bland-Altman plot for Misfit Shine versus Actigraph steps over a 72 hour period (n=21).
Figure 4
Figure 4
Bland-Altman plot for Yamax Digiwalker versus Actigraph steps over a 72 hour period (n=30).
Figure 5
Figure 5
Example of the number of steps over time of the different activity trackers within one participant. The red arrow is pointed towards the number of steps of Misfit Shine that seems to add number of steps overnight or at the beginning of a day.

References

    1. Kolt GS, Rosenkranz RR, Savage TN, Maeder AJ, Vandelanotte C, Duncan MJ, Caperchione CM, Tague R, Hooker C, Mummery WK. WALK 2.0 - using Web 2.0 applications to promote health-related physical activity: a randomised controlled trial protocol. BMC Public Health. 2013 May 03;13:436. doi: 10.1186/1471-2458-13-436.
    1. Davies CA, Spence JC, Vandelanotte C, Caperchione CM, Mummery WK. Meta-analysis of internet-delivered interventions to increase physical activity levels. Int J Behav Nutr Phys Act. 2012;9:52. doi: 10.1186/1479-5868-9-52.
    1. Hurling R, Catt M, Boni MD, Fairley BW, Hurst T, Murray P, Richardson A, Sodhi JS. Using internet and mobile phone technology to deliver an automated physical activity program: randomized controlled trial. J Med Internet Res. 2007;9(2):e7. doi: 10.2196/jmir.9.2.e7.
    1. Noah B, Keller MS, Mosadeghi S, Stein L, Johl S, Delshad S, Tashjian VC, Lew D, Kwan JT, Jusufagic A, Spiegel BM. Impact of remote patient monitoring on clinical outcomes: an updated meta-analysis of randomized controlled trials. NPJ Digit Med. 2018 Jan 15;1(1) doi: 10.1038/s41746-017-0002-4.
    1. Masala G, Bendinelli B, Occhini D, Bruno RM, Caini S, Saieva C, Ungar A, Ghiadoni L, Palli D. Physical activity and blood pressure in 10,000 Mediterranean adults: the EPIC-Florence cohort. Nutr Metab Cardiovasc Dis. 2017 Aug;27(8):670–78. doi: 10.1016/j.numecd.2017.06.003.
    1. Diaz KM, Shimbo D. Physical activity and the prevention of hypertension. Curr Hypertens Rep. 2013 Dec;15(6):659–68. doi: 10.1007/s11906-013-0386-8.
    1. Howard VJ, McDonnell MN. Physical activity in primary stroke prevention: just do it! Stroke. 2015 Jun;46(6):1735–9. doi: 10.1161/STROKEAHA.115.006317.
    1. Warburton DE, Bredin SS. Health benefits of physical activity: a systematic review of current systematic reviews. Curr Opin Cardiol. 2017 Sep;32(5):541–56. doi: 10.1097/HCO.0000000000000437.
    1. Lv N, Xiao L, Simmons ML, Rosas LG, Chan A, Entwistle M. Personalized hypertension Management Using Patient-Generated Health Data Integrated With Electronic Health Records (EMPOWER-H): six-month pre-post study. J Med Internet Res. 2017 Sep 19;19(9):e311. doi: 10.2196/jmir.7831.
    1. Waschki B, Kirsten A, Holz O, Müller KC, Meyer T, Watz H, Magnussen H. Physical activity is the strongest predictor of all-cause mortality in patients with COPD: a prospective cohort study. Chest. 2011 Aug;140(2):331–42. doi: 10.1378/chest.10-2521.
    1. Garcia-Aymerich J, Serra I, Gómez FP, Farrero E, Balcells E, Rodríguez DA, de Batlle J, Gimeno E, Donaire-Gonzalez D, Orozco-Levi M, Sauleda J, Gea J, Rodriguez-Roisin R, Roca J, Agustí AG, Antó JM, PhenotypeCourse of COPD (PAC-COPD) Study Group Physical activity and clinical and functional status in COPD. Chest. 2009 Jul;136(1):62–70. doi: 10.1378/chest.08-2532.
    1. Nguyen HQ, Chu L, Amy LI, Lee JS, Suh D, Korotzer B, Yuen G, Desai S, Coleman KJ, Xiang AH, Gould MK. Associations between physical activity and 30-day readmission risk in chronic obstructive pulmonary disease. Ann Am Thorac Soc. 2014 Jun;11(5):695–705. doi: 10.1513/AnnalsATS.201401-017OC.
    1. Agboola S, Jethwani K, Lopez L, Searl M, O'Keefe S, Kvedar J. Text to move: a randomized controlled trial of a text-messaging program to improve physical activity behaviors in patients with type 2 diabetes mellitus. J Med Internet Res. 2016 Nov 18;18(11):e307. doi: 10.2196/jmir.6439.
    1. Dasgupta K, Rosenberg E, Joseph L, Cooke AB, Trudeau L, Bacon SL, Chan D, Sherman M, Rabasa-Lhoret R, Daskalopoulou SS, SMARTER Trial Group Physician step prescription and monitoring to improve ARTERial health (SMARTER): a randomized controlled trial in patients with type 2 diabetes and hypertension. Diabetes Obes Metab. 2017 Dec;19(5):695–704. doi: 10.1111/dom.12874.
    1. Mercer K, Giangregorio L, Schneider E, Chilana P, Li M, Grindrod K. Acceptance of commercially available wearable activity trackers among adults aged over 50 and with chronic illness: a mixed-methods evaluation. JMIR Mhealth Uhealth. 2016;4(1):e7. doi: 10.2196/mhealth.4225.
    1. Mercer K, Li M, Giangregorio L, Burns C, Grindrod K. Behavior change techniques present in wearable activity trackers: a critical analysis. JMIR Mhealth Uhealth. 2016;4(2):e40. doi: 10.2196/mhealth.4461.
    1. Chiauzzi E, Rodarte C, DasMahapatra P. Patient-centered activity monitoring in the self-management of chronic health conditions. BMC Med. 2015;13:77. doi: 10.1186/s12916-015-0319-2.
    1. Tully MA, McBride C, Heron L, Hunter RF. The validation of Fibit Zip™ physical activity monitor as a measure of free-living physical activity. BMC Res Notes. 2014;7:952. doi: 10.1186/1756-0500-7-952.
    1. Alharbi M, Bauman A, Neubeck L, Gallagher R. Validation of Fitbit-Flex as a measure of free-living physical activity in a community-based phase III cardiac rehabilitation population. Eur J Prev Cardiol. 2016 Dec;23(14):1476–85. doi: 10.1177/2047487316634883.
    1. Wen D, Zhang X, Liu X, Lei J. Evaluating the consistency of current mainstream wearable devices in health monitoring: a comparison under free-living conditions. J Med Internet Res. 2017 Mar 07;19(3):e68. doi: 10.2196/jmir.6874.
    1. Sullivan AN, Lachman ME. Behavior change with fitness technology in sedentary adults: a review of the evidence for increasing physical activity. Front Public Health. 2016;4:289. doi: 10.3389/fpubh.2016.00289. doi: 10.3389/fpubh.2016.00289.
    1. Tabak M, Vollenbroek-Hutten M, van der Valk PD, van der Palen J, Hermens H. A telerehabilitation intervention for patients with chronic obstructive pulmonary disease: a randomized controlled pilot trial. Clin Rehabil. 2014 Jun;28(6):582–91. doi: 10.1177/0269215513512495.
    1. Qiu S, Cai X, Wang X, He C, Zügel M, Steinacker JM, Schumann U. Using step counters to promote physical activity and exercise capacity in patients with chronic obstructive pulmonary disease: a meta-analysis. Ther Adv Respir Dis. 2018;12:1753466618787386. doi: 10.1177/1753466618787386.
    1. Lu TC, Fu CM, Ma MH, Fang CC, Turner AM. Healthcare applications of smart watches. A systematic review. Appl Clin Inform. 2016 Dec 14;7(3):850–69. doi: 10.4338/ACI-2016-03-R-0042.
    1. Takacs J, Pollock C, Guenther J, Bahar M, Napier C, Hunt M. Validation of the Fitbit One activity monitor device during treadmill walking. J Sci Med Sport. 2014 Sep;17(5):496–500. doi: 10.1016/j.jsams.2013.10.241.
    1. Schneider PL, Crouter SE, Lukajic O, Bassett DR. Accuracy and reliability of 10 pedometers for measuring steps over a 400-m walk. Med Sci Sports Exerc. 2003 Oct;35(10):1779–84. doi: 10.1249/01.MSS.0000089342.96098.C4.
    1. Leth S, Hansen J, Nielsen OW, Dinesen B. Evaluation of commercial self-monitoring devices for clinical purposes: results from the future patient trial, phase I. Sensors (Basel) 2017 Jan 22;17(1) doi: 10.3390/s17010211.
    1. Fokkema T, Kooiman TJ, Krijnen WP. Reliability and validity of ten consumer activity trackers depend on walking speed. Med Sci Sports Exerc. 2017 Apr;49(4):793–800. doi: 10.1249/MSS.0000000000001146.
    1. Kooiman TJ, Dontje ML, Sprenger SR, Krijnen WP, van der Schans CP, de Groot M. Reliability and validity of ten consumer activity trackers. BMC Sports Sci Med Rehabil. 2015;7:24. doi: 10.1186/s13102-015-0018-5.
    1. Garatachea N, Torres Luque G, González Gallego J. Physical activity and energy expenditure measurements using accelerometers in older adults. Nutr Hosp. 2010;25(2):224–30.
    1. Santos-Lozano A, Santín-Medeiros F, Cardon G, Torres-Luque G, Bailón R, Bergmeir C, Ruiz JR, Lucia A, Garatachea N. Actigraph GT3X: validation and determination of physical activity intensity cut points. Int J Sports Med. 2013 Nov;34(11):975–82. doi: 10.1055/s-0033-1337945.
    1. Van Blarigan EL, Kenfield SA, Tantum L, Cadmus-Bertram LA, Carroll PR, Chan JM. The Fitbit One physical activity tracker in men with prostate cancer: validation study. JMIR Cancer. 2017 Apr 18;3(1):e5. doi: 10.2196/cancer.6935.
    1. Ferguson T, Rowlands AV, Olds T, Maher C. The validity of consumer-level, activity monitors in healthy adults worn in free-living conditions: a cross-sectional study. Int J Behav Nutr Phys Act. 2015;12:42. doi: 10.1186/s12966-015-0201-9.
    1. Lee JA, Williams SM, Brown DD, Laurson KR. Concurrent validation of the Actigraph gt3x+, Polar Active accelerometer, Omron HJ-720 and Yamax Digiwalker SW-701 pedometer step counts in lab-based and free-living settings. J Sports Sci. 2015;33(10):991–1000. doi: 10.1080/02640414.2014.981848.
    1. Martin Bland J, Altman D. Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet. 1986 Feb;327(8476):307–310. doi: 10.1016/S0140-6736(86)90837-8.
    1. Kooiman T, Dontje M, Sprenger S, Krijnen W, van der Schans Cees P, de Groot Martijn. Reliability and validity of ten consumer activity trackers. BMC Sports Sci Med Rehabil. 2015;7:24. doi: 10.1186/s13102-015-0018-5.
    1. Patel S, Park H, Bonato P, Chan L, Rodgers M. A review of wearable sensors and systems with application in rehabilitation. J Neuroeng Rehabil. 2012;9:21. doi: 10.1186/1743-0003-9-21.
    1. Finkelstein EA, Haaland BA, Bilger M, Sahasranaman A, Sloan RA, Nang EE, Evenson KR. Effectiveness of activity trackers with and without incentives to increase physical activity (TRIPPA): a randomised controlled trial. Lancet Diabetes Endocrinol. 2016 Dec;4(12):983–995. doi: 10.1016/S2213-8587(16)30284-4.
    1. Deci E, Koestner R, Ryan RM. A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychol Bull. 1999;125(6):627–68. doi: 10.1037/0033-2909.125.6.627.
    1. Straiton N, Alharbi M, Bauman A, Neubeck L, Gullick J, Bhindi R, Gallagher R. The validity and reliability of consumer-grade activity trackers in older, community-dwelling adults: a systematic review. Maturitas. 2018 Jun;112:85–93. doi: 10.1016/j.maturitas.2018.03.016.
    1. Van Remoortel H, Raste Y, Louvaris Z, Giavedoni S, Burtin C, Langer D, Wilson F, Rabinovich R, Vogiatzis I, Hopkinson NS, Troosters T, PROactive C. Validity of six activity monitors in chronic obstructive pulmonary disease: a comparison with indirect calorimetry. PLoS One. 2012;7(6):e39198. doi: 10.1371/journal.pone.0039198.
    1. Tudor-Locke C, Ainsworth BE, Thompson RW, Matthews CE. Comparison of pedometer and accelerometer measures of free-living physical activity. Med Sci Sports Exerc. 2002 Dec;34(12):2045–51. doi: 10.1249/01.MSS.0000039300.76400.16.
    1. El-Amrawy F, Nounou M. Are currently available wearable devices for activity tracking and heart rate monitoring accurate, precise, and medically beneficial? Healthc Inform Res. 2015 Oct;21(4):315–20. doi: 10.4258/hir.2015.21.4.315.
    1. Farina N, Lowry RG. The validity of consumer-level activity monitors in healthy older adults in free-living conditions. J Aging Phys Act. 2018 Dec 01;26(1):128–35. doi: 10.1123/japa.2016-0344.
    1. Ehrler F, Weber C, Lovis C. Influence of pedometer position on pedometer accuracy at various walking speeds: a comparative study. J Med Internet Res. 2016 Oct 6;18(10):e268. doi: 10.2196/jmir.5916.
    1. Schneider PL, Crouter SE, Bassett DR. Pedometer measures of free-living physical activity: comparison of 13 models. Med Sci Sports Exerc. 2004 Feb;36(2):331–5. doi: 10.1249/01.MSS.0000113486.60548.E9.

Source: PubMed

Sponsors en medewerkers

Medische omstandigheden

Geneesmiddelinterventies

3
Abonneren