User Engagement and Attrition in an App-Based Physical Activity Intervention: Secondary Analysis of a Randomized Controlled Trial

Sarah Edney, Jillian C Ryan, Tim Olds, Courtney Monroe, François Fraysse, Corneel Vandelanotte, Ronald Plotnikoff, Rachel Curtis, Carol Maher, Sarah Edney, Jillian C Ryan, Tim Olds, Courtney Monroe, François Fraysse, Corneel Vandelanotte, Ronald Plotnikoff, Rachel Curtis, Carol Maher

Abstract

Background: The success of a mobile phone app in changing health behavior is thought to be contingent on engagement, commonly operationalized as frequency of use.

Objective: This subgroup analysis of the 2 intervention arms from a 3-group randomized controlled trial aimed to examine user engagement with a 100-day physical activity intervention delivered via an app. Rates of engagement, associations between user characteristics and engagement, and whether engagement was related to intervention efficacy were examined.

Methods: Engagement was captured in a real-time log of interactions by users randomized to either a gamified (n=141) or nongamified version of the same app (n=160). Physical activity was assessed via accelerometry and self-report at baseline and 3-month follow-up. Survival analysis was used to assess time to nonuse attrition. Mixed models examined associations between user characteristics and engagement (total app use). Characteristics of super users (top quartile of users) and regular users (lowest 3 quartiles) were compared using t tests and a chi-square analysis. Linear mixed models were used to assess whether being a super user was related to change in physical activity over time.

Results: Engagement was high. Attrition (30 days of nonuse) occurred in 32% and 39% of the gamified and basic groups, respectively, with no significant between-group differences in time to attrition (P=.17). Users with a body mass index (BMI) in the healthy range had higher total app use (mean 230.5, 95% CI 190.6-270.5; F2=8.67; P<.001), compared with users whose BMI was overweight or obese (mean 170.6, 95% CI 139.5-201.6; mean 132.9, 95% CI 104.8-161.0). Older users had higher total app use (mean 200.4, 95% CI 171.9-228.9; F1=6.385; P=.01) than younger users (mean 155.6, 95% CI 128.5-182.6). Super users were 4.6 years older (t297=3.6; P<.001) and less likely to have a BMI in the obese range (χ22=15.1; P<.001). At the 3-month follow-up, super users were completing 28.2 (95% CI 9.4-46.9) more minutes of objectively measured physical activity than regular users (F1,272=4.76; P=.03).

Conclusions: Total app use was high across the 100-day intervention period, and the inclusion of gamified features enhanced engagement. Participants who engaged the most saw significantly greater increases to their objectively measured physical activity over time, supporting the theory that intervention exposure is linked to efficacy. Further research is needed to determine whether these findings are replicated in other app-based interventions, including those experimentally evaluating engagement and those conducted in real-world settings.

Trial registration: Australian New Zealand Clinical Trials Registry ACTRN12617000113358; https://www.anzctr.org.au/ACTRN12617000113358.aspx.

Keywords: behavior; physical activity; smartphone.

Conflict of interest statement

Conflicts of Interest: None declared.

©Sarah Martine Edney, Jillian C Ryan, Tim Olds, Courtney Monroe, François Fraysse, Corneel Vandelanotte, Ronald Plotnikoff, Rachel Curtis, Carol Maher. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 27.11.2019.

Figures

Figure 1
Figure 1
Screenshots of the Active Team app Showing (left to right): splashscreen, step calendar, and virtual gifts.
Figure 2
Figure 2
Daily active users and total app use.
Figure 3
Figure 3
Kaplan-Meier survival estimates showing time to nonuse attrition, 30-day nonuse threshold, and total app use.
Figure 4
Figure 4
Kaplan-Meier survival estimates showing time to non-use attrition 14-day non-use threshold - total app use.

References

    1. Kohl LF, Crutzen R, de Vries NK. Online prevention aimed at lifestyle behaviors: a systematic review of reviews. J Med Internet Res. 2013 Jul 16;15(7):e146. doi: 10.2196/jmir.2665.
    1. Lee M, Lee H, Kim Y, Kim J, Cho M, Jang J, Jang H. Mobile app-based health promotion programs: a systematic review of the literature. Int J Environ Res Public Health. 2018 Dec 13;15(12):pii: E2838. doi: 10.3390/ijerph15122838.
    1. Müller AM, Maher CA, Vandelanotte C, Hingle M, Middelweerd A, Lopez ML, DeSmet A, Short CE, Nathan N, Hutchesson MJ, Poppe L, Woods CB, Williams SL, Wark PA. Physical activity, sedentary behavior, and diet-related eHealth and mHealth research: bibliometric analysis. J Med Internet Res. 2018 Apr 18;20(4):e122. doi: 10.2196/jmir.8954.
    1. Maher C, Ferguson M, Vandelanotte C, Plotnikoff R, de Bourdeaudhuij I, Thomas S, Nelson-Field K, Olds T. A web-based, social networking physical activity intervention for insufficiently active adults delivered via Facebook app: randomized controlled trial. J Med Internet Res. 2015 Jul 13;17(7):e174. doi: 10.2196/jmir.4086.
    1. Wing RR, Pinto AM, Crane MM, Kumar R, Weinberg BM, Gorin AA. A statewide intervention reduces BMI in adults: Shape Up Rhode Island results. Obesity (Silver Spring) 2009 May;17(5):991–5. doi: 10.1038/oby.2008.655. doi: 10.1038/oby.2008.655.
    1. Gal R, May AM, van Overmeeren EJ, Simons M, Monninkhof EM. The effect of physical activity interventions comprising wearables and smartphone applications on physical activity: a systematic review and meta-analysis. Sports Med Open. 2018 Sep 3;4(1):42. doi: 10.1186/s40798-018-0157-9.
    1. Romeo A, Edney S, Plotnikoff R, Curtis R, Ryan J, Sanders I, Crozier A, Maher C. Can smartphone apps increase physical activity? Systematic review and meta-analysis. J Med Internet Res. 2019 Mar 19;21(3):e12053. doi: 10.2196/12053.
    1. Maher CA, Lewis LK, Ferrar K, Marshall S, de Bourdeaudhuij I, Vandelanotte C. Are health behavior change interventions that use online social networks effective? A systematic review. J Med Internet Res. 2014 Feb 14;16(2):e40. doi: 10.2196/jmir.2952.
    1. Laranjo L, Arguel A, Neves AL, Gallagher AM, Kaplan R, Mortimer N, Mendes GA, Lau AY. The influence of social networking sites on health behavior change: a systematic review and meta-analysis. J Am Med Inform Assoc. 2015 Jan;22(1):243–56. doi: 10.1136/amiajnl-2014-002841.
    1. Cavallo DN, Tate DF, Ward DS, DeVellis RF, Thayer LM, Ammerman AS. Social support for physical activity-role of Facebook with and without structured intervention. Transl Behav Med. 2014 Dec;4(4):346–54. doi: 10.1007/s13142-014-0269-9.
    1. Perski O, Blandford A, West R, Michie S. Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis. Transl Behav Med. 2017 Jun;7(2):254–67. doi: 10.1007/s13142-016-0453-1.
    1. Short CE, DeSmet A, Woods C, Williams SL, Maher C, Middelweerd A, Müller AM, Wark PA, Vandelanotte C, Poppe L, Hingle MD, Crutzen R. Measuring engagement in eHealth and mHealth behavior change interventions: viewpoint of methodologies. J Med Internet Res. 2018 Nov 16;20(11):e292. doi: 10.2196/jmir.9397.
    1. Pham Q, Graham G, Carrion C, Morita PP, Seto E, Stinson JN, Cafazzo JA. A library of analytic indicators to evaluate effective engagement with consumer mHealth apps for chronic conditions: scoping review. JMIR Mhealth Uhealth. 2019 Jan 18;7(1):e11941. doi: 10.2196/11941.
    1. Dennison L, Morrison L, Conway G, Yardley L. Opportunities and challenges for smartphone applications in supporting health behavior change: qualitative study. J Med Internet Res. 2013 Apr 18;15(4):e86. doi: 10.2196/jmir.2583.
    1. Bonnie E. CleverTap. 2017. [2019-07-25]. Churn Rate: How to Define and Calculate Customer Churn.
    1. Global App Testing. 2019. [2019-07-25]. 6 Secrets of App Stickiness. .
    1. NewGenApps. 2017. [2019-06-25]. 10 Key Metrics to Measure User Engagement in Mobile Apps. .
    1. Valdellon L. CleverTap. 2018. [2019-06-25]. How to Measure Your App’s Health with User Engagement Metrics.
    1. Rojek R. Influencive. 2017. [2019-06-25]. Recognize Your Super Users and Accelerate Your App Growth.
    1. Vandelanotte C, Müller AM, Short CE, Hingle M, Nathan N, Williams SL, Lopez ML, Parekh S, Maher CA. Past, present, and future of eHealth and mHealth research to improve physical activity and dietary behaviors. J Nutr Educ Behav. 2016 Mar;48(3):219–228.e1. doi: 10.1016/j.jneb.2015.12.006.
    1. Vandelanotte C, Maher CA. Why we need more than just randomized controlled trials to establish the effectiveness of online social networks for health behavior change. Am J Health Promot. 2015;30(2):74–6. doi: 10.4278/ajhp.141204-CIT-605.
    1. Vandelanotte C, Kolt GS, Caperchione CM, Savage TN, Rosenkranz RR, Maeder AJ, van Itallie A, Tague R, Oldmeadow C, Mummery WK, Duncan MJ. Effectiveness of a web 2.0 intervention to increase physical activity in real-world settings: randomized ecological trial. J Med Internet Res. 2017 Nov 13;19(11):e390. doi: 10.2196/jmir.8484.
    1. Edney S, Plotnikoff R, Vandelanotte C, Olds T, de Bourdeaudhuij I, Ryan J, Maher C. "Active Team" a social and gamified app-based physical activity intervention: randomised controlled trial study protocol. BMC Public Health. 2017 Nov 2;17(1):859. doi: 10.1186/s12889-017-4882-7.
    1. Deterding S, Dixon D, Khaled R, Nacke L. From Game Design Elements to Gamefulness: Defining 'Gamification'. MindTrek'11; Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments; September 28-30, 2011; Tampere, Finland. 2011. pp. 9–15.
    1. Esliger DW, Rowlands AV, Hurst TL, Catt M, Murray P, Eston RG. Validation of the GENEA accelerometer. Med Sci Sports Exerc. 2011 Jun;43(6):1085–93. doi: 10.1249/MSS.0b013e31820513be.
    1. Australian Institute of Health and Welfare . The Active Australia Survey: A Guide and Manual for Implementation, Analysis and Reporting. Canberra: AIHW; 2003.
    1. Iqbal M. Business of Apps. 2019. [2019-06-30]. Pokémon GO Revenue and Usage Statistics (2019).
    1. Kolt GS, Rosenkranz RR, Vandelanotte C, Caperchione CM, Maeder AJ, Tague R, Savage TN, Van IA, Mummery WK, Oldmeadow C, Duncan MJ. Using Web 2.0 applications to promote health-related physical activity: findings from the WALK 2.0 randomised controlled trial. Br J Sports Med. 2017 Oct;51(19):1433–40. doi: 10.1136/bjsports-2016-096890.
    1. Guertler D, Vandelanotte C, Kirwan M, Duncan MJ. Engagement and nonusage attrition with a free physical activity promotion program: the case of 10,000 steps Australia. J Med Internet Res. 2015 Jul 15;17(7):e176. doi: 10.2196/jmir.4339.
    1. Kaplan EL, Meier P. Nonparametric estimation from incomplete observations. J Am Stat Assoc. 1958 Jun;53(282):457–81. doi: 10.1080/01621459.1958.10501452.
    1. Serrano KJ, Coa KI, Yu M, Wolff-Hughes DL, Atienza AA. Characterizing user engagement with health app data: a data mining approach. Transl Behav Med. 2017 Jun;7(2):277–85. doi: 10.1007/s13142-017-0508-y.
    1. Statista. 2019. [2019-06-30]. Percentage of Mobile Apps That Have Been Used Only Once During the First Six Months of Ownership From 2010 to 2018.
    1. Thrul J, Klein AB, Ramo DE. Smoking cessation intervention on Facebook: which content generates the best engagement? J Med Internet Res. 2015 Nov 11;17(11):e244. doi: 10.2196/jmir.4575.
    1. Ryan J, Edney S, Maher C. Engagement, compliance and retention with a gamified online social networking physical activity intervention. Transl Behav Med. 2017 Dec;7(4):702–8. doi: 10.1007/s13142-017-0499-8.
    1. Rodde T. Localytics. 2018. [2019-06-30]. 21% of Users Abandon an App After One Use. .
    1. Eysenbach G. The law of attrition. J Med Internet Res. 2005 Mar 31;7(1):e11. doi: 10.2196/jmir.7.1.e11.
    1. Kelders SM, Kok RN, Ossebaard HC, van Gemert-Pijnen JE. Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res. 2012 Nov 14;14(6):e152. doi: 10.2196/jmir.2104.
    1. Shin G, Jarrahi MH, Fei Y, Karami A, Gafinowitz N, Byun A, Lu X. Wearable activity trackers, accuracy, adoption, acceptance and health impact: a systematic literature review. J Biomed Inform. 2019 May;93:103153. doi: 10.1016/j.jbi.2019.103153.
    1. Clawson J, Pater J, Miller A, Mynatt E, Mamykina L. No Longer Wearing: Investigating the Abandonment of Personal Health-tracking Technologies on Craigslist. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing; UbiComp'15; September 7-11, 2015; Osaka, Japan. 2015. pp. 07–11.
    1. Maher C, Ryan J, Ambrosi C, Edney S. Users' experiences of wearable activity trackers: a cross-sectional study. BMC Public Health. 2017 Nov 15;17(1):880. doi: 10.1186/s12889-017-4888-1.
    1. Venkatesh V, Thong JY, Xu X. Consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. MIS Quarterly. 2012;36(1):157–78. doi: 10.2307/41410412.
    1. Carpenter BD, Buday S. Computer use among older adults in a naturally occurring retirement community. Comp Hum Behav. 2007 Nov;23(6):3012–24. doi: 10.1016/j.chb.2006.08.015. doi: 10.1016/j.chb.2006.08.015.
    1. Nigg CR, Burbank PM, Padula C, Dufresne R, Rossi JS, Velicer WF, Laforge RG, Prochaska JO. Stages of change across ten health risk behaviors for older adults. Gerontologist. 1999 Aug;39(4):473–82. doi: 10.1093/geront/39.4.473.
    1. Davies C, Corry K, van Itallie A, Vandelanotte C, Caperchione C, Mummery WK. Prospective associations between intervention components and website engagement in a publicly available physical activity website: the case of 10,000 Steps Australia. J Med Internet Res. 2012 Jan 11;14(1):e4. doi: 10.2196/jmir.1792.
    1. Kappen D, Mirza-Babaei P, Nacke L. Gamification of Older Adults' Physical Activity: An Eight-Week Study. Proceedings of the 51st Hawaii International Conference on System Sciences; HICSS'18; January 3-6, 2018; Hawaii. 2018.
    1. Plotnikoff RC, Costigan SA, Williams RL, Hutchesson MJ, Kennedy SG, Robards SL, Allen J, Collins CE, Callister R, Germov J. Effectiveness of interventions targeting physical activity, nutrition and healthy weight for university and college students: a systematic review and meta-analysis. Int J Behav Nutr Phys Act. 2015 Apr 1;12:45. doi: 10.1186/s12966-015-0203-7.
    1. Bonevski B, Randell M, Paul C, Chapman K, Twyman L, Bryant J, Brozek I, Hughes C. Reaching the hard-to-reach: a systematic review of strategies for improving health and medical research with socially disadvantaged groups. BMC Med Res Methodol. 2014 Mar 25;14:42. doi: 10.1186/1471-2288-14-42.
    1. Australian Institute of Health and Welfare . Australia's Health 2018. Australia: AIHW; 2018.
    1. Berg FM. Health risks associated with weight loss and obesity treatment programs. J Soc Issues. 2002 Dec 17;55(2):277–97. doi: 10.1111/0022-4537.00116. doi: 10.1111/0022-4537.00116.
    1. Bazelmans C, Coppieters Y, Godin I, Parent F, Berghmans L, Dramaix M, Levêque A. Is obesity associated with injuries among young people? Eur J Epidemiol. 2004;19(11):1037–42. doi: 10.1007/s10654-004-0158-5.
    1. Sallinen J, Leinonen R, Hirvensalo M, Lyyra T, Heikkinen E, Rantanen T. Perceived constraints on physical exercise among obese and non-obese older people. Prev Med. 2009 Dec;49(6):506–10. doi: 10.1016/j.ypmed.2009.10.001.
    1. Vandelanotte C, Duncan MJ, Maher CA, Schoeppe S, Rebar AL, Power DA, Short CE, Doran CM, Hayman MJ, Alley SJ. The effectiveness of a web-based computer-tailored physical activity intervention using fitbit activity trackers: randomized trial. J Med Internet Res. 2018 Dec 18;20(12):e11321. doi: 10.2196/11321.
    1. Annesi JJ, Porter KJ, Hill GM, Goldfine BD. Effects of instructional physical activity courses on overall physical activity and mood in university students. Res Q Exerc Sport. 2017 Sep;88(3):358–64. doi: 10.1080/02701367.2017.1336280.
    1. Penedo FJ, Dahn JR. Exercise and well-being: a review of mental and physical health benefits associated with physical activity. Curr Opin Psychiatry. 2005 Mar;18(2):189–93. doi: 10.1097/00001504-200503000-00013.
    1. Warburton DE, Nicol CW, Bredin SS. Health benefits of physical activity: the evidence. Can Med Assoc J. 2006 Mar 14;174(6):801–9. doi: 10.1503/cmaj.051351.
    1. Looyestyn J, Kernot J, Boshoff K, Ryan J, Edney S, Maher C. Does gamification increase engagement with online programs? A systematic review. PLoS One. 2017;12(3):e0173403. doi: 10.1371/journal.pone.0173403.
    1. Patel MS, Benjamin EJ, Volpp KG, Fox CS, Small DS, Massaro JM, Lee JJ, Hilbert V, Valentino M, Taylor DH, Manders ES, Mutalik K, Zhu J, Wang W, Murabito JM. Effect of a game-based intervention designed to enhance social incentives to increase physical activity among families: the Be Fit randomized clinical trial. JAMA Intern Med. 2017 Nov 1;177(11):1586–93. doi: 10.1001/jamainternmed.2017.3458.
    1. Sardi L, Idri A, Fernández-Alemán JL. A systematic review of gamification in e-Health. J Biomed Inform. 2017 Jul;71:31–48. doi: 10.1016/j.jbi.2017.05.011.
    1. Johnson D, Deterding S, Kuhn K, Staneva A, Stoyanov S, Hides L. Gamification for health and wellbeing: a systematic review of the literature. Internet Interv. 2016 Nov;6:89–106. doi: 10.1016/j.invent.2016.10.002.
    1. Deci EL, Koestner R, Ryan RM. A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychol Bull. 1999 Nov;125(6):627–68; discussion 692. doi: 10.1037/0033-2909.125.6.627.
    1. Suh A, Wagner C, Liu L. Enhancing user engagement through gamification. J Comput Inform Syst. 2016 Oct 3;58(3):204–13. doi: 10.1080/08874417.2016.1229143.
    1. Attig C, Franke T. I track, therefore I walk – exploring the motivational costs of wearing activity trackers in actual users. Int J Hum-Comput St. 2019 Jul;127:211–24. doi: 10.1016/j.ijhcs.2018.04.007.
    1. Kyewski E, Krämer NC. To gamify or not to gamify? An experimental field study of the influence of badges on motivation, activity, and performance in an online learning course. Comput Educ. 2018 Mar;118:25–37. doi: 10.1016/j.compedu.2017.11.006.
    1. Murray JM, French DP, Patterson CC, Kee F, Gough A, Tang J, Hunter RF. Predicting outcomes from engagement with specific components of an internet-based physical activity intervention with financial incentives: process analysis of a cluster randomized controlled trial. J Med Internet Res. 2019 Apr 19;21(4):e11394. doi: 10.2196/11394.
    1. Vandelanotte C, Spathonis KM, Eakin EG, Owen N. Website-delivered physical activity interventions a review of the literature. Am J Prev Med. 2007 Jul;33(1):54–64. doi: 10.1016/j.amepre.2007.02.041.
    1. Samdal GB, Eide GE, Barth T, Williams G, Meland E. Effective behaviour change techniques for physical activity and healthy eating in overweight and obese adults; systematic review and meta-regression analyses. Int J Behav Nutr Phys Act. 2017 Mar 28;14(1):42. doi: 10.1186/s12966-017-0494-y.
    1. Nelson LA, Coston TD, Cherrington AL, Osborn CY. Patterns of user engagement with mobile- and web-delivered self-care interventions for adults with T2DM: a review of the literature. Curr Diab Rep. 2016 Jul;16(7):66. doi: 10.1007/s11892-016-0755-1.
    1. Danaher BG, Brendryen H, Seeley JR, Tyler MS, Woolley T. From black box to toolbox: outlining device functionality, engagement activities, and the pervasive information architecture of mHealth interventions. Internet Interv. 2015 Mar 1;2(1):91–101. doi: 10.1016/j.invent.2015.01.002.
    1. Wickström G, Bendix T. The 'Hawthorne effect'--what did the original Hawthorne studies actually show? Scand J Work Environ Health. 2000 Aug;26(4):363–7. doi: 10.5271/sjweh.555.
    1. Sedgwick P, Greenwood N. Understanding the Hawthorne effect. Br Med J. 2015 Sep 4;351:h4672. doi: 10.1136/bmj.h4672.
    1. Vandelanotte C, Duncan MJ, Kolt GS, Caperchione CM, Savage TN, van Itallie A, Oldmeadow C, Alley SJ, Tague R, Maeder AJ, Rosenkranz RR, Mummery WK. More real-world trials are needed to establish if web-based physical activity interventions are effective. Br J Sports Med. 2018 Jul 3;:-. doi: 10.1136/bjsports-2018-099437. (forthcoming)
    1. Boekhout JM, Peels DA, Berendsen BA, Bolman CA, Lechner L. An eHealth intervention to promote physical activity and social network of single, chronically impaired older adults: adaptation of an existing intervention using intervention mapping. JMIR Res Protoc. 2017 Nov 23;6(11):e230. doi: 10.2196/resprot.8093.
    1. Lee D, Frey G, Cheng A, Shih PC. Puzzle Walk: A Gamified Mobile App to Increase Physical Activity in Adults with Autism Spectrum Disorder. Proceedings of the 10th International Conference on Virtual Worlds and Games for Serious Applications; VS-Games'18; September 5-7, 2018; Wurzburg, Germany. 2018.
    1. Corepal R, Best P, O'Neill R, Tully MA, Edwards M, Jago R, Miller SJ, Kee F, Hunter RF. Exploring the use of a gamified intervention for encouraging physical activity in adolescents: a qualitative longitudinal study in Northern Ireland. BMJ Open. 2018 Apr 20;8(4):e019663. doi: 10.1136/bmjopen-2017-019663.
    1. van Woudenberg TJ, Bevelander KE, Burk WJ, Smit CR, Buijs L, Buijzen M. A randomized controlled trial testing a social network intervention to promote physical activity among adolescents. BMC Public Health. 2018 Apr 23;18(1):542. doi: 10.1186/s12889-018-5451-4.

Source: PubMed

Sponsor e collaboratori

Condizioni mediche

Interventi farmacologici

3
Sottoscrivi