Measuring Engagement in eHealth and mHealth Behavior Change Interventions: Viewpoint of Methodologies

Camille E Short, Ann DeSmet, Catherine Woods, Susan L Williams, Carol Maher, Anouk Middelweerd, Andre Matthias Müller, Petra A Wark, Corneel Vandelanotte, Louise Poppe, Melanie D Hingle, Rik Crutzen, Camille E Short, Ann DeSmet, Catherine Woods, Susan L Williams, Carol Maher, Anouk Middelweerd, Andre Matthias Müller, Petra A Wark, Corneel Vandelanotte, Louise Poppe, Melanie D Hingle, Rik Crutzen

Abstract

Engagement in electronic health (eHealth) and mobile health (mHealth) behavior change interventions is thought to be important for intervention effectiveness, though what constitutes engagement and how it enhances efficacy has been somewhat unclear in the literature. Recently published detailed definitions and conceptual models of engagement have helped to build consensus around a definition of engagement and improve our understanding of how engagement may influence effectiveness. This work has helped to establish a clearer research agenda. However, to test the hypotheses generated by the conceptual modules, we need to know how to measure engagement in a valid and reliable way. The aim of this viewpoint is to provide an overview of engagement measurement options that can be employed in eHealth and mHealth behavior change intervention evaluations, discuss methodological considerations, and provide direction for future research. To identify measures, we used snowball sampling, starting from systematic reviews of engagement research as well as those utilized in studies known to the authors. A wide range of methods to measure engagement were identified, including qualitative measures, self-report questionnaires, ecological momentary assessments, system usage data, sensor data, social media data, and psychophysiological measures. Each measurement method is appraised and examples are provided to illustrate possible use in eHealth and mHealth behavior change research. Recommendations for future research are provided, based on the limitations of current methods and the heavy reliance on system usage data as the sole assessment of engagement. The validation and adoption of a wider range of engagement measurements and their thoughtful application to the study of engagement are encouraged.

Keywords: evaluation studies; health promotion; internet; outcome and process assessment (health care); telemedicine; treatment adherence and compliance.

Conflict of interest statement

Conflicts of Interest: None declared

©Camille E Short, Ann DeSmet, Catherine Woods, Susan L Williams, Carol Maher, Anouk Middelweerd, Andre Matthias Müller, Petra A Wark, Corneel Vandelanotte, Louise Poppe, Melanie D Hingle, Rik Crutzen. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 16.11.2018.

References

    1. Vandelanotte C, Müller AM, Short CE, Hingle M, Nathan N, Williams SL, Lopez ML, Parekh S, Maher CA. Past, present, and future of eHealth and mHealth research to improve physical activity and dietary behaviors. J Nutr Educ Behav. 2016 Mar;48(3):219–228.e1. doi: 10.1016/j.jneb.2015.12.006.
    1. Michie S, Yardley L, West R, Patrick K, Greaves F. Developing and evaluating digital interventions to promote behavior change in health and health care: recommendations resulting from an international workshop. J Med Internet Res. 2017 Jun 29;19(6):e232. doi: 10.2196/jmir.7126.
    1. Kelders M, Kok RN, Ossebaard HC, Van Gemert-Pijnen JE. Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res. 2012 Nov 14;14(6):e152. doi: 10.2196/jmir.2104.
    1. Perski O, Blandford A, West R, Michie S. Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis. Transl Behav Med. 2017 Dec;7(2):254–67. doi: 10.1007/s13142-016-0453-1.
    1. Yardley L, Spring BJ, Riper H, Morrison LG, Crane DH, Curtis K, Merchant GC, Naughton F, Blandford A. Understanding and promoting effective engagement with digital behavior change interventions. Am J Prev Med. 2016 Dec;51(5):833–42. doi: 10.1016/j.amepre.2016.06.015.
    1. Short CE, Rebar A, Plotnikoff RC, Vandelanotte C. Designing engaging online behaviour change interventions: a proposed model of user engagement. Health Psychol Rev. 2015;17(1):32–8.
    1. Walton H, Spector A, Tombor I, Michie S. Measures of fidelity of delivery of, and engagement with, complex, face-to-face health behaviour change interventions: a systematic review of measure quality. Br J Health Psychol. 2017 Dec;22(4):872–903. doi: 10.1111/bjhp.12260.
    1. Sieverink F, Kelders SM, van Gemert-Pijnen JE. Clarifying the concept of adherence to eHealth technology: systematic review on when usage becomes adherence. J Med Internet Res. 2017 Dec 06;19(12):e402. doi: 10.2196/jmir.8578.
    1. Ryan C, Bergin M, Wells JS. Theoretical perspectives of adherence to web-based interventions: a scoping review. Int J Behav Med. 2018 Dec;25(1):17–29. doi: 10.1007/s12529-017-9678-8.
    1. O'Brien H, Toms EG. What is user engagement? A conceptual framework for defining user engagement with technology. J Am Soc Inf Sci Technol. 2008 Apr;59(6):938–55. doi: 10.1002/asi.20801.
    1. Crutzen R, Ruiter R. Interest in behavior change interventions: a conceptual model. The European Health Psychologist. 2015;17(1):a–11.
    1. Saket B. Stasko, Beyond Usability Performance: A Review of User Experience-focused Evaluations in Visualization, in Proceedings of the Sixth Workshop on Beyond Time Errors on Novel Evaluation Methods for Visualization. BELIV '16 Proceedings of the Sixth Workshop on Beyond Time and Errors on Novel Evaluation Methods for Visualization; October 24, 2016; Baltimore, MD, USA. 2016. pp. A–142.
    1. Denisova A, Nordin AI, Cairns P. The Convergence of Player Experience Questionnaires. Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play; CHI PLAY '16; October 16 - 19, 2016; Austin, Texas, USA. 2016. pp. 33–37.
    1. Boyle EA, Connolly TM, Hainey T, Boyle JM. Engagement in digital entertainment games: a systematic review. Comput Human Behav. 2012 May;28(3):771–80. doi: 10.1016/j.chb.2011.11.020.
    1. Schiefele U. Interest, learning, and motivation. Educ Psychol. 1991 Jun;26(3-4):299–323. doi: 10.1080/00461520.1991.9653136.
    1. Schraw G, Lehman S. Situational interest:a review of the literature and directions for future research. Educ Psychol Rev. 2001;13(1):23–52. doi: 10.1023/A:1009004801455.
    1. Gerrig R, Zimbardo P. Psychology and Life. CA, United States: Pearson; 2012.
    1. James W. In: The Principles of Psychology. Miller G, editor. New York: Dover Publications; 1950.
    1. Duncan S, Barrett LF. Affect is a form of cognition: a neurobiological analysis. Cogn Emot. 2007 Sep;21(6):1184–211. doi: 10.1080/02699930701437931.
    1. Barrett LF, Russell JA. The structure of current affect: controversies and emerging consensus. Curr Dir Psychol Sci. 2016 Jun 22;8(1):10–14. doi: 10.1111/1467-8721.00003.
    1. Csikszentmihalyi M. Flow: The Psychology of Optimal Experience. New York: Harper Perennial; 1990.
    1. Brockmyer JH, Fox CM, Curtiss KA, McBroom E, Burkhart KM, Pidruzny JN. The development of the Game Engagement Questionnaire: a measure of engagement in video game-playing. J Exp Soc Psychol. 2009 Jul;45(4):624–34. doi: 10.1016/j.jesp.2009.02.016.
    1. Baños RM, Botella C, Alcañiz M, Liaño V, Guerrero B, Rey B. Immersion and emotion: their impact on the sense of presence. Cyberpsychol Behav. 2004 Dec;7(6):734–41. doi: 10.1089/cpb.2004.7.734.
    1. Lombard M, Ditton T. At the heart of it all: the concept of presence. J Comput Mediat Commun. 1997 Sep 1;3(2) doi: 10.1111/j.1083-6101.1997.tb00072.x.
    1. Dennison L, Morrison L, Conway G, Yardley L. Opportunities and challenges for smartphone applications in supporting health behavior change: qualitative study. J Med Internet Res. 2013 Apr 18;15(4):e86. doi: 10.2196/jmir.2583.
    1. Milward J, Khadjesari Z, Fincham-Campbell S, Deluca P, Watson R, Drummond C. User preferences for content, features, and style for an app to reduce harmful drinking in young adults: analysis of user feedback in app stores and focus group interviews. JMIR Mhealth Uhealth. 2016 May 24;4(2):e47. doi: 10.2196/mhealth.5242.
    1. Kernot J, Olds T, Lewis LK, Maher C. Usability testing and piloting of the Mums Step It Up program--a team-based social networking physical activity intervention for women with young children. PLoS One. 2014;9(10):e108842. doi: 10.1371/journal.pone.0108842.
    1. Short C, James EL, Rebar AL, Duncan MJ, Courneya KS, Plotnikoff RC, Crutzen R, Bidargaddi N, Vandelanotte C. Designing more engaging computer-tailored physical activity behaviour change interventions for breast cancer survivors: lessons from the iMove More for Life study. Support Care Cancer. 2017 Dec;25(11):3569–85. doi: 10.1007/s00520-017-3786-5.
    1. Kirwan M, Duncan MJ, Vandelanotte C, Mummery WK. Design, development, and formative evaluation of a smartphone application for recording and monitoring physical activity levels: the 10,000 Steps “iStepLog”. Health Educ Behav. 2013 Apr;40(2):140–51. doi: 10.1177/1090198112449460.
    1. Perski O, Blandford A, Ubhi HK, West R, Michie S. Smokers' and drinkers' choice of smartphone applications and expectations of engagement: a think aloud and interview study. BMC Med Inform Decis Mak. 2017 Dec 28;17(1):25. doi: 10.1186/s12911-017-0422-8.
    1. Bradbury K, Morton K, Band R, van Woezik A, Grist R, McManus RJ, Little P, Yardley L. Using the person-based approach to optimise a digital intervention for the management of hypertension. PLoS One. 2018;13(5):e0196868. doi: 10.1371/journal.pone.0196868.
    1. Crane D, Garnett C, Brown J, West R, Michie S. Factors influencing usability of a smartphone app to reduce excessive alcohol consumption: Think Aloud and Interview Studies. Front Public Health. 2017;5:39. doi: 10.3389/fpubh.2017.00039. doi: 10.3389/fpubh.2017.00039.
    1. Alkhaldi G, Modrow K, Hamilton F, Pal K, Ross J, Murray E. Promoting engagement with a dimgital health intervention (HeLP-Diabetes) using email and text message prompts: mixed-methods study. Interact J Med Res. 2017 Aug 22;6(2):e14. doi: 10.2196/ijmr.6952.
    1. El-Hilly A, Iqbal SS, Ahmed M, Sherwani Y, Muntasir M, Siddiqui S, Al-Fagih Z, Usmani O, Eisingerich AB. Game On? Smoking cessation through the gamification of mHealth: a longitudinal qualitative study. JMIR Serious Games. 2016 Oct 24;4(2):e18. doi: 10.2196/games.5678.
    1. Hwang M, Hong J, Hao Y, Jong J. Elders' usability, dependability, and flow experiences on embodied interactive video games. Educ Gerontol. 2011 Aug;37(8):715–31. doi: 10.1080/03601271003723636.
    1. Morrison L, Moss-Morris R, Michie S, Yardley L. Optimizing engagement with Internet-based health behaviour change interventions: comparison of self-assessment with and without tailored feedback using a mixed methods approach. Br J Health Psychol. 2014 Nov;19(4):839–55. doi: 10.1111/bjhp.12083. doi: 10.1111/bjhp.12083.
    1. Horsch C, Lancee J, Beun RJ, Neerincx MA, Brinkman WP. Adherence to technology-mediated insomnia treatment: a meta-analysis, interviews, and focus groups. J Med Internet Res. 2015 Sep 04;17(9):e214. doi: 10.2196/jmir.4115.
    1. Ritterband LM, Thorndike FP, Cox DJ, Kovatchev BP, Gonder-Frederick LA. A behavior change model for internet interventions. Ann Behav Med. 2009 Aug;38(1):18–27. doi: 10.1007/s12160-009-9133-4.
    1. Irvine L, Melson AJ, Williams B, Sniehotta FF, McKenzie A, Jones C, Crombie IK. Real time monitoring of engagement with a text message intervention to reduce binge drinking among men living in socially disadvantaged areas of Scotland. Int J Behav Med. 2017 Dec;24(5):713–721. doi: 10.1007/s12529-017-9666-z.
    1. McDonald S, Quinn F, Vieira R, O'Brien N, White M, Johnston DW, Sniehotta FF. The state of the art and future opportunities for using longitudinal n-of-1 methods in health behaviour research: a systematic literature overview. Health Psychol Rev. 2017 Dec;11(4):307–23. doi: 10.1080/17437199.2017.1316672.
    1. Kevin C, Xiaozhong L, Eileen E.A. Machine learning and rule-based automated coding of qualitative data. ASIS&T '10 Proceedings of the 73rd ASIS&T Annual Meeting on Navigating Streams in an Information Ecosystem; October 22-27, 2010; Pittsburgh, Pennsylvania. 2010. pp. 1–2.
    1. Lefebvre C, Tada Y, Hilfiker SW, Baur C. The assessment of user engagement with eHealth content: the eHealth engagement scale. J Comput Mediat Commun. 2010;15(4):666–81. doi: 10.1111/j.1083-6101.2009.01514.x.
    1. Perski O. Osf. 2017. [2018-07-13]. Study protocol: Development and psychometric evaluation of a self-report instrument to measure engagement with digital behaviour change interventions
    1. O'Brien HL, Toms EG. The development and evaluation of a survey to measure user engagement. J Am Soc Inf Sci. 2009 Oct 19;61(1):50–69. doi: 10.1002/asi.21229.
    1. Laugwitz B, Held T, Schrepp M. Construction and Evaluation of a User Experience Questionnaire. Symposium of the Austrian HCI and Usability Engineering Group USAB 2008: HCI and Usability for Education and Work; November 20-21, 2008; Graz, Austria. 2008. pp. 63–76.
    1. Jackson S, Marsh HW. Development and validation of a scale to measure optimal experience: the flow state scale. J Sport Exerc Psychol. 1996 Mar;18(1):17–35. doi: 10.1123/jsep.18.1.17.
    1. Direito A, Jiang Y, Whittaker R, Maddison R. Apps for IMproving FITness and increasing physical activity among young people: the AIMFIT pragmatic randomized controlled trial. J Med Internet Res. 2015 Aug 27;17(8):e210. doi: 10.2196/jmir.4568.
    1. Boase J, Ling R. Measuring mobile phone use: self-report versus log data. J Comput-Mediat Comm. 2013 Jun 10;18(4):508–19. doi: 10.1111/jcc4.12021.
    1. Scharkow M. The accuracy of self-reported internet use—a validation study using client log data. Commun Methods Meas. 2016 Mar 24;10(1):13–27. doi: 10.1080/19312458.2015.1118446.
    1. Sigerson L, Cheng C. Scales for measuring user engagement with social network sites: A systematic review of psychometric properties. Comput Human Behav. 2018 Jun;83:87–105. doi: 10.1016/j.chb.2018.01.023.
    1. MacKinnon D, Fairchild AJ, Fritz MS. Mediation analysis. Annu Rev Psychol. 2007;58:593–614. doi: 10.1146/annurev.psych.58.110405.085542.
    1. Murray J, Brennan SF, French DP, Patterson CC, Kee F, Hunter RF. Mediators of behavior change maintenance in physical activity interventions for young and middle-aged adults: a systematic review. Ann Behav Med. 2018 May 18;52(6):513–29. doi: 10.1093/abm/kay012.
    1. Rhodes R, Pfaeffli LA. Mediators of physical activity behaviour change among adult non-clinical populations: a review update. Int J Behav Nutr Phys Act. 2010 May 11;7:37. doi: 10.1186/1479-5868-7-37.
    1. Dewar DL, Lubans DR, Morgan PJ, Plotnikoff RC. Development and evaluation of social cognitive measures related to adolescent physical activity. J Phys Act Health. 2013 May;10(4):544–55. doi: 10.1186/1479-5868-9-36.
    1. Rhodes R, Hunt Matheson D, Mark R. Evaluation of social cognitive scaling response options in the physical activity domain. Meas Phys Educ Exerc Sci. 2010 Jul 28;14(3):137–50. doi: 10.1080/1091367X.2010.495539.
    1. Hall E, Chai W, Koszewski W, Albrecht J. Development and validation of a social cognitive theory-based survey for elementary nutrition education program. Int J Behav Nutr Phys Act. 2015 Apr 09;12:47. doi: 10.1186/s12966-015-0206-4.
    1. Francis J, Johnston M, Eccles M, Walker A, Grimshaw JM, Foy R, Kaner EF, Smith L, Bonetti D. Constructing questionnaires based on the theory of planned behaviour: a manual for Health Services Researchers. Newcastle upon Tyne, UK: Centre for Health Service Research; 2004. .
    1. Bandura A. Guide for constructing self-efficacy scales. J Phys Act Res. 2006:307–37. doi: 10.12691/jpar-2-1-2.
    1. Crutzen R, Peters GJ. Scale quality: alpha is an inadequate estimate and factor-analytic evidence is needed first of all. Health Psychol Rev. 2017 Dec;11(3):242–47. doi: 10.1080/17437199.2015.1124240.
    1. Doherty K, Doherty G. The construal of experience in HCI: uUnderstanding self-reports. Int J Hum Comput Stud. 2018 Feb;110:63–74. doi: 10.1016/j.ijhcs.2017.10.006.
    1. Reis HT. Why Researchers Should Think “Real World”: A Conceptual Rationale. In: Mehl MR, Conner TS, editors. Handbook of Research Methods for Studying Daily Life. New York: The Guilford Press; 2013. pp. 3–22.
    1. Hernandez J, McDuff D, Infante C, Maes P, Quigley K, Picard R. Wearable ESM: differences in the experience sampling method across wearable devices. MobileHCI '16 Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services; September 6 to 9, 2016; Florance, Italy. 2016. pp. 195–205.
    1. Dunton GF, Liao Y, Intille SS, Spruijt-Metz D, Pentz M. Investigating children's physical activity and sedentary behavior using ecological momentary assessment with mobile phones. Obesity (Silver Spring) 2011 Jun;19(6):1205–12. doi: 10.1038/oby.2010.302. doi: 10.1038/oby.2010.302.
    1. Fanning J, Mackenzie M, Roberts S, Crato I, Ehlers D, McAuley E. Physical activity, mind wandering, affect, and sleep: an ecological momentary assessment. JMIR Mhealth Uhealth. 2016 Aug 31;4(3):e104. doi: 10.2196/mhealth.5855.
    1. Chung J, Gardner HJ. Temporal presence variation in immersive computer games. Int J Hum Comput Stud. 2012 Aug;28(8):511–29. doi: 10.1080/10447318.2011.627298.
    1. Wen C, Schneider S, Stone AA, Spruijt-Metz D. Compliance with mobile ecological momentary assessment protocols in children and adolescents: a systematic review and meta-analysis. J Med Internet Res. 2017 Dec 26;19(4):e132. doi: 10.2196/jmir.6641.
    1. Cain AE, Depp CA, Jeste DV. Ecological momentary assessment in aging research: a critical review. J Psychiatr Res. 2009 Jul;43(11):987–96. doi: 10.1016/j.jpsychires.2009.01.014.
    1. Modecki KL, Mazza GL. Are we making the most of ecological momentary assessment data? A comment on Richardson, Fuller-Tyszkiewicz, O'Donnell, Ling, & Staiger, 2017. Health Psychol Rev. 2017 Dec;11(3):295–97. doi: 10.1080/17437199.2017.1347513.
    1. Richardson B, Fuller-Tyszkiewicz M, O'Donnell R, Ling M, Staiger PK. Regression tree analysis of ecological momentary assessment data. Health Psychol Rev. 2017 Dec;11(3):235–41. doi: 10.1080/17437199.2017.1343677.
    1. Ginexi EM, Riley W, Atienza AA, Mabry PL. The promise of intensive longitudinal data capture for behavioral health research. Nicotine Tob Res. 2014 May;16 Suppl 2:S73–5. doi: 10.1093/ntr/ntt273.
    1. Hamaker E, Wichers M. No Time Like the Present. Curr Dir Psychol Sci. 2017 Feb 08;26(1):10–15. doi: 10.1177/0963721416666518.
    1. Burke L, Shiffman S, Music E, Styn MA, Kriska A, Smailagic A, Siewiorek D, Ewing LJ, Chasens E, French B, Mancino J, Mendez D, Strollo P, Rathbun SL. Ecological momentary assessment in behavioral research: addressing technological and human participant challenges. J Med Internet Res. 2017 Dec 15;19(3):e77. doi: 10.2196/jmir.7138.
    1. Kelders S, Van Gemert-Pijnen JE, Werkman A, Nijland N, Seydel ER. Effectiveness of a Web-based intervention aimed at healthy dietary and physical activity behavior: a randomized controlled trial about users and usage. J Med Internet Res. 2011 Apr 14;13(2):e32. doi: 10.2196/jmir.1624.
    1. Danaher BG, Boles SM, Akers L, Gordon JS, Severson HH. Defining participant exposure measures in Web-based health behavior change programs. J Med Internet Res. 2006 Aug 30;8(3):e15. doi: 10.2196/jmir.8.3.e15.
    1. Graham ML, Strawderman MS, Demment M, Olson CM. Does usage of an eHealth intervention reduce the risk of excessive gestational weight gain? Secondary analysis from a randomized controlled trial. J Med Internet Res. 2017 Dec 09;19(1):e6. doi: 10.2196/jmir.6644.
    1. Mattila E, Lappalainen R, Välkkynen P, Sairanen E, Lappalainen P, Karhunen L, Peuhkuri K, Korpela R, Kolehmainen M, Ermes M. Usage and dose response of a mobile acceptance and commitment therapy app: secondary analysis of the intervention arm of a randomized controlled trial. JMIR Mhealth Uhealth. 2016 Jul 28;4(3):e90. doi: 10.2196/mhealth.5241.
    1. Taki S, Lymer S, Russell CG, Campbell K, Laws R, Ong K, Elliott R, Denney-Wilson E. Assessing user engagement of an mHealth intervention: development and implementation of the growing healthy app engagement index. JMIR Mhealth Uhealth. 2017 Jun 29;5(6):e89. doi: 10.2196/mhealth.7236.
    1. McCallum C, Rooksby J, Gray CM. Evaluating the impact of physical activity apps and wearables: interdisciplinary review. JMIR Mhealth Uhealth. 2018 Mar 23;6(3):e58. doi: 10.2196/mhealth.9054.
    1. Baltierra NB, Muessig KE, Pike EC, LeGrand S, Bull SS, Hightow-Weidman LB. More than just tracking time: Complex measures of user engagement with an internet-based health promotion intervention. J Biomed Inform. 2016 Feb;59:299–307. doi: 10.1016/j.jbi.2015.12.015.
    1. Barisic A, Leatherdale ST, Kreiger N. Importance of frequency, intensity, time and type (FITT) in physical activity assessment for epidemiological research. Can J Public Health. 2011;102(3):174–5.
    1. Funk KL, Stevens VJ, Appel LJ, Bauck A, Brantley PJ, Champagne CM, Coughlin J, Dalcin AT, Harvey-Berino J, Hollis JF, Jerome GJ, Kennedy BM, Lien LF, Myers VH, Samuel-Hodge C, Svetkey LP, Vollmer WM. Associations of internet website use with weight change in a long-term weight loss maintenance program. J Med Internet Res. 2010 Jul 27;12(3):e29. doi: 10.2196/jmir.1504.
    1. Arden-Close EJ, Smith E, Bradbury K, Morrison L, Dennison L, Michaelides D, Yardley L. A visualization tool to analyse usage of web-based interventions: the example of positive online weight reduction (POWeR) JMIR Hum Factors. 2015 May 19;2(1):e8. doi: 10.2196/humanfactors.4310.
    1. Manwaring JL, Bryson SW, Goldschmidt AB, Winzelberg AJ, Luce KH, Cunning D, Wilfley DE, Taylor CB. Do adherence variables predict outcome in an online program for the prevention of eating disorders? J Consult Clin Psychol. 2008 Apr;76(2):341–6. doi: 10.1037/0022-006X.76.2.341.
    1. van Mierlo Trevor. The 1% rule in four digital health social networks: an observational study. J Med Internet Res. 2014 Feb 04;16(2):e33. doi: 10.2196/jmir.2966.
    1. Forbes C, Blanchard CM, Mummery WK, Courneya KS. Feasibility and preliminary efficacy of an online intervention to increase physical activity in Nova Scotian cancer survivors: a randomized controlled trial. JMIR Cancer. 2015 Nov 23;1(2):e12. doi: 10.2196/cancer.4586.
    1. Short CE, Rebar A, James EL, Duncan MJ, Courneya KS, Plotnikoff RC, Crutzen R, Vandelanotte C. How do different delivery schedules of tailored web-based physical activity advice for breast cancer survivors influence intervention use and efficacy? J Cancer Surviv. 2017 Feb;11(1):80–91. doi: 10.1007/s11764-016-0565-0.
    1. Hales SB, Davidson C, Turner-McGrievy GM. Varying social media post types differentially impacts engagement in a behavioral weight loss intervention. Transl Behav Med. 2014 Dec;4(4):355–62. doi: 10.1007/s13142-014-0274-z.
    1. Couper MP, Alexander GL, Zhang N, Little RJ, Maddy N, Nowak MA, McClure JB, Calvi JJ, Rolnick SJ, Stopponi MA, Cole JC. Engagement and retention: measuring breadth and depth of participant use of an online intervention. J Med Internet Res. 2010 Nov 18;12(4):e52. doi: 10.2196/jmir.1430.
    1. Mohr DC, Duffecy J, Ho J, Kwasny M, Cai X, Burns MN, Begale M. A randomized controlled trial evaluating a manualized TeleCoaching protocol for improving adherence to a web-based intervention for the treatment of depression. PLoS One. 2013;8(8):e70086. doi: 10.1371/journal.pone.0070086.
    1. Cussler EC, Teixeira PJ, Going SB, Houtkooper LB, Metcalfe LL, Blew RM, Ricketts JR, Lohman J, Stanford VA, Lohman TG. Maintenance of weight loss in overweight middle-aged women through the Internet. Obesity (Silver Spring) 2008 May;16(5):1052–60. doi: 10.1038/oby.2008.19. doi: 10.1038/oby.2008.19.
    1. Glasgow RE, Christiansen SM, Kurz D, King DK, Woolley T, Faber AJ, Estabrooks PA, Strycker L, Toobert D, Dickman J. Engagement in a diabetes self-management website: usage patterns and generalizability of program use. J Med Internet Res. 2011 Jan 25;13(1):e9. doi: 10.2196/jmir.1391.
    1. Davies C, Corry K, Van Itallie Anetta, Vandelanotte C, Caperchione C, Mummery WK. Prospective associations between intervention components and website engagement in a publicly available physical activity website: the case of 10,000 Steps Australia. J Med Internet Res. 2012 Jan 11;14(1):e4. doi: 10.2196/jmir.1792.
    1. Kim JY, Wineinger NE, Taitel M, Radin JM, Akinbosoye O, Jiang J, Nikzad N, Orr G, Topol E, Steinhubl S. Self-Monitoring Utilization Patterns Among Individuals in an Incentivized Program for Healthy Behaviors. J Med Internet Res. 2016 Dec 17;18(11):e292. doi: 10.2196/jmir.6371.
    1. Crutzen R, Roosjen Johanna L, Poelman Jos. Using Google Analytics as a process evaluation method for Internet-delivered interventions: an example on sexual health. Health Promot Int. 2013 Mar;28(1):36–42. doi: 10.1093/heapro/das008.
    1. Scherer EA, Ben-Zeev D, Li Z, Kane JM. Analyzing mHealth Engagement: joint models for intensively collected user engagement data. JMIR Mhealth Uhealth. 2017 Jan 12;5(1):e1. doi: 10.2196/mhealth.6474.
    1. Fan W, Bifet A. Mining big data. SIGKDD Explor Newsl. 2013 Apr 30;14(2):1–5. doi: 10.1145/2481244.2481246.
    1. Cugelman B, Thelwall M, Dawes P. Online interventions for social marketing health behavior change campaigns: a meta-analysis of psychological architectures and adherence factors. J Med Internet Res. 2011 Feb 14;13(1):e17. doi: 10.2196/jmir.1367.
    1. Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res. 2011 Aug 05;13(3):e52. doi: 10.2196/jmir.1772.
    1. Kidwell K, Hyde LW. Adaptive interventions and SMART designs: application to child behavior research in a community setting. Am J Eval. 2016 Sep;37(3):344–63. doi: 10.1177/1098214015617013.
    1. Collins LM, Murphy SA, Bierman KL. A conceptual framework for adaptive preventive interventions. Prev Sci. 2004 Sep;5(3):185–96. doi: 10.1023/B:PREV.0000037641.26017.00.
    1. Althoff T, Sosič R, Hicks JL, King AC, Delp SL, Leskovec J. Large-scale physical activity data reveal worldwide activity inequality. Nature. 2017 Dec 20;547(7663):336–39. doi: 10.1038/nature23018.
    1. Mohr D, Schueller SM, Riley WT, Brown CH, Cuijpers P, Duan N, Kwasny MJ, Stiles-Shields C, Cheung K. Trials of intervention principles: evaluation methods for evolving behavioral intervention technologies. J Med Internet Res. 2015 Jul 08;17(7):e166. doi: 10.2196/jmir.4391.
    1. Jacobs M, Graham Al. Iterative development and evaluation methods of mHealth behavior change interventions. Curr Opin Psychol. 2016 Jun;9:33–7. doi: 10.1016/j.copsyc.2015.09.001.
    1. Vandelanotte C, Duncan MJ, Kolt GS, Caperchione CM, Savage TN, Van Itallie A, Oldmeadow C, Alley SJ, Tague R, Maeder AJ, Rosenkranz RR, Mummery WK. More real-world trials are needed to establish if web-based physical activity interventions are effective. Br J Sports Med. 2018 Jul 03; doi: 10.1136/bjsports-2018-099437. Epub ahead of print.
    1. Collins L, Murphy SA, Strecher V. The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART): new methods for more potent eHealth interventions. Am J Prev Med. 2007 May;32(5 Suppl):S112–8. doi: 10.1016/j.amepre.2007.01.022.
    1. Shaw R, Steinberg DM, Zullig LL, Bosworth HB, Johnson CM, Davis LL. mHealth interventions for weight loss: a guide for achieving treatment fidelity. J Am Med Inform Assoc. 2014;21(6):959–63. doi: 10.1136/amiajnl-2013-002610.
    1. Cornet VP, Holden RJ. Systematic review of smartphone-based passive sensing for health and wellbeing. J Biomed Inform. 2018 Jan;77:120–32. doi: 10.1016/j.jbi.2017.12.008.
    1. Barkhuus L, Dey A. Location-Based Services for Mobile Telephony: a Study of Users' Privacy Concerns (2003). 9TH IFIP TC13 International Conference On Human-Computer Interaction, Interact 2003; 1st-5th September, 2003; Zürich, Switzerland. 2003.
    1. Case MA, Burwick HA, Volpp KG, Patel MS. Accuracy of smartphone applications and wearable devices for tracking physical activity data. J Am Med Assoc. 2015 Feb 10;313(6):625–6. doi: 10.1001/jama.2014.17841.
    1. Gordon BA, Bruce L, Benson AC. Physical activity intensity can be accurately monitored by smartphone global positioning system 'app'. Eur J Sport Sci. 2016 Aug;16(5):624–31. doi: 10.1080/17461391.2015.1105299.
    1. Maher C, Lewis LK, Ferrar K, Marshall S, De Bourdeaudhuij I, Vandelanotte C. Are health behavior change interventions that use online social networks effective? A systematic review. J Med Internet Res. 2014 Feb 14;16(2):e40. doi: 10.2196/jmir.2952.
    1. Hale TM, Pathipati AS, Zan S, Jethwani K. Representation of health conditions on Facebook: content analysis and evaluation of user engagement. J Med Internet Res. 2014 Aug 04;16(8):e182. doi: 10.2196/jmir.3275.
    1. D'heer E, Verdegem P. What social media data mean for audience studies: a multidimensional investigation of Twitter use during a current affairs TV programme. Inform Comm Soc. 2014 Sep 22;18(2):221–34. doi: 10.1080/1369118X.2014.952318.
    1. Ryan J, Edney S, Maher C. Engagement, compliance and retention with a gamified online social networking physical activity intervention. Transl Behav Med. 2017 Dec;7(4):702–8. doi: 10.1007/s13142-017-0499-8.
    1. Bouman M, Drossaert CH, Pieterse ME. Mark My Words: the design of an innovative methodology to detect and analyze interpersonal health conversations in web and social media. J Technol Hum Serv. 2012 Jul;30(3-4):312–26. doi: 10.1080/15228835.2012.743394.
    1. Delahaye Paine K. Measure What Matters: Online Tools for Understanding Customers, Social Media, Engagement, and Key Relationships. United Kingdom: John Wiley and Sons Ltd, United Kingdom; 2018.
    1. Murdough C. Social Media Measurement. J Interact Advert. 2009 Sep;10(1):94–99. doi: 10.1080/15252019.2009.10722165.
    1. Peters GY, Crutzen R. Pragmatic nihilism: how a Theory of Nothing can help health psychology progress. Health Psychol Rev. 2017 Dec;11(2):103–21. doi: 10.1080/17437199.2017.1284015.
    1. Dirican A, Göktürk M. Psychophysiological measures of human cognitive states applied in human computer interaction. Procedia Comp Sci. 2011;3:1361–67. doi: 10.1016/j.procs.2011.01.016.
    1. Cowley B, Filetti M, Lukander K, Torniainen J, Henelius A, Ahonen L, Barral O, Kosunen I, Valtonen T, Huotilainen M, Ravaja N, Jacucci G. The Psychophysiology Primer: a guide to methods and a broad review with a focus on human-computer interaction. Found Trends Hum Comp Interact. 2015;9(3-4):151–308.
    1. Ganglbauer E, Schrammel J, Deutsch S, Tscheligi M. Applying psychophysiological methods for measuring user experience: possibilities, challenges and feasibility. User Experience Evaluation Methods in Product Development (UXEM'09) in conjunction with Interact'09; August 24-28, 2009; Sweden. 2009.
    1. Harmat L, de Manzano Örjan, Theorell T, Högman Lennart, Fischer H, Ullén Fredrik. Physiological correlates of the flow experience during computer game playing. Int J Psychophysiol. 2015 Jul;97(1):1–7. doi: 10.1016/j.ijpsycho.2015.05.001.
    1. Burns C, Fairclough SH. Use of auditory event-related potentials to measure immersion during a computer game. Int J Hum Comput Stud. 2015 Jan;73(Supplement C):107–14. doi: 10.1016/j.ijhcs.2014.09.002.
    1. Martey R, Kenski K, Folkestad J, Feldman L, Gordis E, Shaw A, Stromer-Galley J, Clegg B, Zhang H, Kaufman N, Rabkin AN, Shaikh S, Strzalkowski T. Measuring Game Engagement. Simul Gaming. 2014 Nov 04;45(4-5):528–47. doi: 10.1177/1046878114553575.
    1. Dhamija S, Boult TE. Automated mood-aware engagement prediction. Seventh International Conference on Affective Computing Intelligent Interaction (ACII); 23-26 October, 2017; San Antonio, TX, USA. 2017.
    1. Huynh S, Kim S, Ko J, Balan RK, Lee Y. EngageMon. Proc ACM Interact Mob Wearable Ubiquitous Technol. 2018 Mar 26;2(1):1–27. doi: 10.1145/3191745.
    1. Bevilacqua F, Engström H, Backlund P. Changes in heart rate and facial actions during a gaming session with provoked boredom and stress. Entertainment Computing. 2018 Jan;24:10–20. doi: 10.1016/j.entcom.2017.10.004.
    1. Reinecke K, Cordes M, Lerch C, Koutsandréou F, Schubert M, Weiss M, Baumeister J. From lab to field conditions: a pilot study on EEG methodology in applied sports sciences. Appl Psychophysiol Biofeedback. 2011 Dec;36(4):265–71. doi: 10.1007/s10484-011-9166-x.
    1. Kayser J, Tenke CE. Issues and considerations for using the scalp surface Laplacian in EEG/ERP research: A tutorial review. Int J Psychophysiol. 2015 Sep;97(3):189–209. doi: 10.1016/j.ijpsycho.2015.04.012.
    1. Delorme A, Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods. 2004 Mar 15;134(1):9–21. doi: 10.1016/j.jneumeth.2003.10.009.
    1. Luck SJ. An Introduction to the Event-Related Potential Technique. Cambridge, MA: MIT Press; 2005.
    1. Leiker A, Miller M, Brewer L, Nelson M, Siow M, Lohse K. The relationship between engagement and neurophysiological measures of attention in motion-controlled video games: a randomized controlled trial. JMIR Serious Games. 2016 Apr 21;4(1):e4. doi: 10.2196/games.5460.
    1. Dyke FB, Leiker AM, Grand KF, Godwin MM, Thompson AG, Rietschel JC, McDonald CG, Miller MW. The efficacy of auditory probes in indexing cognitive workload is dependent on stimulus complexity. Int J Psychophysiol. 2015 Jan;95(1):56–62. doi: 10.1016/j.ijpsycho.2014.12.008.
    1. Takeda Y, Okuma T, Kimura M, Kurata T, Takenaka T, Iwaki S. Electrophysiological measurement of interest during walking in a simulated environment. Int J Psychophysiol. 2014 Sep;93(3):363–70. doi: 10.1016/j.ijpsycho.2014.05.012.
    1. Rayner K. Eye movements in reading and information processing: 20 years of research. Psychol Bull. 1998 Nov;124(3):372–422.
    1. Hermans D, Vansteenwegen D, Eelen P. Eye movement registration as a continuous index of attention deployment: data from a group of spider anxious students. Cognition & Emotion. 1999 Jul;13(4):419–34. doi: 10.1080/026999399379249.
    1. Jennett C, Cox Al, Cairns P, Dhoparee S, Epps A, Tijs T, Walton A. Measuring and defining the experience of immersion in games. Int J Hum Comput Stud. 2008 Sep;66(9):641–61. doi: 10.1016/j.ijhcs.2008.04.004.
    1. Alley S, Jennings C, Persaud N, Plotnikoff RC, Horsley M, Vandelanotte C. Do personally tailored videos in a web-based physical activity intervention lead to higher attention and recall? - An eye-tracking study. Front Public Health. 2014;2:13. doi: 10.3389/fpubh.2014.00013. doi: 10.3389/fpubh.2014.00013.
    1. Crutzen R, Cyr D, Larios H, Ruiter RA, de Vries NK. Social presence and use of internet-delivered interventions: a multi-method approach. PLoS One. 2013;8(2):e57067. doi: 10.1371/journal.pone.0057067.
    1. Nacke L, Grimshaw MN, Lindley CA. More than a feeling: measurement of sonic user experience and psychophysiology in a first-person shooter game. Interact Comput. 2010 Sep;22(5):336–43. doi: 10.1016/j.intcom.2010.04.005.

Source: PubMed

Szponzorok és közreműködők

Egészségi állapot

Kábítószer-beavatkozások

3
Iratkozz fel