Use of In-Game Rewards to Motivate Daily Self-Report Compliance: Randomized Controlled Trial

Sara Taylor, Craig Ferguson, Fengjiao Peng, Magdalena Schoeneich, Rosalind W Picard, Sara Taylor, Craig Ferguson, Fengjiao Peng, Magdalena Schoeneich, Rosalind W Picard

Abstract

Background: Encouraging individuals to report daily information such as unpleasant disease symptoms, daily activities and behaviors, or aspects of their physical and emotional state is difficult but necessary for many studies and clinical trials that rely on patient-reported data as primary outcomes. Use of paper diaries is the traditional method of completing daily diaries, but digital surveys are becoming the new standard because of their increased compliance; however, they still fall short of desired compliance levels.

Objective: Mobile games using in-game rewards offer the opportunity to increase compliance above the rates of digital diaries and paper diaries. We conducted a 5-week randomized control trial to compare the completion rates of a daily diary across 3 conditions: a paper-based participant-reported outcome diary (Paper PRO), an electronic-based participant-reported outcome diary (ePRO), and a novel ePRO diary with in-game rewards (Game-Motivated ePRO).

Methods: We developed a novel mobile game that is a combination of the idle and pet collection genres to reward individuals who complete a daily diary with an in-game reward. Overall, 197 individuals aged 6 to 24 years (male: 100 and female: 97) were enrolled in a 5-week study after being randomized into 1 of the 3 methods of daily diary completion. Moreover, 157 participants (male: 84 and female: 69) completed at least one diary and were subsequently included in analysis of compliance rates.

Results: We observed a significant difference (F2,124=6.341; P=.002) in compliance to filling out daily diaries, with the Game-Motivated ePRO group having the highest compliance (mean completion 86.4%, SD 19.6%), followed by the ePRO group (mean completion 77.7%, SD 24.1%), and finally, the Paper PRO group (mean completion 70.6%, SD 23.4%). The Game-Motivated ePRO (P=.002) significantly improved compliance rates above the Paper PRO. In addition, the Game-Motivated ePRO resulted in higher compliance rates than the rates of ePRO alone (P=.09). Equally important, even though we observed significant differences in completion of daily diaries between groups, we did not observe any statistically significant differences in association between the responses to a daily mood question and study group, the average diary completion time (P=.52), or the System Usability Scale score (P=.88).

Conclusions: The Game-Motivated ePRO system encouraged individuals to complete the daily diaries above the compliance rates of the Paper PRO and ePRO without altering the participants' responses.

Trial registration: ClinicalTrials.gov NCT03738254; https://ichgcp.net/clinical-trials-registry/NCT03738254 (Archived by WebCite at http://www.webcitation.org/74T1p8u52).

Keywords: protocol compliance; recreational games; self-reports.

Conflict of interest statement

Conflicts of Interest: None declared.

©Sara Taylor, Craig Ferguson, Fengjiao Peng, Magdalena Schoeneich, Rosalind W Picard. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 03.01.2019.

Figures

Figure 1
Figure 1
Screenshots of The Guardians while the player completes a daily diary. Players collect pets by filling out a daily diary. The pets are needed to complete quests and gather Light to awaken the guardian.
Figure 2
Figure 2
Screenshots of leveling menu and housing placement puzzle.
Figure 3
Figure 3
Flow of participants through the study. ePRO: electronic-based participant-reported outcome; PRO: participant-reported outcome.
Figure 4
Figure 4
Distribution of participants who filled out the prestudy survey by age and daily diary condition. ePRO: electronic-based participant-reported outcome; PRO: participant-reported outcome.
Figure 5
Figure 5
Game-motivated electronic participant-reported outcome participants’ engagement with various components of the game and the corresponding diary completion.
Figure 6
Figure 6
Average daily diary completion rate for each study arm. The error bars show the 95% CI and P values are from the Tukey posthoc test and are adjusted for multiple comparisons. ePRO: electronic-based participant-reported outcome; PRO: participant-reported outcome.
Figure 7
Figure 7
Kaplan-Meier curves of survival (ie, missing 3 or fewer diaries) for each study group. ePRO: electronic-based participant-reported outcome; PRO: participant-reported outcome.
Figure 8
Figure 8
Percentage of participants who completed at least 90% of the daily diaries (ie, 32 out of 35 diaries) in each study arm. The 95% CI and P values are shown. ePRO: electronic-based participant-reported outcome; PRO: participant-reported outcomes.

References

    1. Stone AA, Shiffman S, Schwartz JE, Broderick JE, Hufford MR. Patient compliance with paper and electronic diaries. Control Clin Trials. 2003 Apr;24(2):182–99. doi: 10.1016/S0197-2456(02)00320-3.
    1. Jamison RN, Raymond SA, Levine JG, Slawsby EA, Nedeljkovic SS, Katz NP. Electronic diaries for monitoring chronic pain: 1-year validation study. Pain. 2001;91(3):277–85. doi: 10.1016/S0304-3959(00)00450-4.
    1. Palermo TM, Valenzuela D, Stork PP. A randomized trial of electronic versus paper pain diaries in children: impact on compliance, accuracy, and acceptability. Pain. 2004;107(3):213–9. doi: 10.1016/j.pain.2003.10.005.
    1. Pew Research Center. 2018. Feb 05, [2018-12-10]. Mobile Fact Sheet
    1. Harris C, Daniels K, Briner R. A daily diary study of goals and affective well-being at work. J Occup Organ Psychol. 2003;76:401–10. doi: 10.1348/096317903769647256.
    1. Totterdell P, Kellett S, Teuchmann K, Briner R. Evidence of mood linkage in work groups. Journal of Personality and Social Psychology. 1998;74(6):1504–1515. doi: 10.1037//0022-3514.74.6.1504.
    1. Csikszentmihalyi M, Larson R, Prescott S. The ecology of adolescent activity and experience. J Youth Adolesc. 1977 Sep;6(3):281–94. doi: 10.1007/BF02138940.
    1. Hernandez J, McDuff D, Infante C, Maes P, Quigley K, Picard R. Wearable ESM: Differences in the experience sampling method across wearable devices. Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services; Mobile HCI'16; September 06-09, 2016; Florence, Italy. 2016. Sep, pp. 195–205.
    1. Intille S, Haynes C, Maniar D, Ponnada A, Manjourides J. μEMA: Microinteraction-based Ecological Momentary Assessment (EMA) Using a Smartwatch. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing; UbiComp'16; September 12-16, 2016; Heidelberg, Germany. 2016. Sep, pp. 1124–8.
    1. Statista. [2018-10-03]. Mobile Phone Gaming Penetration in The United States from 2011 to 2020
    1. Statista. [2018-10-03]. Share of Teenagers Who Play Games on Their Mobile Phone or Tablet in The United States from Spring 2013 to Fall 2016
    1. Statista. [2018-10-03]. Frequency with Which Children in The United States Play Their Favorite Gaming Apps in 2015
    1. Kosecki D. Fitbit. [2018-03-27]. Presenting the Official List of Fitbit Badges - How Many Do You Have?
    1. Apple. [2018-03-27]. Earn Activity achievements using your Apple Watch .
    1. Nike. [2018-03-27]. Nike+ Running Club App: What's New .
    1. Khan Academy. [2018-03-27]. Badges .
    1. Wikipedia. [2018-03-27]. Wikipedia: Service awards .
    1. Stack Overflow. [2018-03-27]. Badges .
    1. Lyft Blog. [2018-03-27]. Lyft Badge Glossary
    1. Hamari J, Koivisto J, Sarsa H. Does Gamification Work? -- A Literature Review of Empirical Studies on Gamification. Proceedings of the 2014 47th Hawaii International Conference on System Sciences; HICSS'14; January 06-09, 2014; Waikoloa, HI, USA. 2014. Mar 10, pp. 3025–34.
    1. Hanus MD, Fox J. Assessing the effects of gamification in the classroom: a longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Comput Educ. 2015 Jan;80:152–61. doi: 10.1016/j.compedu.2014.08.019.
    1. Anderson A, Huttenlocher D, Kleinberg J, Leskovec J. Steering user behavior with badges. Proceedings of the 22nd international conference on World Wide Web; WWW'13; May 13-17, 2013; Rio de Janeiro, Brazil. ACM; 2013. May, pp. 95–105.
    1. Mejia J. Grand Valley State University. 2013. Apr, [2018-12-10]. Impact of Gamification and Shared Situated Displays on Smartphone Application Engagement .
    1. Farzan R, DiMicco J, Millen D, Dugan C, Geyer W, Brownholtz E. Results from deploying a participation incentive mechanism within the enterprise. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; CHI'08; April 05-10, 2008; Florence, Italy. 2018. Apr, pp. 563–72.
    1. Koivisto J, Hamari J. Demographic differences in perceived benefits from gamification. Comput Human Behav. 2014 Jun;35:179–88. doi: 10.1016/j.chb.2014.03.007.
    1. Malone T. Heuristics for designing enjoyable user interfaces: Lessons from computer games. Proceedings of the 1982 Conference on Human Factors in Computing Systems; CHI'82; March 15-17, 1982; Gaithersburg, Maryland, USA. 1982. pp. 63–8.
    1. Lu AS, Baranowski J, Islam N, Baranowski T. How to engage children in self-administered dietary assessment programmes. J Hum Nutr Diet. 2014 Jan;27(Suppl 1):5–9. doi: 10.1111/j.1365-277X.2012.01258.x.
    1. Alharthi S, Olaa A, Toups Z, Tanenbaum J, Hammer J. Playing to Wait: A Taxonomy of Idle Games. Proceedings of the 2018 Conference on Human Factors in Computing Systems; CHI'18; April 21-26, 2018; Montreal QC, Canada. 2018. Apr, pp. 1–15.
    1. Sano A, Taylor S, McHill AW, Phillips AJ, Barger LK, Klerman E, Picard R. Identifying objective physiological markers and modifiable behaviors for self-reported stress and mental health status using wearable sensors and mobile phones: observational study. J Med Internet Res. 2018 Jun 08;20(6):e210. doi: 10.2196/jmir.9410.
    1. Suhara Y, Yinzhan X, Pentland A. DeepMood: Forecasting depressed mood based on self-reported histories via recurrent neural networks. Proceedings of the 26th International Conference on World Wide Web; WWW'17; April 03-07, 2017; Perth, Australia. 2017. Apr, pp. 715–24.
    1. Canzian L, Musolesi M. Trajectories of depression: unobtrusive monitoring of depressive states by means of smartphone mobility traces analysis. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing; UbiComp'15; September 07-11, 2015; Osaka, Japan. 2015. Sep, pp. 1293–304.
    1. Influence Central. [2018-10-02]. Kids & Tech: The Evolution of Today's Digital Natives
    1. Ogletree SM, Drake R. College students’ video game participation and perceptions: gender differences and implications. Sex Roles. 2007 Mar 21;56(7-8):537–42. doi: 10.1007/s11199-007-9193-5.
    1. Wright J, Huston A, Vandewater E, Bickham D, Scantlin R, Kotler J, Caplovitz A, Lee J, Hofferth S, Finkelstein J. American children's use of electronic media in 1997: a national survey. J Appl Dev Psychol. 2001 Jan;22(1):31–47. doi: 10.1016/S0193-3973(00)00064-2.
    1. Kaplan EL, Meier P. Nonparametric estimation from incomplete observations. J Am Stat Assoc. 1958 Jun;53(282):457. doi: 10.2307/2281868.
    1. Therneau T. The Comprehensive R Archive Network. 2015. [2018-12-10]. A Package for Survival Analysis in R version 2.38 .
    1. Mantel N. Evaluation of survival data and two new rank order statistics arising in its consideration. Cancer Chemother Rep. 1966 Mar;50(3):163–70.
    1. Holm S. A simple sequentially rejective multiple test procedure. Scand Stat Theory Appl. 1979;6(2):65–70.
    1. Bürkner P. brms: an R package for Bayesian multilevel models using Stan. J Stat Soft. 2017;80(1):-. doi: 10.18637/jss.v080.i01.
    1. Brooke J. SUS-A quick and dirty usability scale. In: Jordan P, Thomas B, McClelland I, Weerdmeester B, editors. Usability Evaluation in Industry. London: Taylor & Francis, Inc; 1996. pp. 189–94.
    1. Cechanowicz J, Gutwin C, Brownell B, Goodfellow L. Effects of gamification on participation and data quality in a real-world market research domain. Proceedings of the First International Conference on Gameful Design, Research, and Applications; Gamification'13; October 02-04, 2013; Toronto, Ontario, Canada. 2013. Oct, pp. 58–65.
    1. Li W, Grossman T, Fitzmaurice G. GamiCAD: a gamified tutorial system for first time autocad users. Proceedings of the 25th annual ACM symposium on User interface software and technology; UIST'12; October 07-10, 2012; Cambridge, Massachusetts, USA. 2012. Oct, pp. 103–12.
    1. Bindoff I, de Salas K, Peterson G, Ling T, Lewis I, Wells L, Gee P, Ferguson SG. Quittr: the design of a video game to support smoking cessation. JMIR Serious Games. 2016 Dec 01;4(2):e19. doi: 10.2196/games.6258.

Source: PubMed

Szponzorok és közreműködők

Egészségi állapot

Kábítószer-beavatkozások

3
Iratkozz fel