Analyzing User Engagement Within a Patient-Reported Outcomes Texting Tool for Diabetes Management: Engagement Phenotype Study

Soumik Mandal, Hayley M Belli, Jocelyn Cruz, Devin Mann, Antoinette Schoenthaler, Soumik Mandal, Hayley M Belli, Jocelyn Cruz, Devin Mann, Antoinette Schoenthaler

Abstract

Background: Patient-reported outcomes (PROs) capture patients' views on their health conditions and its management, and are increasingly used in clinical trials, including those targeting type 2 diabetes (T2D). Mobile health (mHealth) tools offer novel solutions for collecting PRO data in real time. Although patients are at the center of any PRO-based intervention, few studies have examined user engagement with PRO mHealth tools.

Objective: This study aimed to evaluate user engagement with a PRO mHealth tool for T2D management, identify patterns of user engagement and similarities and differences between the patients, and identify the characteristics of patients who are likely to drop out or be less engaged with a PRO mHealth tool.

Methods: We extracted user engagement data from an ongoing clinical trial that tested the efficacy of a PRO mHealth tool designed to improve hemoglobin A1c levels in patients with uncontrolled T2D. To date, 61 patients have been randomized to the intervention, where they are sent 6 PRO text messages a day that are relevant to T2D self-management (healthy eating and medication adherence) over the 12-month study. To analyze user engagement, we first compared the response rate (RR) and response time between patients who completed the 12-month intervention and those who dropped out early (noncompleters). Next, we leveraged latent class trajectory modeling to classify patients from the completer group into 3 subgroups based on similarity in the longitudinal engagement data. Finally, we investigated the differences between the subgroups of completers from various cross-sections (time of the day and day of the week) and PRO types. We also explored the patient demographics and their distribution among the subgroups.

Results: Overall, 19 noncompleters had a lower RR to PRO questions and took longer to respond to PRO questions than 42 completers. Among completers, the longitudinal RRs demonstrated differences in engagement patterns over time. The completers with the lowest engagement showed peak engagement during month 5, almost at the midstage of the program. The remaining subgroups showed peak engagement at the beginning of the intervention, followed by either a steady decline or sustained high engagement. Comparisons of the demographic characteristics showed significant differences between the high engaged and low engaged subgroups. The high engaged completers were predominantly older, of Hispanic descent, bilingual, and had a graduate degree. In comparison, the low engaged subgroup was composed mostly of African American patients who reported the lowest annual income, with one of every 3 patients earning less than US $20,000 annually.

Conclusions: There are discernible engagement phenotypes based on individual PRO responses, and their patterns vary in the timing of peak engagement and demographics. Future studies could use these findings to predict engagement categories and tailor interventions to promote longitudinal engagement.

Trial registration: Clinicaltrials.gov NCT03652389; https://ichgcp.net/clinical-trials-registry/NCT03652389.

International registered report identifier (irrid): RR2-10.2196/18554.

Keywords: SMS; digital health; digital phenotyping; health behavior; mHealth; mobile health; patient-reported outcomes; type 2 diabetes; user engagement.

Conflict of interest statement

Conflicts of Interest: AS is a former consultant of the company that developed the digital tool.

©Soumik Mandal, Hayley M Belli, Jocelyn Cruz, Devin Mann, Antoinette Schoenthaler. Originally published in JMIR Diabetes (https://diabetes.jmir.org), 14.11.2022.

Figures

Figure 1
Figure 1
Duration in the program for patients who did not complete (noncompleters) the study.
Figure 2
Figure 2
Distribution of response rate for the completers group.
Figure 3
Figure 3
Comparison of response rate (A) and response time (B) between the completers and noncompleters group.
Figure 4
Figure 4
Trajectories of user engagement among completers in the Investigating an mHealth texting tool for embedding patient-reported data in diabetes management (i-Matter) intervention, (A) individual raw engagement in the left panel, (B) smoothed mean engagement in the right panel.

References

    1. Patient-Reported Outcomes. National Quality Forum. [2022-05-12]. .
    1. Guidance for Industry: Patient-reported outcome measures: use in medical product development to support labeling claims. U.S. Food & Drug Administration. 2009. Dec, [2022-05-12]. .
    1. Doward LC, McKenna SP. Defining patient-reported outcomes. Value Health. 2004;7 Suppl 1:S4–8. doi: 10.1111/j.1524-4733.2004.7s102.x. S1098-3015(10)60228-8
    1. Lohr KN, Zebrack BJ. Using patient-reported outcomes in clinical practice: challenges and opportunities. Qual Life Res. 2009 Feb;18(1):99–107. doi: 10.1007/s11136-008-9413-7.
    1. Deshpande PR, Rajan S, Sudeepthi BL, Abdul Nazir CP. Patient-reported outcomes: a new era in clinical research. Perspect Clin Res. 2011 Oct;2(4):137–44. doi: 10.4103/2229-3485.86879. PCR-2-137
    1. Chin R, Lee B. Principles and Practice of Clinical Trial Medicine. Amsterdam, The Netherlands: Elsevier; 2008.
    1. Black N. Patient reported outcome measures could help transform healthcare. BMJ. 2013 Jan 28;346:f167. doi: 10.1136/bmj.f167.
    1. What is patient-centred health care? A review of definitions and principles. International Alliance of Patients' Organizations. 2007. [2022-10-31]. .
    1. Leonardsson O, Rolfson O, Hommel A, Garellick G, Åkesson K, Rogmark C. Patient-reported outcome after displaced femoral neck fracture: a national survey of 4467 patients. J Bone Joint Surg Am. 2013 Sep 18;95(18):1693–9. doi: 10.2106/JBJS.L.00836.1734765
    1. Adans-Dester CP, Bamberg S, Bertacchi FP, Caulfield B, Chappie K, Demarchi D, Erb MK, Estrada J, Fabara EE, Freni M, Friedl KE, Ghaffari R, Gill G, Greenberg MS, Hoyt RW, Jovanov E, Kanzler CM, Katabi D, Kernan M, Kigin C, Lee SI, Leonhardt S, Lovell NH, Mantilla J, McCoy Jr TH, Luo NM, Miller GA, Moore J, O'Keeffe D, Palmer J, Parisi F, Patel S, Po J, Pugliese BL, Quatieri T, Rahman T, Ramasarma N, Rogers JA, Ruiz-Esparza GU, Sapienza S, Schiurring G, Schwamm L, Shafiee H, Kelly Silacci S, Sims NM, Talkar T, Tharion WJ, Toombs JA, Uschnig C, Vergara-Diaz GP, Wacnik P, Wang MD, Welch J, Williamson L, Zafonte R, Zai A, Zhang YT, Tearney GJ, Ahmad R, Walt DR, Bonato P. Can mHealth technology help mitigate the effects of the COVID-19 pandemic? IEEE Open J Eng Med Biol. 2020 Aug 7;1:243–8. doi: 10.1109/OJEMB.2020.3015141.
    1. Trautmann F, Hentschel L, Hornemann B, Rentsch A, Baumann M, Ehninger G, Schmitt J, Schuler M. Electronic real-time assessment of patient-reported outcomes in routine care-first findings and experiences from the implementation in a comprehensive cancer center. Support Care Cancer. 2016 Jul;24(7):3047–56. doi: 10.1007/s00520-016-3127-0.10.1007/s00520-016-3127-0
    1. Kallen MA, Yang D, Haas N. A technical solution to improving palliative and hospice care. Support Care Cancer. 2012 Jan;20(1):167–74. doi: 10.1007/s00520-011-1086-z.
    1. Boyes A, Newell S, Girgis A, McElduff P, Sanson-Fisher R. Does routine assessment and real-time feedback improve cancer patients' psychosocial well-being? Eur J Cancer Care (Engl) 2006 May;15(2):163–71. doi: 10.1111/j.1365-2354.2005.00633.x.ECC633
    1. Hilarius DL, Kloeg PH, Gundy CM, Aaronson NK. Use of health-related quality-of-life assessments in daily clinical oncology nursing practice: a community hospital-based intervention study. Cancer. 2008 Aug 01;113(3):628–37. doi: 10.1002/cncr.23623. doi: 10.1002/cncr.23623.
    1. Fox CS, Golden SH, Anderson C, Bray GA, Burke LE, de Boer IH, Deedwania P, Eckel RH, Ershow AG, Fradkin J, Inzucchi SE, Kosiborod M, Nelson RG, Patel MJ, Pignone M, Quinn L, Schauer PR, Selvin E, Vafiadis DK, American Heart Association Diabetes Committee of the Council on Lifestyle and Cardiometabolic Health, Council on Clinical Cardiology, Council on Cardiovascular and Stroke Nursing, Council on Cardiovascular Surgery and Anesthesia, Council on Quality of Care and Outcomes Research, and the American Diabetes Association Update on prevention of cardiovascular disease in adults with type 2 diabetes mellitus in light of recent evidence: a scientific statement from the American Heart Association and the American Diabetes Association. Circulation. 2015 Aug 25;132(8):691–718. doi: 10.1161/CIR.0000000000000230.CIR.0000000000000230
    1. Geiss LS, Wang J, Cheng YJ, Thompson TJ, Barker L, Li Y, Albright AL, Gregg EW. Prevalence and incidence trends for diagnosed diabetes among adults aged 20 to 79 years, United States, 1980-2012. JAMA. 2014 Sep 24;312(12):1218–26. doi: 10.1001/jama.2014.11494.1906616
    1. Cella D, Riley W, Stone A, Rothrock N, Reeve B, Yount S, Amtmann D, Bode R, Buysse D, Choi S, Cook K, Devellis R, DeWalt D, Fries JF, Gershon R, Hahn EA, Lai JS, Pilkonis P, Revicki D, Rose M, Weinfurt K, Hays R, PROMIS Cooperative Group The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. J Clin Epidemiol. 2010 Nov;63(11):1179–94. doi: 10.1016/j.jclinepi.2010.04.011. S0895-4356(10)00173-3
    1. Snyder CF, Jensen RE, Segal JB, Wu AW. Patient-reported outcomes (PROs): putting the patient perspective in patient-centered outcomes research. Med Care. 2013 Aug;51(8 Suppl 3):S73–9. doi: 10.1097/MLR.0b013e31829b1d84.
    1. Speight J, Barendse SM. FDA guidance on patient reported outcomes. BMJ. 2010 Jun 21;340:c2921. doi: 10.1136/bmj.c2921.
    1. Funnell MM, Brown TL, Childs BP, Haas LB, Hosey GM, Jensen B, Maryniuk M, Peyrot M, Piette J, Reader D, Siminerio LM, Weinger K, Weiss MA. National Standards for diabetes self-management education. Diabetes Care. 2011 Jan;34 Suppl 1:S89–96. doi: 10.2337/dc11-S089. 34/Supplement_1/S89
    1. Cho JH, Ha SJ, Kao LR, Megraw TL, Chae CB. A novel DNA-binding protein bound to the mitochondrial inner membrane restores the null mutation of mitochondrial histone Abf2p in Saccharomyces cerevisiae. Mol Cell Biol. 1998 Oct;18(10):5712–23. doi: 10.1128/MCB.18.10.5712.
    1. Barsdorf AI, Rubinstein E, Jaksa A. Patient-reported outcomes (Pros) in diabetes clinical trials. Value Health. 2013 May 1;16(3):A168–9. doi: 10.1016/j.jval.2013.03.842.
    1. Ridgeway JL, Beebe TJ, Chute CG, Eton DT, Hart LA, Frost MH, Jensen D, Montori VM, Smith JG, Smith SA, Tan AD, Yost KJ, Ziegenfuss JY, Sloan JA. A brief Patient-Reported Outcomes Quality of Life (PROQOL) instrument to improve patient care. PLoS Med. 2013 Nov;10(11):e1001548. doi: 10.1371/journal.pmed.1001548. PMEDICINE-D-12-03304
    1. Marshall S, Haywood K, Fitzpatrick R. Impact of patient-reported outcome measures on routine practice: a structured review. J Eval Clin Pract. 2006 Oct;12(5):559–68. doi: 10.1111/j.1365-2753.2006.00650.x.JEP650
    1. Vieta A, Badia X, Sacristán JA. A systematic review of patient-reported and economic outcomes: value to stakeholders in the decision-making process in patients with type 2 diabetes mellitus. Clin Ther. 2011 Sep;33(9):1225–45. doi: 10.1016/j.clinthera.2011.07.013.S0149-2918(11)00505-4
    1. Peters M, Crocker H, Jenkinson C, Doll H, Fitzpatrick R. The routine collection of patient-reported outcome measures (PROMs) for long-term conditions in primary care: a cohort survey. BMJ Open. 2014 Feb 21;4(2):e003968. doi: 10.1136/bmjopen-2013-003968. bmjopen-2013-003968
    1. Valderas JM, Kotzeva A, Espallargues M, Guyatt G, Ferrans CE, Halyard MY, Revicki DA, Symonds T, Parada A, Alonso J. The impact of measuring patient-reported outcomes in clinical practice: a systematic review of the literature. Qual Life Res. 2008 Mar;17(2):179–93. doi: 10.1007/s11136-007-9295-0.
    1. Maddigan SL, Majumdar SR, Guirguis LM, Lewanczuk RZ, Lee TK, Toth EL, Johnson JA. Improvements in patient-reported outcomes associated with an intervention to enhance quality of care for rural patients with type 2 diabetes: results of a controlled trial. Diabetes Care. 2004 Jun;27(6):1306–12. doi: 10.2337/diacare.27.6.1306.27/6/1306
    1. Baron JS, Hirani SP, Newman SP. Investigating the behavioural effects of a mobile-phone based home telehealth intervention in people with insulin-requiring diabetes: results of a randomized controlled trial with patient interviews. J Telemed Telecare. 2017 Jun;23(5):503–12. doi: 10.1177/1357633X16655911.1357633X16655911
    1. Årsand W, Frøisland DH, Skrøvseth SO, Chomutare T, Tatara N, Hartvigsen G, Tufano JT. Mobile health applications to assist patients with diabetes: lessons learned and design implications. J Diabetes Sci Technol. 2012 Sep 01;6(5):1197–206. doi: 10.1177/193229681200600525.
    1. Schoenthaler A, Cruz J, Payano L, Rosado M, Labbe K, Johnson C, Gonzalez J, Patxot M, Patel S, Leven E, Mann D. Investigation of a mobile health texting tool for embedding patient-reported data into diabetes management (i-Matter): development and usability study. JMIR Form Res. 2020 Aug 31;4(8):e18554. doi: 10.2196/18554. v4i8e18554
    1. Holtz B, Lauckner C. Diabetes management via mobile phones: a systematic review. Telemed J E Health. 2012 Apr;18(3):175–84. doi: 10.1089/tmj.2011.0119.
    1. Baron JS, Hirani S, Newman SP. A randomised, controlled trial of the effects of a mobile telehealth intervention on clinical and patient-reported outcomes in people with poorly controlled diabetes. J Telemed Telecare. 2017 Feb;23(2):207–16. doi: 10.1177/1357633X16631628.1357633X16631628
    1. Wei Y, Zheng P, Deng H, Wang X, Li X, Fu H. Design features for improving mobile health intervention user engagement: systematic review and thematic analysis. J Med Internet Res. 2020 Dec 09;22(12):e21687. doi: 10.2196/21687. v22i12e21687
    1. Mayberry LS, Lyles CR, Oldenburg B, Osborn CY, Parks M, Peek ME. mHealth interventions for disadvantaged and vulnerable people with type 2 diabetes. Curr Diab Rep. 2019 Nov 25;19(12):148. doi: 10.1007/s11892-019-1280-9. 10.1007/s11892-019-1280-9
    1. Nelson LA, Spieker A, Greevy R, LeStourgeon LM, Wallston KA, Mayberry LS. User engagement among diverse adults in a 12-month text message-delivered diabetes support intervention: results from a randomized controlled trial. JMIR Mhealth Uhealth. 2020 Jul 21;8(7):e17534. doi: 10.2196/17534. v8i7e17534
    1. Nelson LA, Wallston KA, Kripalani S, LeStourgeon LM, Williamson SE, Mayberry LS. Assessing barriers to diabetes medication adherence using the Information-Motivation-Behavioral skills model. Diabetes Res Clin Pract. 2018 Aug;142:374–84. doi: 10.1016/j.diabres.2018.05.046. S0168-8227(17)31944-7
    1. Lennon H, Kelly S, Sperrin M, Buchan I, Cross AJ, Leitzmann M, Cook MB, Renehan AG. Framework to construct and interpret latent class trajectory modelling. BMJ Open. 2018 Jul 07;8(7):e020683. doi: 10.1136/bmjopen-2017-020683. bmjopen-2017-020683
    1. Nagin DS. Group-Based Modeling of Development. Cambridge, MA, USA: Harvard University Press; 2005.
    1. Svedbo Engström M, Leksell J, Johansson UB, Gudbjörnsdottir S. What is important for you? A qualitative interview study of living with diabetes and experiences of diabetes care to establish a basis for a tailored Patient-Reported Outcome Measure for the Swedish National Diabetes Register. BMJ Open. 2016 Mar 24;6(3):e010249. doi: 10.1136/bmjopen-2015-010249. bmjopen-2015-010249
    1. Steiner JF, Zeng C, Comer AC, Barrow JC, Langer JN, Steffen DA, Steiner CA. Factors associated with opting out of automated text and telephone messages among adult members of an integrated health care system. JAMA Netw Open. 2021 Mar 01;4(3):e213479. doi: 10.1001/jamanetworkopen.2021.3479. 2777872
    1. Willi SM, Miller KM, DiMeglio LA, Klingensmith GJ, Simmons JH, Tamborlane WV, Nadeau KJ, Kittelsrud JM, Huckfeldt P, Beck RW, Lipman TH, T1D Exchange Clinic Network Racial-ethnic disparities in management and outcomes among children with type 1 diabetes. Pediatrics. 2015 Mar;135(3):424–34. doi: 10.1542/peds.2014-1774. peds.2014-1774
    1. Silverman-Lloyd LG, Dominguez Cortez J, Godage SK, Valenzuela Araujo D, Rivera T, Polk S, DeCamp LR. Immigrant Latino parents demonstrated high interactivity with pediatric primary care text messaging intervention. Mhealth. 2020 Oct 5;6:45. doi: 10.21037/mhealth.2020.01.06. doi: 10.21037/-06-2020.01.06
    1. Cartujano-Barrera F, Arana-Chicas E, Ramírez-Mantilla M, Perales J, Cox LS, Ellerbeck EF, Catley D, Cupertino AP. "Every day I think about your messages": assessing text messaging engagement among Latino smokers in a mobile cessation program. Patient Prefer Adherence. 2019 Jul 22;13:1213–9. doi: 10.2147/PPA.S209547. doi: 10.2147/PPA.S209547.209547
    1. Watterson JL, Rodriguez HP, Shortell SM, Aguilera A. Improved diabetes care management through a text-message intervention for low-income patients: mixed-methods pilot study. JMIR Diabetes. 2018 Oct 30;3(4):e15. doi: 10.2196/diabetes.8645. v3i4e15
    1. Mobile Fact Sheet. Pew Research Center. 2021. Apr 7, [2022-05-23].
    1. Mayberry LS, Bergner EM, Chakkalakal RJ, Elasy TA, Osborn CY. Self-care disparities among adults with type 2 diabetes in the USA. Curr Diab Rep. 2016 Nov;16(11):113. doi: 10.1007/s11892-016-0796-5. 10.1007/s11892-016-0796-5
    1. Nelson LA, Mulvaney SA, Johnson KB, Osborn CY. mHealth intervention elements and user characteristics determine utility: a mixed-methods analysis. Diabetes Technol Ther. 2017 Jan;19(1):9–17. doi: 10.1089/dia.2016.0294.
    1. Dobson KG, Hall P. A pilot study examining patient attitudes and intentions to adopt assistive technologies into type 2 diabetes self-management. J Diabetes Sci Technol. 2015 Mar;9(2):309–15. doi: 10.1177/1932296814560395. 1932296814560395
    1. Humble JR, Tolley EA, Krukowski RA, Womack CR, Motley TS, Bailey JE. Use of and interest in mobile health for diabetes self-care in vulnerable populations. J Telemed Telecare. 2016 Jan;22(1):32–8. doi: 10.1177/1357633X15586641.1357633X15586641
    1. AARP research finds 90 percent of adults 50+ use personal technology to stay connected. American Association of Retired Persons. 2018. Feb 28, [2022-05-23]. .
    1. McCurdie T, Taneva S, Casselman M, Yeung M, McDaniel C, Ho W, Cafazzo J. mHealth consumer apps: the case for user-centered design. Biomed Instrum Technol. 2012;Suppl:49–56. doi: 10.2345/0899-8205-46.s2.49.
    1. Nundy S, Razi RR, Dick JJ, Smith B, Mayo A, O'Connor A, Meltzer DO. A text messaging intervention to improve heart failure self-management after hospital discharge in a largely African-American population: before-after study. J Med Internet Res. 2013 Mar 11;15(3):e53. doi: 10.2196/jmir.2317. v15i3e53
    1. Fortin M, Bravo G, Hudon C, Vanasse A, Lapointe L. Prevalence of multimorbidity among adults seen in family practice. Ann Fam Med. 2005;3(3):223–8. doi: 10.1370/afm.272. 3/3/223
    1. Sasseville M, Chouinard MC, Fortin M. Patient-reported outcomes in multimorbidity intervention research: a scoping review. Int J Nurs Stud. 2018 Jan;77:145–53. doi: 10.1016/j.ijnurstu.2017.09.016.S0020-7489(17)30223-7

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe