Web-Based Module for the Collection of Electronic Patient-Reported Outcomes in People Living With HIV in Nouvelle Aquitaine, France: Usability Evaluation

Diana Barger, Olivier Leleux, Valérie Conte, Vincent Sapparrart, Marie Gapillout, Isabelle Crespel, Marie Erramouspe, Sandrine Delveaux, Linda Wittkop, François Dabis, Fabrice Bonnet, Diana Barger, Olivier Leleux, Valérie Conte, Vincent Sapparrart, Marie Gapillout, Isabelle Crespel, Marie Erramouspe, Sandrine Delveaux, Linda Wittkop, François Dabis, Fabrice Bonnet

Abstract

Background: Patient-reported outcomes (PROs) can be of great value for both research and chronic disease management. We developed a new module of the ANRS CO3 Aquitaine cohort study's Web-based data capture and visualization solution (APPEGE 2.0) for the collection of electronic PROs among people living with HIV cared for in Nouvelle Aquitaine, France.

Objective: This study aimed to evaluate the usability of 2 successively developed prototypes of ARPEGE 2.0's electronic PROs module before launching a pilot study, owing to the novelty of the proposed data collection method for our setting and specific characteristics of the target population.

Methods: A total of 2 sequential rounds of empirical, task-based usability evaluations were conducted, involving 8 research staff and then 7 people living with HIV. Evaluators provided written feedback during round 1 and oral feedback during round 2. Evaluators who completed the full set of tasks responded to the System Usability Scale (SUS). We assessed changes in SUS scores between rounds and concluded usability testing when SUS scores reached a ceiling effect, defining good usability a priori as a usability score of 70.

Results: Insights were generated regarding the visibility of system status and the match between the system and the real world that improved the module's usability. Research staff evaluators reported mean SUS scores of 65 (SD 18.87) and patient evaluators reported mean SUS scores of 85 (SD 5.4; P=.032).

Conclusions: Software modifications, informed by successive rounds of usability testing, resulted in sufficient gains in usability to undertake piloting. Insights generated during evaluations prompted us to find the appropriate balance between optimal security and ease of use.

Trial registration: ClinicalTrials.gov NCT03296202; https://ichgcp.net/clinical-trials-registry/NCT03296202.

International registered report identifier (irrid): RR2-10.2196/10.2196/resprot.9439.

Keywords: patient generated health data; patient reported outcome measures; quality of life.

Conflict of interest statement

Conflicts of Interest: DB has received a speaking fee from Gilead. FB declares to have received reimbursement for attending a symposium from ViiV Healthcare, Gilead, Bristol-Myers Squib, Merck and Janssen, speaking fee and consultancy fee from ViiV Healthcare, Gilead, Bristol-Myers Squib, Merck and Janssen, and funds for research from Gilead and ViiV Healthcare. The other authors do not have any conflict of interest to declare.

©Diana Barger, Olivier Leleux, Valérie Conte, Vincent Sapparrart, Marie Gapillout, Isabelle Crespel, Marie Erramouspe, Sandrine Delveaux, Linda Wittkop, François Dabis, Fabrice Bonnet. Originally published in JMIR Formative Research (http://formative.jmir.org), 18.12.2019.

Figures

Figure 1
Figure 1
Initial log-in page (round 1).
Figure 2
Figure 2
Revised log-in page (round 2).

References

    1. Deeks SG, Lewin SR, Havlir DV. The end of AIDS: HIV infection as a chronic disease. Lancet. 2013 Nov 2;382(9903):1525–33. doi: 10.1016/S0140-6736(13)61809-7.
    1. INSIGHT START Study Group. Lundgren JD, Babiker AG, Gordin F, Emery S, Grund B, Sharma S, Avihingsanon A, Cooper DA, Fätkenheuer G, Llibre JM, Molina JM, Munderi P, Schechter M, Wood R, Klingman KL, Collins S, Lane HC, Phillips AN, Neaton JD. Initiation of antiretroviral therapy in early asymptomatic HIV infection. N Engl J Med. 2015 Aug 27;373(9):795–807. doi: 10.1056/NEJMoa1506816.
    1. Rasmussen LD, Obel N. How do we preserve health among adults living with HIV? Lancet HIV. 2019 Feb;6(2):e69–70. doi: 10.1016/S2352-3018(18)30336-9.
    1. Antiretroviral Therapy Cohort Collaboration Survival of HIV-positive patients starting antiretroviral therapy between 1996 and 2013: a collaborative analysis of cohort studies. Lancet HIV. 2017 Aug;4(8):e349–56. doi: 10.1016/S2352-3018(17)30066-8.
    1. Lazarus JV, Safreed-Harmon K, Barton SE, Costagliola D, Dedes N, Valero J, Gatell JM, Baptista-Leite R, Mendão L, Porter K, Vella S, Rockstroh JK. Beyond viral suppression of HIV - the new quality of life frontier. BMC Med. 2016 Jun 22;14(1):94. doi: 10.1186/s12916-016-0640-4.
    1. Engler K, Lessard D, Lebouché B. A review of HIV-specific patient-reported outcome measures. Patient. 2017 Apr;10(2):187–202. doi: 10.1007/s40271-016-0195-7.
    1. Food and Drug Administration. 2009. Oct, [2019-11-12]. Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims .
    1. Snyder CF, Aaronson NK, Choucair AK, Elliott TE, Greenhalgh J, Halyard MY, Hess R, Miller DM, Reeve BB, Santana M. Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations. Qual Life Res. 2012 Oct;21(8):1305–14. doi: 10.1007/s11136-011-0054-x.
    1. Greenhalgh J, Long AF, Flynn R. The use of patient reported outcome measures in routine clinical practice: lack of impact or lack of theory? Soc Sci Med. 2005 Feb;60(4):833–43. doi: 10.1016/j.socscimed.2004.06.022.
    1. Jones JB, Snyder CF, Wu AW. Issues in the design of internet-based systems for collecting patient-reported outcomes. Qual Life Res. 2007 Oct;16(8):1407–17. doi: 10.1007/s11136-007-9235-z.
    1. Gensheimer SG, Wu AW, Snyder CF, PRO-EHR Users’ Guide Steering Group. PRO-EHR Users’ Guide Working Group Oh, the places we'll go: patient-reported outcomes and electronic health records. Patient. 2018 Dec;11(6):591–8. doi: 10.1007/s40271-018-0321-9.
    1. Crane HM, Lober W, Webster E, Harrington RD, Crane PK, Davis TE, Kitahata MM. Routine collection of patient-reported outcomes in an HIV clinic setting: the first 100 patients. Curr HIV Res. 2007 Jan;5(1):109–18. doi: 10.2174/157016207779316369.
    1. Kozak MS, Mugavero MJ, Ye J, Aban I, Lawrence ST, Nevin CR, Raper JL, McCullumsmith C, Schumacher JE, Crane HM, Kitahata MM, Saag MS, Willig JH. Patient reported outcomes in routine care: advancing data capture for HIV cohort research. Clin Infect Dis. 2012 Jan 1;54(1):141–7. doi: 10.1093/cid/cir727.
    1. Schumacher JE, McCullumsmith C, Mugavero MJ, Ingle-Pang PE, Raper JL, Willig JH, You Z, Batey DS, Crane H, Lawrence ST, Wright C, Treisman G, Saag MS. Routine depression screening in an HIV clinic cohort identifies patients with complex psychiatric co-morbidities who show significant response to treatment. AIDS Behav. 2013 Oct;17(8):2781–91. doi: 10.1007/s10461-012-0342-7.
    1. Barger D, Leleux O, Conte V, Sapparrart V, Gapillout M, Crespel I, Erramouspe M, Delveaux S, Dabis F, Bonnet F. Integrating electronic patient-reported outcome measures into routine HIV care and the ANRS CO3 Aquitaine Cohort's Data Capture and Visualization System (QuAliV): protocol for a formative research study. JMIR Res Protoc. 2018 Jun 7;7(6):e147. doi: 10.2196/resprot.9439.
    1. Ministry of Solidarity and Health. 2013. [2017-11-16]. Medical management of people living with HIV, recommendations of the expert group, 2013 report .
    1. Labbe E, Blanquet M, Gerbaud L, Poirier G, Sass C, Vendittelli F, Moulin J. A new reliable index to measure individual deprivation: the EPICES score. Eur J Public Health. 2015 Aug;25(4):604–9. doi: 10.1093/eurpub/cku231.
    1. O'Connell KA, Skevington SM. An international quality of life instrument to assess wellbeing in adults who are HIV-positive: a short form of the WHOQOL-HIV (31 items) AIDS Behav. 2012 Feb;16(2):452–60. doi: 10.1007/s10461-010-9863-0.
    1. Tran V, Montori VM, Eton DT, Baruch D, Falissard B, Ravaud P. Development and description of measurement properties of an instrument to assess treatment burden among patients with multiple chronic conditions. BMC Med. 2012 Jul 4;10:68. doi: 10.1186/1741-7015-10-68.
    1. Dawson DA, Grant BF, Stinson FS, Zhou Y. Effectiveness of the derived Alcohol Use Disorders Identification Test (AUDIT-C) in screening for alcohol use disorders and risk drinking in the US general population. Alcohol Clin Exp Res. 2005 May;29(5):844–54. doi: 10.1097/01.alc.0000164374.32229.a2.
    1. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001 Sep;16(9):606–13. doi: 10.1046/j.1525-1497.2001.016009606.x.
    1. Coons SJ, Gwaltney CJ, Hays RD, Lundy JJ, Sloan JA, Revicki DA, Lenderking WR, Cella D, Basch E, ISPOR ePRO Task Force Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force report. Value Health. 2009 Jun;12(4):419–29. doi: 10.1111/j.1524-4733.2008.00470.x.
    1. Nielsen J, Landauer TK. A Mathematical Model of the Finding of Usability Problems. Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems; CHI'93; April 24-29, 1993; Amsterdam, The Netherlands. 1993. pp. 206–13.
    1. Brooke J. Hell - Jens Oliver Meiert. 1996. [2019-11-12]. SUS - A Quick and Dirty Usability Scale .
    1. Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. Int J Human-Comput Interact. 2008;24(6):574–94. doi: 10.1080/10447310802205776.
    1. Sauro J. User Experience Magazine. 2011. Aug, [2019-06-15]. SUStisfied? Little-Known System Usability Scale Facts
    1. Nielsen J. Nielsen Norman Group: UX Training, Consulting, & Research. 2012. Jan 3, [2019-04-12]. Usability 101: Introduction to Usability
    1. Dumas JS, Redish JC. A Practical Guide To Usability Testing. Bristol: Intellect Ltd; 1994.
    1. Ericsson AK, Herbert SA. Protocol Analysis: Verbal Reports As Data. Boston: The Mit Press; 1993.

Source: PubMed

Sponsoren und Mitarbeiter

Krankheiten

Drogeninterventionen

3
Abonnieren