Mode equivalence and acceptability of tablet computer-, interactive voice response system-, and paper-based administration of the U.S. National Cancer Institute's Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE)

Antonia V Bennett, Amylou C Dueck, Sandra A Mitchell, Tito R Mendoza, Bryce B Reeve, Thomas M Atkinson, Kathleen M Castro, Andrea Denicoff, Lauren J Rogak, Jay K Harness, James D Bearden, Donna Bryant, Robert D Siegel, Deborah Schrag, Ethan Basch, National Cancer Institute PRO-CTCAE Study Group, Antonia V Bennett, Amylou C Dueck, Sandra A Mitchell, Tito R Mendoza, Bryce B Reeve, Thomas M Atkinson, Kathleen M Castro, Andrea Denicoff, Lauren J Rogak, Jay K Harness, James D Bearden, Donna Bryant, Robert D Siegel, Deborah Schrag, Ethan Basch, National Cancer Institute PRO-CTCAE Study Group

Abstract

Background: PRO-CTCAE is a library of items that measure cancer treatment-related symptomatic adverse events (NCI Contracts: HHSN261201000043C and HHSN 261201000063C). The objective of this study is to examine the equivalence and acceptability of the three data collection modes (Web-enabled touchscreen tablet computer, Interactive voice response system [IVRS], and paper) available within the US National Cancer Institute (NCI) Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) measurement system.

Methods: Participants (n = 112; median age 56.5; 24 % high school or less) receiving treatment for cancer at seven US sites completed 28 PRO-CTCAE items (scoring range 0-4) by three modes (order randomized) at a single study visit. Subjects completed one page (approx. 15 items) of the EORTC QLQ-C30 between each mode as a distractor. Item scores by mode were compared using intraclass correlation coefficients (ICC); differences in scores within the 3-mode crossover design were evaluated with mixed-effects models. Difficulties with each mode experienced by participants were also assessed.

Results: 103 (92 %) completed questionnaires by all three modes. The median ICC comparing tablet vs IVRS was 0.78 (range 0.55-0.90); tablet vs paper: 0.81 (0.62-0.96); IVRS vs paper: 0.78 (0.60-0.91); 89 % of ICCs were ≥0.70. Item-level mean differences by mode were small (medians [ranges] for tablet vs. IVRS = -0.04 [-0.16-0.22]; tablet vs paper = -0.02 [-0.11-0.14]; IVRS vs paper = 0.02 [-0.07-0.19]), and 57/81 (70 %) items had bootstrapped 95 % CI around the effect sizes within +/-0.20. The median time to complete the questionnaire by tablet was 3.4 min; IVRS: 5.8; paper: 4.0. The proportion of participants by mode who reported "no problems" responding to the questionnaire was 86 % tablet, 72 % IVRS, and 98 % paper.

Conclusions: Mode equivalence of items was moderate to high, and comparable to test-retest reliability (median ICC = 0.80). Each mode was acceptable to a majority of respondents. Although the study was powered to detect moderate or larger discrepancies between modes, the observed ICCs and very small mean differences between modes provide evidence to support study designs that are responsive to patient or investigator preference for mode of administration, and justify comparison of results and pooled analyses across studies that employ different PRO-CTCAE modes of administration.

Trial registration: NCT Clinicaltrials.gov identifier: NCT02158637.

References

    1. Basch E, Reeve BB, Mitchell SA, et al.: Development of the National Cancer Institute's patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). J Natl Cancer Inst. 2014;106(9).
    1. Xiao C, Polomano R, Bruner DW. Comparison between patient-reported and clinician-observed symptoms in oncology. Cancer Nurs. 2013;36:E1–E16. doi: 10.1097/NCC.0b013e318269040f.
    1. Dueck AC, Mendoza TR, Mitchell SA, et al.: Validity and Reliability of the U.S. National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). JAMA Oncol. 2015;1(8):1051-9.
    1. Bruner DW, Hanisch LJ, Reeve BB, et al. Stakeholder perspectives on implementing the National Cancer Institute’s patient-reported outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) Transl Behav Med. 2011;1:110–22. doi: 10.1007/s13142-011-0025-3.
    1. Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health. 2008;11:322–33. doi: 10.1111/j.1524-4733.2007.00231.x.
    1. Muehlhausen W, Doll H, Quadri N, et al. Equivalence of electronic and paper administration of patient-reported outcome measures: a systematic review and meta-analysis of studies conducted between 2007 and 2013. Health Qual Life Outcomes. 2015;13:167. doi: 10.1186/s12955-015-0362-x.
    1. Lundy JJ, Coons SJ, Aaronson NK. Testing the measurement equivalence of paper and interactive voice response system versions of the EORTC QLQ-C30. Qual Life Res. 2014;23:229–37. doi: 10.1007/s11136-013-0454-1.
    1. Bjorner JB, Rose M, Gandek B, et al. Method of administration of PROMIS scales did not significantly impact score level, reliability, or validity. J Clin Epidemiol. 2014;67:108–13. doi: 10.1016/j.jclinepi.2013.07.016.
    1. Agel J, Rockwood T, Mundt JC, et al. Comparison of interactive voice response and written self-administered patient surveys for clinical research. Orthopedics. 2001;24:1155–7.
    1. Dunn JA, Arakawa R, Greist JH, et al. Assessing the onset of antidepressant-induced sexual dysfunction using interactive voice response technology. J Clin Psychiatry. 2007;68:525–32. doi: 10.4088/JCP.v68n0406.
    1. Lundy JJ, Coons SJ. Measurement equivalence of interactive voice response and paper versions of the EQ-5D in a cancer patient sample. Value Health. 2011;14:867–71. doi: 10.1016/j.jval.2011.03.001.
    1. Rush AJ, Bernstein IH, Trivedi MH, et al. An evaluation of the quick inventory of depressive symptomatology and the hamilton rating scale for depression: a sequenced treatment alternatives to relieve depression trial report. Biol Psychiatry. 2006;59:493–501. doi: 10.1016/j.biopsych.2005.08.022.
    1. Bennett AV, Keenoy K, Shouery M, et al.: Evaluation of mode equivalence of the MSKCC Bowel Function Instrument, LASA Quality of Life, and Subjective Significance Questionnaire items administered by Web, interactive voice response system (IVRS), and paper. Qual Life Res, 2015, Nov 21 [epub ahead of print].
    1. Coons SJ, Gwaltney CJ, Hays RD, et al. Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force report. Value Health. 2009;12:419–29. doi: 10.1111/j.1524-4733.2008.00470.x.
    1. Eremenco S, Coons SJ, Paty J, et al. PRO data collection in clinical trials using mixed modes: report of the ISPOR PRO mixed modes good research practices task force. Value Health. 2014;17:501–16. doi: 10.1016/j.jval.2014.06.005.
    1. Fawzy MR, Abernethy A, Schoen MW, et al.: Usability testing of the PRO-CTCAE measurement system in patients with cancer. J Clin Oncol 31 (suppl; abstr e17560), 2013.
    1. Reeve BB, Mitchell SA, Dueck AC, et al.: Recommended patient-reported core set of symptoms to measure in adult cancer treatment trials. J Natl Cancer Inst. 2014;106(7).
    1. Reilly CM, Bruner DW, Mitchell SA, et al. A literature synthesis of symptom prevalence and severity in persons receiving active cancer treatment. Support Care Cancer. 2013;21:1525–50. doi: 10.1007/s00520-012-1688-0.
    1. Aaronson NK, Ahmedzai S, Bergman B, et al. The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst. 1993;85:365–76. doi: 10.1093/jnci/85.5.365.
    1. Walter SD, Eliasziw M, Donner A. Sample size and optimal designs for reliability studies. Stat Med. 1998;17:101–10. doi: 10.1002/(SICI)1097-0258(19980115)17:1<101::AID-SIM727>;2-E.
    1. Nunnally JC, Berstein I. Psychometric Methods (ed 3rd) New York: McGraw-Hill; 1994.
    1. Murray J. Likert data: what to use, parametric or non-parametric? Int J Business and Soc Sci. 2013;4:258–64.
    1. Norman G. LIkert scales, levels of measurement adn the “laws” of statistics. Adv Health Sci Educ Theory Pract. 2010;15:625–32. doi: 10.1007/s10459-010-9222-y.
    1. Yarandi H. Crossover Designs and Proc Mixed in SAS, Paper SD04. Nashville: The Proceedings of the SouthEast SAS Users Group; 2004.
    1. Dunlop WP, Cortina JM, Vaslow JB, et al. Meta-analysis of experiments with matched groups or repeated measures designs. Psychol Methods. 1996;1:170–177. doi: 10.1037/1082-989X.1.2.170.
    1. Fleiss JL. Reliability of measurement. In: Fleiss JL, editor. The design and analysis of clinical experiments. New York: Wiley; 1986. pp. 1–32.
    1. Atkinson TM, Li Y, Coffey CW, et al. Reliability of adverse symptom event reporting by clinicians. Qual Life Res. 2012;21:1159–64. doi: 10.1007/s11136-011-0031-4.

Source: PubMed

スポンサーと協力者

医学的状態

薬物療法

3
購読する