Development and validation of a new patient experience tool in patients with serious illness

Karl M Fernstrom, Nathan D Shippee, Alissa L Jones, Heather R Britt, Karl M Fernstrom, Nathan D Shippee, Alissa L Jones, Heather R Britt

Abstract

Background: Patients with serious chronic illnesses face increasingly complex care and are at risk of poor experience due to a fragmented health system. Most current patient experience tools are not designed to address the unique care aspects of this population and the few that exist are delivered too late in the disease trajectory and are not administered longitudinally which makes them less useful across settings.

Methods: We developed a new tool designed to address these gaps. The 25 item scale was tested and refined using randomly cross-validated exploratory and confirmatory factor analyses. Participants were not yet hospice eligible but sick enough to receive benefits of a supportive care approach in the last 2 to 3 years of life. Full information maximum likelihood models were run to confirm the factor structure developed in exploratory analyses. Goodness-of-fit was assessed with the Comparative Fit Index, the Tucker-Lewis Index, and the Root Mean Square Error of Approximation. Test-retest reliability was assessed with the intraclass correlation coefficient and internal consistency of the final scale was examined using Cronbach's alpha.

Results: Exploratory factor analysis revealed three domains - Care Team, Communication, and Care Goals - after removing weak loading and cross loading items. The initial three domain measurement model suggested in the development cohort was tested in the validation cohort and exhibited poor fit X 2 (206) = 565.37, p < 0.001; CFI = 0.879; TLI = 0.864; RMSEA = 0.076. After model respecification, including removing one additional item and allowing paths between theoretically plausible error terms, the final 21 item tool exhibited good fit X 2 (173) = 295.63, p < 0.001; CFI = 0.958; TLI = 0.949; RMSEA = 0.048. Cronbach's alpha revealed high reliability of each domain (Care Team = 0.92, Communication = 0.83, Care Goals = 0.77) and the entire scale (α = 0.91). ICC showed adequate test-retest validity (ICC = 0.58; 95% CI: 0.52-0.65) of the full scale.

Conclusions: When administered earlier in the chronic illness trajectory, a new patient experience scale focused on care teams across settings, communication, and care goals, displayed strong reliability and performed well psychometrically.

Trial registrations: This trial ( NCT01746446 ) was registered at ClinicalTrials.gov on November 27, 2012 (retrospectively registered).

Keywords: Chronic disease; Health care surveys; Patient experience; Patient-centered care; Patient-reported outcome measures.

Figures

Fig. 1
Fig. 1
Patient eligibility screening, enrollment, and analysis flow
Fig. 2
Fig. 2
Final confirmatory factor analysis (CFA) model of the LifeCourse patient experience tool (n = 303)

References

    1. Davis K, Stremikis K, Schoen C, Squires D: Mirror, mirror on the wall, 2014 update: how the US health care system compares internationally. The Commonwealth Fund. 2014;16: 1-31.
    1. Berwick DM, Nolan TW, Whittington J. The triple aim: care, health, and cost. Health Aff. 2008;27(3):759–769. doi: 10.1377/hlthaff.27.3.759.
    1. Obama B: United States health care reform: progress to date and next steps. JAMA. 2016;316(5):525-32.
    1. Centers for Medicare; Medicaid Services, HHS Medicare program; hospital inpatient value-based purchasing program. Final rule. Fed Regist. 2011;76(88):26490.
    1. Wagner EH, Austin BT, Davis C, Hindmarsh M, Schaefer J, Bonomi A. Improving chronic illness care: translating evidence into action. Health Aff. 2001;20(6):64–78. doi: 10.1377/hlthaff.20.6.64.
    1. Jani B, Blane D, Browne S, Montori V, May C, Shippee N, Mair FS. Identifying treatment burden as an important concept for end of life care in those with advanced heart failure. Curr Opin Supportive Palliative Care. 2013;7(1):3–7. doi: 10.1097/SPC.0b013e32835c071f.
    1. VanLare JM, Conway PH. Value-based purchasing — national programs to move from volume to value. N Engl J Med. 2012;367(4):292–295. doi: 10.1056/NEJMp1204939.
    1. Kahn CN, Ault T, Potetz L, Walke T, Chambers JH, Burch S. Assessing Medicare’s hospital pay-for-performance programs and whether they are achieving their goals. Health Aff. 2015;34(8):1281–1288. doi: 10.1377/hlthaff.2015.0158.
    1. Dyer N, Sorra JS, Smith SA, Cleary PD, Hays RD. Psychometric properties of the consumer assessment of healthcare providers and systems (CAHPS (R)) clinician and group adult visit survey. Med Care. 2012;50(Suppl):S28–34. doi: 10.1097/MLR.0b013e31826cbc0d.
    1. O’Malley AJ, Zaslavsky AM, Hays RD, Hepner KA, Keller S, Cleary PD. Exploratory factor analyses of the CAHPS hospital pilot survey responses across and within medical, surgical, and obstetric services. Health Serv Res. 2005;40(6 Pt 2):2078–2095. doi: 10.1111/j.1475-6773.2005.00471.x.
    1. Price RA, Quigley DD, Bradley MA, Teno JM, Parast L, Elliott MN, Haas AC, Stucky BD, Mingura BE, Lorenz K. Hospice experience of care survey. 2014.
    1. Mularski RA, Dy SM, Shugarman LR, Wilkinson AM, Lynn L, Shekelle PG, Morton SC, Sun VC, Hughes RG, Hilton LK, et al. A systematic review of measures of end-of-life care and its outcomes. Health Serv Res. 2007;42(5):1848–1870. doi: 10.1111/j.1475-6773.2007.00721.x.
    1. Garratt A. National and cross-national surveys of patient experiences: Nasjonalt kunnskapssenter for helsetjenesten (Norwegian Knowledge Centre for the Health Services) 2008.
    1. Coulter A, Locock L, Ziebland S, Calabrese J. Collecting data on patient experience is not enough: they must be used to improve care. BMJ. 2014;348(1):g2225. doi: 10.1136/bmj.g2225.
    1. Patient Experience Survey Instruments []. Accessed 3 Nov 2016.
    1. Claessen SJJ, Francke AL, Sixma HJ, de Veer AJE, Deliens L. Measuring relatives’ perspectives on the quality of palliative care: the consumer quality index palliative care. J Pain Symptom Manag. 2013;45(5):875–884. doi: 10.1016/j.jpainsymman.2012.05.007.
    1. Downey L, Curtis JR, Lafferty WE, Herting JR, Engelberg RA. The Quality of Dying and Death Questionnaire (QODD): empirical domains and theoretical perspectives. J Pain Symptom Manag. 2010;39(1):9–22. doi: 10.1016/j.jpainsymman.2009.05.012.
    1. Coleman EA, Mahoney E, Parry C. Assessing the quality of preparation for posthospital care from the patient’s perspective: the care transitions measure. Med Care. 2005;43(3):246–255. doi: 10.1097/00005650-200503000-00007.
    1. Bass PF, 3rd, Wilson JF, Griffith CH. A shortened instrument for literacy screening. J Gen Intern Med. 2003;18(12):1036–1038. doi: 10.1111/j.1525-1497.2003.10651.x.
    1. Gagne JJ, Glynn RJ, Avorn J, Levin R, Schneeweiss S. A combined comorbidity score predicted mortality in elderly patients better than existing scores. J Clin Epidemiol. 2011;64(7):749–759. doi: 10.1016/j.jclinepi.2010.10.004.
    1. StataCorp. Stata Statistical Software: Release 14. College Station: StataCorp LP; 2015.
    1. Gerbing DW, Hamiltion JG. Viability of exploratory factor analysis as a precursor to confirmatory factor analysis. Struct Equ Model. 1996;3(1):62–72. doi: 10.1080/10705519609540030.
    1. Kaiser HF. The varimax criterion for analytic rotation in factor analysis. Psychometrika. 1958;23(3):187–200. doi: 10.1007/BF02289233.
    1. Fabrigar LR, Wegener DT, MacCallum RC, Strahan EJ. Evaluating the use of exploratory factor analysis in psychological research. Psychol Methods. 1999;4(3):272. doi: 10.1037/1082-989X.4.3.272.
    1. Dempster AP, Laird NM, Rubin DB. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society Series B (Methodological). 1977;39(1):1-38.
    1. Enders CK, Bandalos DL. The relative performance of full information maximum likelihood estimation for missing data in structural equation models. Struct Equ Model. 2001;8(3):430–457. doi: 10.1207/S15328007SEM0803_5.
    1. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model. 1999;6(1):1–55. doi: 10.1080/10705519909540118.
    1. Butler M, Ratner E, McCreedy E, Shippee N, Kane RL. Decision aids for advance care planning: an overview of the state of the science. Ann Intern Med. 2014;161(6):408–418. doi: 10.7326/M14-0644.
    1. Tiedje K, Shippee ND, Johnson AM, Flynn PM, Finnie DM, Liesinger JT, May CR, Olson ME, Ridgeway JL, Shah ND. ‘They leave at least believing they had a part in the discussion’: understanding decision aid use and patient-clinician decision-making through qualitative research. Patient Educ Couns. 2013;93(1):86–94. doi: 10.1016/j.pec.2013.03.013.
    1. Shippee ND, Shah N, May CR, Mair F, Montori VM. Cumulative complexity: a functional, patient-centered model of patient complexity can improve research and practice. J Clin Epidemiol. 2012;65(10):1041–1051. doi: 10.1016/j.jclinepi.2012.05.005.
    1. Mathers C, Fat DM, Boerma JT. The global burden of disease: 2004 update. Geneva: World Health Organization; 2008.
    1. Farmer C, Fenu E, O’Flynn N, Guthrie B. Clinical assessment and management of multimorbidity: summary of NICE guidance. BMJ. 2016;354:i4843. doi: 10.1136/bmj.i4843.
    1. Gallacher K, May CR, Montori VM, Mair FS. Understanding patients’ experiences of treatment burden in chronic heart failure using normalization process theory. Ann Fam Med. 2011;9(3):235–243. doi: 10.1370/afm.1249.
    1. Tran V-T, Harrington M, Montori VM, Barnes C, Wicks P, Ravaud P. Adaptation and validation of the Treatment Burden Questionnaire (TBQ) in English using an internet platform. BMC Med. 2014;12(1):109. doi: 10.1186/1741-7015-12-109.
    1. Lynch T, Connor S, Clark D. Mapping levels of palliative care development: a global update. J Pain Symptom Manag. 2013;45(6):1094–1106. doi: 10.1016/j.jpainsymman.2012.05.011.
    1. Ahmedzai SH, Costa A, Blengini C, Bosch A, Sanz-Ortiz J, Ventafridda V, Verhagen SC. A new international framework for palliative care. Eur J Cancer. 2004;40(15):2192–2200. doi: 10.1016/j.ejca.2004.06.009.
    1. Garralda-Domezain E, Hasselaar J, Carrasco-Gimeno JM, Van Beek K, Siouta N, Csikos A, Menten J, Centeno-Cortes C. Integrated palliative care in the Spanish context: a systematic review of the literature. 2016.
    1. Siouta N, van Beek K, Preston N, Hasselaar J, Hughes S, Payne S, Garralda E, Centeno C, van der Eerden M, Groot M, et al. Towards integration of palliative care in patients with chronic heart failure and chronic obstructive pulmonary disease: a systematic literature review of European guidelines and pathways. BMC Palliative Care. 2016;15(1):18. doi: 10.1186/s12904-016-0089-4.
    1. Campbell J, Smith P, Nissen S, Bower P, Elliott M, Roland M. The GP patient survey for use in primary care in the national health service in the UK-development and psychometric characteristics. BMC Fam Pract. 2009;10:57. doi: 10.1186/1471-2296-10-57.
    1. Ritter PL, Stewart AL, Kaymaz H, Sobel DS, Block DA, Lorig KR. Self-reports of health care utilization compared to provider records. J Clin Epidemiol. 2001;54(2):136–141. doi: 10.1016/S0895-4356(00)00261-4.
    1. Richards SH, Coast J, Peters TJ. Patient‐reported use of health service resources compared with information from health providers. Health Soc Care Community. 2003;11(6):510–518. doi: 10.1046/j.1365-2524.2003.00457.x.
    1. CG-CAHPS 12-Month Survey and Instructions []. Accessed 17 Nov 2016.
    1. Zaslavsky AM, Zaborski LB, Cleary PD. Factors affecting response rates to the consumer assessment of health plans study survey. Med Care. 2002;40(6):485–499. doi: 10.1097/00005650-200206000-00006.
    1. Elliott MN, Edwards C, Angeles J, Hambarsoomians K, Hays RD. Patterns of unit and item nonresponse in the CAHPS hospital survey. Health Serv Res. 2005;40(6 Pt 2):2096–2119. doi: 10.1111/j.1475-6773.2005.00476.x.
    1. Sprangers MA, Aaronson NK. The role of health care providers and significant others in evaluating the quality of life of patients with chronic disease: a review. J Clin Epidemiol. 1992;45(7):743–760. doi: 10.1016/0895-4356(92)90052-O.
    1. McHorney CA, Ware JE, Jr, Lu JF, Sherbourne CD. The MOS 36-item short-form health survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Med Care. 1994;32(1):40–66. doi: 10.1097/00005650-199401000-00004.
    1. Truxillo C. Maximum likelihood parameter estimation with incomplete data. Cary: SAS Institute Inc; 2005. pp. 1–19.
    1. Tabachnick BG, Fidell LS: Using Multivariate Statistics. Boston: Allyn and Bacon; 2001.

Source: PubMed

Sponsorzy i współpracownicy

Warunki medyczne

Interwencje lekowe

3
Subskrybuj