Psychometric evaluation of the Care Transition Measure in TRACE-CORE: do we need a better measure?

Milena D Anatchkova, Constance M Barysauskas, Rebecca L Kinney, Catarina I Kiefe, Arlene S Ash, Lisa Lombardini, Jeroan J Allison, Milena D Anatchkova, Constance M Barysauskas, Rebecca L Kinney, Catarina I Kiefe, Arlene S Ash, Lisa Lombardini, Jeroan J Allison

Abstract

Background: The quality of transitional care is associated with important health outcomes such as rehospitalization and costs. The widely used Care Transitions Measure (CTM-15) was developed with a classic test theory approach; its short version (CTM-3) was included in the CAHPS Hospital Survey. We conducted a psychometric evaluation of both measures and explored whether item response theory (IRT) could produce a more precise measure.

Methods and results: As part of the Transitions, Risks, and Actions in Coronary Events Center for Outcomes Research and Education, 1545 participants were interviewed during an acute coronary syndrome hospitalization, providing information on general health status (Short Form-36), CTM-15, health utilization, and care process questions at 1 month postdischarge. We used classic and IRT analyses and compared the measurement precision of CTM-15-, CTM-3-, and CTM-IRT-based score using relative validity. Participants were 79% non-Hispanic white and 67% male, with an average age of 62 years. The CTM-15 had good internal consistency (Cronbach's α=0.95) but demonstrated acquiescence bias (8.7% participants responded "Strongly agree" and 19% responded "Agree" to all items) and limited score variability. These problems were more pronounced for the CTM-3. The CTM-15 differentiated between patient groups defined by self-reported health status, health care utilization, and care transition process indicators. Differences between groups were small (2 to 3 points). There was no gain in measurement precision from IRT scoring. The CTM-3 was not significantly lower for patients reporting rehospitalization or emergency department visits.

Conclusion: We identified psychometric challenges of the CTM, which may limit its value in research and practice. These results are in line with emerging evidence of gaps in the validity of the measure.

Keywords: IRT scoring; acute coronary syndromes; care transitions measure; validity.

© 2014 The Authors. Published on behalf of the American Heart Association, Inc., by Wiley Blackwell.

Figures

Figure 1.
Figure 1.
Scatter plot of CTM‐15 sum scores and CTM‐15 IRT scores (r=0.98). IRT, item response theory.

References

    1. Anderson G. Chronic Care: Making the Case for Ongoing Care. 2010Princeton, NJ: Robert Wood Johnson Foundation
    1. Coleman EA, Smith JD, Frank JC, Eilertsen TB, Thiare JN, Kramer AM. Development and testing of a measure designed to assess the quality of care transitions. Int J Integr Care. 2002; 2:e02.
    1. Coleman EA, Mahoney E, Parry C. Assessing the quality of preparation for posthospital care from the patient's perspective: the care transitions measure. Med Care. 2005; 43:246-255.
    1. Parry C, Mahoney E, Chalmers SA, Coleman EA. Assessing the quality of transitional care: further applications of the care transitions measure. Med Care. 2008; 46:317-322.
    1. Petersen MA, Groenvold M, Aaronson N, Brenne E, Fayers P, Nielsen JD, Sprangers M, Bjorner JB. European Organisation for Research and Treatment of Cancer Quality of Life Group. Scoring based on item response theory did not alter the measurement ability of EORTC QLQ‐C30 scales. J Clin Epidemiol. 2005; 58:902-908.
    1. Shadmi E, Zisberg A, Coleman EA. Translation and validation of the care transition measure into Hebrew and Arabic. Int J Qual Health Care. 2009; 21:97-102.
    1. Waring ME, McManus RH, Saczynski JS, Anatchkova MD, McManus DD, Devereaux RS, Goldberg RJ, Allison JJ, Kiefe CITRACE‐CORE Investigators. Transitions, risks, and actions in coronary events–center for outcomes research and education (TRACE‐CORE): design and rationale. Circ Cardiovasc Qual Outcomes. 2012; 5:e44-e50.
    1. Nunnally JC, Bernstein I. Psychometric Theory. 19943rd edNew York, NY: McGraw‐Hill Book Company
    1. Traub RE. Reliability for the Social Sciences: Theory and Applications. 1994Oaks, CA: Sage
    1. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951; 16:297-334.
    1. Hambleton RK, Swaminathan H, Rogers J. Fundamentals of Item Response Theory. 1991Newbury Park, CA: Sage Publications, Inc
    1. McDonald RP. The dimensionality of tests and items. British Journal of Mathematical and Statistical Psychology. 1981; 34:100-117.
    1. Muthen LK, Muthen BO. Mplus User's Guide. 1998Los Angeles, CA: Muthen & Muthen; ‐
    1. Muthen B. A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika. 1984; 49:115-132.
    1. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model. 1999; 6:1-55.
    1. Reeve BB, Hays RD, Bjorner JB, Cook KF, Crane PK, Teresi JA, Thissen D, Revicki DA, Weiss DJ, Hambleton RK, Liu H, Gershon R, Reise SP, Lai JS, Cella D. Psychometric evaluation and calibration of health‐related quality of life item banks: plans for the patient‐reported outcomes measurement information system (PROMIS). Med Care. 2007; 45:S22-S31.
    1. Bjorner JB, Kosinski M, Ware JE., Jr The feasibility of applying item response theory to measures of migraine impact: a re‐analysis of three clinical studies. Qual Life Res. 2003; 12:887-902.
    1. Bjorner JB, Chang CH, Thissen D, Reeve BB. Developing tailored instruments: item banking and computerized adaptive assessment. Qual Life Res. 2007; 16:95-108.
    1. Samejima F. In: Van der >Linden W, Hambleton R. (eds.). Graded response model. Handbook of Modern Item Response Theory. 1997New York, NY: Springer‐Verlag; 85-100.
    1. Muraki E. In: Linden VD, Hambleton RK. (eds.). Generalized partial credit model. Handbook of Item Response Theory. 1997New York, NY: Springer‐Verlag; 153-164.
    1. Andrich D. Rasch Models for Measurement. 1998Beverly Hills, CA: Sage Publications, Inc
    1. Andrich D. A rating formulation for ordered response categories. Psychometrika. 1978; 43:561-573.
    1. Muraki E, Bock R. Parscale—IRT Based Test Scoring and Item Analysis for Graded Open‐Ended Exercises and Performance Tasks. 1996Chicago, IL: Scientific Software, Inc
    1. Bjorner JB, Smith K, Stone C, Sun X. IRTFIT: a macro for item fit and local dependence tests under IRT models. 2007.
    1. Kerlinger FN. Foundations of Behavioral Research. 1973New York, NY: Holt, Rinehart, & Winston
    1. Deng N, Allison JJ, Fang HJ, Ash AS, Ware JE., Jr Using the bootstrap to establish statistical significance for relative validity comparisons among patient‐reported outcome measures. Health Qual Life Outcomes. 2013; 11:89.
    1. Coleman EA, Parry C, Chalmers SA, Chugh A, Mahoney E. The central role of performance measurement in improving the quality of transitional care. Home Health Care Serv Q. 2007; 26:93-104.
    1. Ryvicker M, McDonald MV, Trachtenberg M, Peng TR, Sridharan S, Feldman PH. Can the care transitions measure predict rehospitalization risk or home health nursing use of home healthcare patients? J Healthc Qual. 2013; 35:32-40.
    1. Holbrook A. In: Lavrakas PJ. (ed.). Acquiescence response bias. Encyclopedia of Survey Research Methods. 2008Thousand Oaks, CA: Sage Publications, Inc; 4-5.
    1. Podsakoff PM, MacKenzie SB, Lee JY, Podsakoff NP. Common method biases in behavioral research: a critical review of the literature and recommended remedies. J Appl Psychol. 2003; 88:879-903.
    1. McLeod J, Stolee P, Walker J, Heckman G. Measuring care transition quality for older patients with musculoskeletal disorders. Musculoskeletal Care. 2014; 12:13-21.
    1. Coleman EA, Smith JD, Frank JC, Min SJ, Parry C, Kramer AM. Preparing patients and caregivers to participate in care delivered across settings: the care transitions intervention. J Am Geriatr Soc. 2004; 52:1817-1825.
    1. McHorney CA, Haley SM, Ware JE., Jr Evaluation of the MOS SF‐36 physical functioning scale (PF‐10): II. Comparison of relative precision using likert and rasch scoring methods. J Clin Epidemiol. 1997; 50:451-461.
    1. Birbeck GL, Kim S, Hays RD, Vickrey BG. Quality of life measures in epilepsy: how well can they detect change over time? Neurology. 2000; 54:1822-1827.
    1. Fitzpatrick R, Norquist JM, Dawson J, Jenkinson C. Rasch scoring of outcomes of total hip replacement. J Clin Epidemiol. 2003; 56:68-74.
    1. Norquist JM, Fitzpatrick R, Jenkinson C. Rasch measurement in the assessment of amytrophic lateral sclerosis patients. J Appl Meas. 2003; 4:249-257.
    1. Khan A, Chien CW, Brauer SG. Rasch‐based scoring offered more precision in differentiating patient groups in measuring upper limb function. J Clin Epidemiol. 2013; 66:681-687.
    1. Las HayasC, Bilbao A, Quintana JM, Garcia S, Lafuente IIRYSS Cataract Group. A comparison of standard scoring versus rasch scoring of the visual function index‐14 in patients with cataracts. Invest Ophthalmol Vis Sci. 2011; 52:4800-4807.
    1. Wolfe F, Kong SX. Rasch analysis of the western Ontario MacMaster questionnaire (WOMAC) in 2205 patients with osteoarthritis, rheumatoid arthritis, and fibromyalgia. Ann Rheum Dis. 1999; 58:563-568.
    1. Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient‐reported outcomes. J Clin Epidemiol. 2008; 61:102-109.
    1. Copay AG, Subach BR, Glassman SD, Polly DW, Jr, Schuler TC. Understanding the minimum clinically important difference: a review of concepts and methods. Spine J. 2007; 7:541-546.
    1. Turner D, Schunemann HJ, Griffith LE, Beaton DE, Griffiths AM, Critch JN, Guyatt GH. The minimal detectable change cannot reliably replace the minimal important difference. J Clin Epidemiol. 2010; 63:28-36.

Source: PubMed

Sponsorit ja yhteistyökumppanit

Sairaudet

Huumeiden interventiot

3
Tilaa