Development of a predictive model to identify inpatients at risk of re-admission within 30 days of discharge (PARR-30)

John Billings, Ian Blunt, Adam Steventon, Theo Georghiou, Geraint Lewis, Martin Bardsley, John Billings, Ian Blunt, Adam Steventon, Theo Georghiou, Geraint Lewis, Martin Bardsley

Abstract

Objectives: To develop an algorithm for identifying inpatients at high risk of re-admission to a National Health Service (NHS) hospital in England within 30 days of discharge using information that can either be obtained from hospital information systems or from the patient and their notes.

Design: Multivariate statistical analysis of routinely collected hospital episode statistics (HES) data using logistic regression to build the predictive model. The model's performance was calculated using bootstrapping.

Setting: HES data covering all NHS hospital admissions in England.

Participants: The NHS patients were admitted to hospital between April 2008 and March 2009 (10% sample of all admissions, n=576 868).

Main outcome measures: Area under the receiver operating characteristic curve for the algorithm, together with its positive predictive value and sensitivity for a range of risk score thresholds.

Results: The algorithm produces a 'risk score' ranging (0-1) for each admitted patient, and the percentage of patients with a re-admission within 30 days and the mean re-admission costs of all patients are provided for 20 risk bands. At a risk score threshold of 0.5, the positive predictive value (ie, percentage of inpatients identified as high risk who were subsequently re-admitted within 30 days) was 59.2% (95% CI 58.0% to 60.5%); representing 5.4% (95% CI 5.2% to 5.6%) of all inpatients who would be re-admitted within 30 days (sensitivity). The area under the receiver operating characteristic curve was 0.70 (95% CI 0.69 to 0.70).

Conclusions: We have developed a method of identifying inpatients at high risk of unplanned re-admission to NHS hospitals within 30 days of discharge. Though the models had a low sensitivity, we show how to identify subgroups of patients that contain a high proportion of patients who will be re-admitted within 30 days. Additional work is necessary to validate the model in practice.

Figures

Figure 1
Figure 1
Receiver operating characteristic curve for the bootstrapped central estimate (red line) and 95% CI (shaded area).

References

    1. McKee M, Nolte E. Chronic care. In: Smith PC, Mossialos E, Papanicolas I, Leatherman S. Performance measurement for health system improvement: experiences, challenges and prospects. Cambridge: Cambridge University Press, 2009
    1. Billings J, Zeitel L, Lukomnik J, et al. Impact of socioeconomic status on hospital use in New York City. Health Aff 1993; 12:162–73
    1. Health System Priorities in the Aftermath of the Crisis Paris: OECD, 2010. (accessed 10 Sept 2011)
    1. Department of Health Payment by Results Guidance for 2011–12. London: Department of Health, 2011
    1. Department of Health Healthy Lives, Healthy People: transparency in outcomes, proposals for a public health outcomes framework. London: Department of Health, 2010
    1. Blunt I, Bardsley M, Clarke A. Analysis of emergency 30-day readmissions in England using routine hospital data 2004–2010. Is there scope for reduction? (Forthcoming)
    1. Robinson P. Hospitals readmissions and the 30 day threshold. London: CHKS, 2010. (accessed 2 Dec 2011).
    1. Purdy S. Avoiding hospital admissions: what does the research evidence say? London: King's Fund, 2011. (accessed 10 Sept 2011).
    1. Hansen LO, Young RS, Hinami K, et al. Interventions to reduce 30-day rehospitalization: a systematic review. Ann Intern Med 2011;155:520–8
    1. Steventon A, Bardsley M, Billings J, et al. An evaluation of the impact of community-based interventions on hospital use. London: Nuffield Trust, 2011. (accessed 11 Sept 2011).
    1. Roland M, Dusheiko M, Gravelle H, et al. Follow up of people aged 65 and over with a history of emergency admissions: analysis of routine admission data. BMJ 2005;330:289–92
    1. Curry N, Billings J, Darin B, et al. Predictive risk Project Literature Review. London: King's Fund, 2005. (accessed 12 June 2012).
    1. Allaudeen N, Schnipper JL, Orav EJ, et al. Inability of providers to predict unplanned readmissions. J Gen Intern Med 2011; 26:771–6
    1. van Walraven C, Bennett C, Jennings A, et al. Proportion of hospital readmissions deemed avoidable: a systematic review. CMAJ 2011;183:E391–402
    1. Epstein AM, Jha AK, Orav EJ. The relationship between hospital admission rates and rehospitalizations. N Engl J Med 2011;365;2287–95
    1. Billings J, Dixon J, Mijanovich T, et al. Case finding for patients at risk of readmission to hospital: development of algorithm to identify high risk patients. Br Med J 2006;333:327.
    1. Wennberg D, Siegel M, Darin B, et al. Combined predictive model: final report and technical documentation. London: Health Dialog/King's Fund/New York University, 2006. (accessed 10 Sept 2011)
    1. SPARRA: Scottish Patients At Risk of Readmission and Admission Edinburgh: NHS National Services Scotland, 2006. (accessed 10 Sept 2011)
    1. NHS Scotland Information Services Division SPARRA Mental Health: Scottish Patients at Risk of Readmission and Admission (to psychiatric hospitals or units). Edinburgh: NHS Scotland Information Services Division, 2009. (accessed 10 Sept 2011)
    1. Donnan PT, Dorward DW, Mutch B, et al. Development and validation of a model for predicting emergency admissions over the next year (PEONY): a UK historical cohort study. Arch Intern Med 2008;168:1416–22
    1. NHS Wales Informatics Service Reducing emergency risk: the prism tool. NHS Wales Informatics Service; (accessed 12 June 2012)
    1. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the medicare fee-for-service program. N Engl J Med 2009;360:1418–28
    1. Puhan MA, Scharplatz M, Troosters T, et al. Respiratory rehabilitation after acute exacerbation of COPD may reduce risk for readmission and mortality: a systematic review. Respir Res 2005;6:54.
    1. Rowland K, Maitra AK, Richardson DA, et al. The discharge of elderly patients from an accident and emergency department: functional changes and risk of readmission. Age Ageing 1990;19:415–18
    1. Hasan O, Meltzer DO, Shaykevich SA, et al. Hospital readmission in general medicine patients: a prediction model. J Gen Intern Med 2010;25:211–19
    1. Novotny NL, Anderson MA. Prediction of early readmission in medical inpatients using the probability of repeated admission instrument. Nurs Res 2008;57:406–15
    1. van Walraven C, Dhalla IA, Bell C, et al. Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community. CMAJ 2010;182:551–7
    1. Salvi F, Morichi V, Grilli A, et al. Predictive validity of the Identification of Seniors At Risk (ISAR) screening tool in elderly patients presenting to two Italian Emergency Departments. Aging Clin Exp Res 2009;21:69–75
    1. Meldon SW, Mion LC, Palmer RM, et al. A brief risk-stratification tool to predict repeat emergency department visits and hospitalizations in older patients discharged from the emergency department. Acad Emerg Med 2003;10:224–32
    1. Department of Health The operating framework for the NHS in England 2011/12. London: Department of Health, 2010
    1. Dowler C. Emergency readmissions payment ban being ignored. Health Serv J 2011. (accessed 11 Sept 2011)
    1. Fan J, Worster A, Fernandes CM. Predictive validity of the triage risk screening tool for elderly patients in a Canadian emergency department. Am J Emerg Med 2006;24:540–4
    1. NHS Information Centre for Health and Social Care The SUS programme. Leeds: NHS Information Centre for Health and Social Care, 2011. (accessed 12 Sept 2011)
    1. Information Centre for Health and Social Care Hospital episode statistics. Leeds: Information Centre, 2009
    1. Payment by Results Guidance for 2011–12. London: Department of Health, 2011. (accessed 12 June 2012).
    1. Pope GC, Kautter J, Ellis RP, et al. Risk adjustment of Medicare capitation payments using the CMS-HCC model. Health Care Financ Rev 2004;25:119–41
    1. Efron B, Tibshirani R. An introduction to the bootstrap. In: Monographs on statistics and applied probability. New York: Chapman and Hall; 1993; p. xvi, 436
    1. Steyerberg EW, Harrell FE, Jr, Borsboom GJJM, et al. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol 2001;54:774–81
    1. Gu J, Ghosal S, Roy A. Bayesian bootstrap estimation of ROC curve. Stat Med 2008;27:5407–20
    1. Department of Health Payment by results guidance for 2010–11. London: Department of Health, 2010
    1. PBRA3 team Updating and enhancing a resource allocation formula at general practice level based on individual level characteristics (person-based resource allocation). (Forthcoming)
    1. Department of Health NHS reference costs 2007–08. London: Department of Health, 2009
    1. Dixon J, Smith P, Gravelle H, et al. l A person based formula for allocating commissioning funds to general practices in England: development of a statistical model. BMJ 2011;343:d6608 doi: (Published 22 November 2011)
    1. Department of Health Payment by results guidance for 2010–11. London: Department of Health, 2010b
    1. Department for Communities and Local Government The English Indices of Deprivation 2010. (accessed 12 June 2012).
    1. Charlson ME, Popei P, Ales KL, et al. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987;40:373–83
    1. Kansagara D, Englander H, Salanitro A, et al. Risk prediction models for hospital readmission: a systematic review. JAMA 2011;306:1688–98
    1. Bellón JA, Lardelli P, de Dios LJ, et al. Validity of self reported utilisation of primary health care services in an urban population in Spain. J Epidemiol Community Health 2000;54:544–51
    1. Cleary PD, Jette AM. The validity of self-reported physician utilization measures. Med Care 1984;22:796–803
    1. Hansen LO, Young RS, Hinami K, et al. Interventions to reduce 30-day rehospitalisation: a systematic review. Ann Intern Med 2011;155:520–8

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe