Prediction models for risk of developing type 2 diabetes: systematic literature search and independent external validation study

Abstract

Objective: To identify existing prediction models for the risk of development of type 2 diabetes and to externally validate them in a large independent cohort.

Data sources: Systematic search of English, German, and Dutch literature in PubMed until February 2011 to identify prediction models for diabetes.

Design: Performance of the models was assessed in terms of discrimination (C statistic) and calibration (calibration plots and Hosmer-Lemeshow test).The validation study was a prospective cohort study, with a case cohort study in a random subcohort.

Setting: Models were applied to the Dutch cohort of the European Prospective Investigation into Cancer and Nutrition cohort study (EPIC-NL).

Participants: 38,379 people aged 20-70 with no diabetes at baseline, 2506 of whom made up the random subcohort.

Outcome measure: Incident type 2 diabetes.

Results: The review identified 16 studies containing 25 prediction models. We considered 12 models as basic because they were based on variables that can be assessed non-invasively and 13 models as extended because they additionally included conventional biomarkers such as glucose concentration. During a median follow-up of 10.2 years there were 924 cases in the full EPIC-NL cohort and 79 in the random subcohort. The C statistic for the basic models ranged from 0.74 (95% confidence interval 0.73 to 0.75) to 0.84 (0.82 to 0.85) for risk at 7.5 years. For prediction models including biomarkers the C statistic ranged from 0.81 (0.80 to 0.83) to 0.93 (0.92 to 0.94). Most prediction models overestimated the observed risk of diabetes, particularly at higher observed risks. After adjustment for differences in incidence of diabetes, calibration improved considerably.

Conclusions: Most basic prediction models can identify people at high risk of developing diabetes in a time frame of five to 10 years. Models including biomarkers classified cases slightly better than basic ones. Most models overestimated the actual risk of diabetes. Existing prediction models therefore perform well to identify those at high risk, but cannot sufficiently quantify actual risk of future diabetes.

Conflict of interest statement

Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

Figures

Fig 1 Overview of systematic literature search of studies that derived prediction models for risk of type 2 diabetes

Fig 2 Calibration plots for recalibrated basic models risk of diabetes at 7.5 years depicting predicted risk against observed risk of developing type 2 diabetes in validation dataset. Dashed line (45° line) from zero denotes ideal calibration line (slope=1, intercept=0) and other lines are smooth calibration curve for each model

Fig 3 Calibration plots for recalibrated extended models risk of diabetes at 7.5 years depicting predicted risk against observed risk of developing type 2 diabetes in validation dataset. Dashed line (45° line) from zero denotes ideal calibration line (slope=1, intercept=0) and other lines are smooth calibration curve for each model