Development and validation of a decision support tool for the diagnosis of acute heart failure: systematic review, meta-analysis, and modelling study

Abstract

Objectives: To evaluate the diagnostic performance of N-terminal pro-B-type natriuretic peptide (NT-proBNP) thresholds for acute heart failure and to develop and validate a decision support tool that combines NT-proBNP concentrations with clinical characteristics.

Design: Individual patient level data meta-analysis and modelling study.

Setting: Fourteen studies from 13 countries, including randomised controlled trials and prospective observational studies.

Participants: Individual patient level data for 10 369 patients with suspected acute heart failure were pooled for the meta-analysis to evaluate NT-proBNP thresholds. A decision support tool (Collaboration for the Diagnosis and Evaluation of Heart Failure (CoDE-HF)) that combines NT-proBNP with clinical variables to report the probability of acute heart failure for an individual patient was developed and validated.

Main outcome measure: Adjudicated diagnosis of acute heart failure.

Results: Overall, 43.9% (4549/10 369) of patients had an adjudicated diagnosis of acute heart failure (73.3% (2286/3119) and 29.0% (1802/6208) in those with and without previous heart failure, respectively). The negative predictive value of the guideline recommended rule-out threshold of 300 pg/mL was 94.6% (95% confidence interval 91.9% to 96.4%); despite use of age specific rule-in thresholds, the positive predictive value varied at 61.0% (55.3% to 66.4%), 73.5% (62.3% to 82.3%), and 80.2% (70.9% to 87.1%), in patients aged <50 years, 50-75 years, and >75 years, respectively. Performance varied in most subgroups, particularly patients with obesity, renal impairment, or previous heart failure. CoDE-HF was well calibrated, with excellent discrimination in patients with and without previous heart failure (area under the receiver operator curve 0.846 (0.830 to 0.862) and 0.925 (0.919 to 0.932) and Brier scores of 0.130 and 0.099, respectively). In patients without previous heart failure, the diagnostic performance was consistent across all subgroups, with 40.3% (2502/6208) identified at low probability (negative predictive value of 98.6%, 97.8% to 99.1%) and 28.0% (1737/6208) at high probability (positive predictive value of 75.0%, 65.7% to 82.5%) of having acute heart failure.

Conclusions: In an international, collaborative evaluation of the diagnostic performance of NT-proBNP, guideline recommended thresholds to diagnose acute heart failure varied substantially in important patient subgroups. The CoDE-HF decision support tool incorporating NT-proBNP as a continuous measure and other clinical variables provides a more consistent, accurate, and individualised approach.

Study registration: PROSPERO CRD42019159407.

Conflict of interest statement

Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/disclosure-of-interest/ and declare: support from the British Heart Foundation, Medical Research Council, and Chief Scientist Office; Y-EC has received honorariums for lectures and presentations from Biomérieux, Roche Diagnostics, and Thermo Fisher; CdF has received consulting fees from Fuji Rebio, Ortho Diagnostics, Quidel, and Roche Diagnostics and a patent entitled “Methods for assessing differential risk for developing heart failure” (patent number: PCT/US2015/029838); SS has received a grant from Roche Diagnostics and a patent entitled “Methods for assessing differential risk for developing heart failure” (patent number: PCT/US2015/029838); ABG has received personal fees and non-financial support from Roche Diagnostics during the conduct of the study and personal fees from Abbott and AstraZeneca, grants, personal fees, and non-financial support from Boehringer-Ingelheim, and personal fees and non-financial support from Novartis and Vifor outside the submitted work. YP has received consulting fees from Roche Diagnostics, Pfizer, and Forbion and honorariums from CVOI and Daiichi Sankyo; HKG has received grants from Roche Diagnostics, Jana Care, Ortho Clinical, Novartis, Pfizer, Alnylam, and Akcea (IONIS), consulting fees from Amgen, Eko, Merck, and Pfizer, and stock in Eko; JCW works as a biostatistician at the biotech company BRAHMS GmbH, part of Thermo Fisher Scientific; MM has received grants from Health Care Research Projects and Biomarker Research and personal fees from Consulting outside the submitted work; ASVS has received speaker fees from Abbott Diagnostics outside the submitted work; AMR has received grants, personal fees, and non-financial support from Roche Diagnostics outside the submitted work; JJVM has received consulting fees from Alnylam, AstraZeneca, Bayer, Boehringer Ingelheim, BMS, Cardurion, Dal-Cor, GSK, Ionis, KBP Biosciences, Novartis, Pfizer, and Theracos, payments for advisory boards, symposiums, or lectures from Abbott, Alkem Metabolics, Canadian Medical and Surgical Knowledge Translation Research Group, Eris Lifesciences, Hikma, Lupin, Sun Pharmaceuticals, Medscape/Heart.Org, ProAdWise Communications, Radcliffe Cardiology, Servier, and the Corpus, has participated on a data safety monitoring board or advisory board for Cardialysis (MONITOR study) and Merck (VICTORIA trial), and works as company director for Global Clinical Trial Partners Ltd (GCTP) outside the submitted work; CM has received grants and non-financial support from several diagnostic companies during the conduct of the study and grants, personal fees, and non-financial support from several diagnostic companies outside the submitted work; JJ has received grants from Abbott Diagnostics, Innolife, and Novartis and consulting fees from Abbott, Jana Care, Novartis, Roche Diagnostics, Bristol-Myers Squibb, Janssen, and Prevencio; NLM has received grants from Siemens Healthineers, consulting fees from Roche Diagnostics and LumiraDx, and speaker fees from Abbott Diagnostics and Siemens Healthineers outside the submitted work; KKL, DD, and NLM are employed by the University of Edinburgh, which has filed a patent on the CoDE-HF score (patent reference: PCT/GB2021/051470); no other relationships or activities that could appear to have influenced the submitted work.

© Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY. No commercial re-use. See rights and permissions. Published by BMJ.

Figures

Fig 1

N-terminal pro-B-type natriuretic peptide (NT-proBNP) thresholds for acute heart failure. Top left: negative predictive values of NT-proBNP concentrations to rule out diagnosis of acute heart failure. Bottom left: cumulative proportion of patients presenting with suspected acute heart failure with NT-proBNP concentrations below each threshold. Top right: positive predictive values of NT-proBNP concentrations to rule in diagnosis of acute heart failure. Bottom right: cumulative proportion of patients presenting with suspected acute heart failure with NT-proBNP concentrations above each threshold

Fig 2

Diagnostic performance of guideline recommended N-terminal pro-B-type natriuretic peptide thresholds across patient subgroups: negative predictive value of threshold of 300 pg/mL. COPD=chronic obstructive pulmonary disease; eGFR=estimated glomerular filtration rate

Fig 3

Diagnostic performance of guideline recommended NT-proBNP thresholds across patient subgroups: positive predictive value of age specific thresholds across patient subgroups (450, 900, and 1800 pg/mL for 75 years, respectively). COPD=chronic obstructive pulmonary disease; eGFR=estimated glomerular filtration rate

Fig 4

Calibration of Collaboration for the Diagnosis and Evaluation of Heart Failure (CoDE-HF) score with observed proportion of patients with acute heart failure. Dashed line represents perfect calibration. Each point represents 100 patients. Top: calibration of CoDE-HF in patients with no previous heart failure. Bottom: calibration of CoDE-HF in patients with previous heart failure

Fig 5

Diagnostic performance of Collaboration for the Diagnosis and Evaluation of Heart Failure (CoDE-HF) score across patient subgroups: negative predictive value of CoDE-HF rule-out score of 4.7 in patients without previous heart failure across patient subgroups. CoDE-HF incorporates N-terminal pro-B-type natriuretic peptide concentrations as continuous measure and predefined simple objective clinical variables (age, estimated glomerular filtration rate (eGFR), haemoglobin, body mass index, heart rate, blood pressure, peripheral oedema, chronic obstructive pulmonary disease (COPD), and ischaemic heart disease) to provide individualised assessment of likelihood of diagnosis of acute heart failure

Fig 6

Diagnostic performance of Collaboration for the Diagnosis and Evaluation of Heart Failure (CoDE-HF) score across patient subgroups: positive predictive value of CoDE-HF rule-in score of 51.2 in patients without previous heart failure across patient subgroups. CoDE-HF incorporates NT-proBNP concentrations as continuous measure and predefined simple objective clinical variables (age, estimated glomerular filtration rate (eGFR), haemoglobin, body mass index, heart rate, blood pressure, peripheral oedema, chronic obstructive pulmonary disease (COPD), and ischaemic heart disease) to provide individualised assessment of likelihood of diagnosis of acute heart failure

Fig 7

Diagnostic performance of Collaboration for the Diagnosis and Evaluation of Heart Failure (CoDE-HF) score across patient subgroups: positive predictive value of CoDE-HF rule-in score of 84.5 in patients with previous heart failure across patient subgroups. CoDE-HF incorporates NT-proBNP concentrations as continuous measure and predefined simple objective clinical variables (age, estimated glomerular filtration rate (eGFR), haemoglobin, body mass index, heart rate, blood pressure, peripheral oedema, chronic obstructive pulmonary disease (COPD), and ischaemic heart disease) to provide individualised assessment of likelihood of diagnosis of acute heart failure

Fig 8

Diagnostic performance of Collaboration for the Diagnosis and Evaluation of Heart Failure (CoDE-HF) score in patients without previous heart failure. Top: negative and positive predictive values of CoDE-HF scores. Blue vertical dashed line represents target rule-out score of 4.7. Red vertical dashed line represents target rule-in score of 51.2. Bottom: density plot of CoDE-HF score in patients without previous heart failure. Target rule-out and rule-in scores identify 40.3% of patients as low probability and 28.0% as high probability respectively

Fig 9

Cumulative incidence of all cause mortality stratified by Collaboration for the Diagnosis and Evaluation of Heart Failure (CoDE-HF) probability group