Validation of the myocardial-ischaemic-injury-index machine learning algorithm to guide the diagnosis of myocardial infarction in a heterogenous population: a prespecified exploratory analysis

Dimitrios Doudesis, Kuan Ken Lee, Jason Yang, Ryan Wereski, Anoop S V Shah, Athanasios Tsanas, Atul Anand, John W Pickering, Martin P Than, Nicholas L Mills, High-STEACS Investigators, Nicholas L Mills, Fiona E Strachan, Christopher Tuck, Anoop Sv Shah, Atul Anand, Andrew R Chapman, Amy V Ferry, Kuan Ken Lee, Dimitrios Doudesis, Anda Bularga, Ryan Wereski, Caelan Taggart, Matthew Th Lowry, Filip Mendusic, Dorien M Kimenai, Dennis Sandeman, Philip D Adamson, Catherine L Stables, Catalina A Vallejos, Athanasios Tsanas, Lucy Marshall, Stacey D Stewart, Takeshi Fujisawa, Mischa Hautvast, Jean McPherson, Lynn McKinlay, Ian Ford, David E Newby, Keith Aa Fox, Colin Berry, Simon Walker, Christopher J Weir, Alasdair Gray, Paul O Collinson, Fred S Apple, Alan Reid, Anne Cruikshank, Iain Findlay, Shannon Amoils, David A McAllister, Donogh Maguire, Jennifer Stevens, John Norrie, Jack Pm Andrews, Alastair Moss, Mohamed S Anwar, John Hung, Jonathan Malo, Colin Fischbacher, Bernard L Croal, Stephen J Leslie, Catriona Keerie, Richard A Parker, Allan Walker, Ronnie Harkess, Tony Wackett, Roma Armstrong, Laura Stirling, Claire MacDonald, Imran Sadat, Frank Finlay, Heather Charles, Pamela Linksted, Stephen Young, Bill Alexander, Chris Duncan, Dimitrios Doudesis, Kuan Ken Lee, Jason Yang, Ryan Wereski, Anoop S V Shah, Athanasios Tsanas, Atul Anand, John W Pickering, Martin P Than, Nicholas L Mills, High-STEACS Investigators, Nicholas L Mills, Fiona E Strachan, Christopher Tuck, Anoop Sv Shah, Atul Anand, Andrew R Chapman, Amy V Ferry, Kuan Ken Lee, Dimitrios Doudesis, Anda Bularga, Ryan Wereski, Caelan Taggart, Matthew Th Lowry, Filip Mendusic, Dorien M Kimenai, Dennis Sandeman, Philip D Adamson, Catherine L Stables, Catalina A Vallejos, Athanasios Tsanas, Lucy Marshall, Stacey D Stewart, Takeshi Fujisawa, Mischa Hautvast, Jean McPherson, Lynn McKinlay, Ian Ford, David E Newby, Keith Aa Fox, Colin Berry, Simon Walker, Christopher J Weir, Alasdair Gray, Paul O Collinson, Fred S Apple, Alan Reid, Anne Cruikshank, Iain Findlay, Shannon Amoils, David A McAllister, Donogh Maguire, Jennifer Stevens, John Norrie, Jack Pm Andrews, Alastair Moss, Mohamed S Anwar, John Hung, Jonathan Malo, Colin Fischbacher, Bernard L Croal, Stephen J Leslie, Catriona Keerie, Richard A Parker, Allan Walker, Ronnie Harkess, Tony Wackett, Roma Armstrong, Laura Stirling, Claire MacDonald, Imran Sadat, Frank Finlay, Heather Charles, Pamela Linksted, Stephen Young, Bill Alexander, Chris Duncan

Abstract

Background: Diagnostic pathways for myocardial infarction rely on fixed troponin thresholds, which do not recognise that troponin varies by age, sex, and time within individuals. To overcome this limitation, we recently introduced a machine learning algorithm that predicts the likelihood of myocardial infarction. Our aim was to evaluate whether this algorithm performs well in routine clinical practice and predicts subsequent events.

Methods: The myocardial-ischaemic-injury-index (MI3) algorithm was validated in a prespecified exploratory analysis using data from a multi-centre randomised trial done in Scotland, UK that included consecutive patients with suspected acute coronary syndrome undergoing serial high-sensitivity cardiac troponin I measurement. Patients with ST-segment elevation myocardial infarction were excluded. MI3 incorporates age, sex, and two troponin measurements to compute a value (0-100) reflecting an individual's likelihood of myocardial infarction during the index visit and estimates diagnostic performance metrics (including area under the receiver-operating-characteristic curve, and the sensitivity, specificity, negative predictive value, and positive predictive value) at the computed score. Model performance for an index diagnosis of myocardial infarction (type 1 or type 4b), and for subsequent myocardial infarction or cardiovascular death at 1 year was determined using the previously defined low-probability threshold (1·6) and high-probability MI3 threshold (49·7). The trial is registered with ClinicalTrials.gov, NCT01852123.

Findings: In total, 20 761 patients (64 years [SD 16], 9597 [46%] women) enrolled between June 10, 2013, and March 3, 2016, were included from the High-STEACS trial cohort, of whom 3272 (15·8%) had myocardial infarction. MI3 had an area under the receiver-operating-characteristic curve of 0·949 (95% CI 0·946-0·952) identifying 12 983 (62·5%) patients as low-probability for myocardial infarction at the pre-specified threshold (MI3 score <1·6; sensitivity 99·3% [95% CI 99·0-99·6], negative predictive value 99·8% [99·8-99·9]), and 2961 (14·3%) as high-probability at the pre-specified threshold (MI3 score ≥49·7; specificity 95·0% [94·6-95·3], positive predictive value 70·4% [68·7-72·0]). At 1 year, subsequent myocardial infarction or cardiovascular death occurred more often in high-probability patients than low-probability patients (520 [17·6%] of 2961 vs 197 [1·5%] of 12 983], p<0·0001).

Interpretation: In consecutive patients undergoing serial cardiac troponin measurement for suspected acute coronary syndrome, the MI3 algorithm accurately estimated the likelihood of myocardial infarction and predicted subsequent adverse cardiovascular events. By providing individual probabilities the MI3 algorithm could improve the diagnosis and assessment of risk in patients with suspected acute coronary syndrome.

Funding: Medical Research Council, British Heart Foundation, National Institute for Health Research, and NHSX.

Conflict of interest statement

Declaration of interests NLM has received honoraria or consultancy from Abbott Diagnostics, Roche Diagnostics, Siemens Healthineers, and LumiraDx. KKL has received honoraria from Abbott Diagnostics. ASVS's institution (the University of Edinburgh) has received speaker fees from Abbott Diagnostics. JWP has undertaken consultancy for Abbott Diagnostics. MPT has received consulting fees, honoraria, or payment from Abbott, Roche, and Siemens; funding for clinical research from Radiometer; and participated on a data safety monitoring board or an advisory board for Abbott, Radiometer, Roche, and Siemens. All other authors declare no competing interests.

Copyright © 2022 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 license. Published by Elsevier Ltd.. All rights reserved.

Figures

Figure 1
Figure 1
Flow diagram of the analysis population
Figure 2
Figure 2
Overall diagnostic performance of the MI3 algorithm (A) Receiver-operating-characteristic curve illustrating discrimination of the MI3 algorithm for type 1 or type 4b myocardial infarction. (B) Calibration of the MI3 algorithm with the observed proportion of patients with type 1 or type 4b myocardial infarction. The dashed line represents perfect calibration. Each point represents 100 patients. (C) Precision-recall curve illustrating discrimination of the MI3 algorithm for type 1 or type 4b myocardial infarction. MI3=myocardial-ischaemic-injury-index.
Figure 3
Figure 3
Performance of MI3 at pre-defined thresholds MI3=myocardial-ischaemic-injury-index. NPV=negative predictive value. PPV=positive predictive value.
Figure 4
Figure 4
Cumulative incidence of myocardial infarction or cardiovascular death over 1 year (A) and death from any cause stratified by MI3 score (B) Low probability was an MI3 score of less than 1·6, intermediate probability was an MI3 score of 1·6 to 49·6, and high probability was an MI3 score of 49·7 or more. Log-rank between groups for both endpoints, p<0·0001.

References

    1. Sanchis-Gomar F, Perez-Quilis C, Leischik R, Lucia A. Epidemiology of coronary heart disease and acute coronary syndrome. Ann Transl Med. 2016;4:256.
    1. Collet JP, Thiele H, Barbato E, et al. The ‘Ten Commandments’ for the 2020 ESC Guidelines for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation. Eur Heart J. 2020;41:3495–3497.
    1. Goodacre S, Cross E, Arnold J, Angelini K, Capewell S, Nicholl J. The health care burden of acute chest pain. Heart. 2005;91:229–230.
    1. Babuin L, Jaffe AS. Troponin: the biomarker of choice for the detection of cardiac injury. CMAJ. 2005;173:1191–1202.
    1. Body R, Carley S, McDowell G, et al. Rapid exclusion of acute myocardial infarction in patients with undetectable troponin using a high-sensitivity assay. J Am Coll Cardiol. 2011;58:1332–1339.
    1. Body R, Carlton E, Sperrin M, et al. Troponin-only Manchester Acute Coronary Syndromes (T-MACS) decision aid: single biomarker re-derivation and external validation in three cohorts. Emerg Med J. 2017;34:349–356.
    1. Reichlin T, Schindler C, Drexler B, et al. One-hour rule-out and rule-in of acute myocardial infarction using high-sensitivity cardiac troponin T. Arch Intern Med. 2012;172:1211–1218.
    1. Than M, Flaws D, Sanders S, et al. Development and validation of the Emergency Department Assessment of Chest pain Score and 2 h accelerated diagnostic protocol. Emerg Med Australas. 2014;26:34–44.
    1. Chapman AR, Anand A, Boeddinghaus J, et al. Comparison of the efficacy and safety of early rule-out pathways for acute myocardial infarction. Circulation. 2017;135:1586–1596.
    1. Thygesen K, Alpert JS, Jaffe AS, Simoons ML, Chaitman BR, White HD. Third universal definition of myocardial infarction. Circulation. 2012;126:2020–2035.
    1. Than MP, Pickering JW, Sandoval Y, et al. Machine learning to predict the likelihood of acute myocardial infarction. Circulation. 2019;140:899–909.
    1. Shah ASV, Anand A, Strachan FE, et al. High-sensitivity troponin in the evaluation of patients with suspected acute coronary syndrome: a stepped-wedge, cluster-randomised controlled trial. Lancet. 2018;392:919–928.
    1. Scottish Intercollegiate Guidelines Network Acute coronary syndromes. 2013.
    1. Shah AS, Anand A, Sandoval Y, et al. High-sensitivity cardiac troponin I at presentation in patients with suspected acute coronary syndrome: a cohort study. Lancet. 2015;386:2481–2488.
    1. Mueller-Hennessen M, Lindahl B, Giannitsis E, et al. Diagnostic and prognostic implications using age- and gender-specific cut-offs for high-sensitivity cardiac troponin T — sub-analysis from the TRAPID-AMI study. Int J Cardiol. 2016;209:26–33.
    1. Lee KK, Ferry AV, Anand A, et al. Sex-specific thresholds of high-sensitivity troponin in patients with suspected acute coronary syndrome. J Am Coll Cardiol. 2019;74:2032–2043.
    1. Shah ASV, Sandoval Y, Noaman A, et al. Patient selection for high sensitivity cardiac troponin testing and diagnosis of myocardial infarction: prospective cohort study. BMJ. 2017;359
    1. Eggers KM, Ellenius J, Dellborg M, et al. Artificial neural network algorithms for early diagnosis of acute myocardial infarction and prediction of infarct size in chest pain patients. Int J Cardiol. 2007;114:366–374.
    1. Green M, Björk J, Forberg J, Ekelund U, Edenbrandt L, Ohlsson M. Comparison between neural networks and multiple logistic regression to predict acute coronary syndrome in the emergency room. Artif Intell Med. 2006;38:305–318.
    1. Rahimi K, Bennett D, Conrad N, et al. Risk prediction in patients with heart failure: a systematic review and analysis. JACC Heart Fail. 2014;2:440–446.
    1. Wessler BS, Lai Yh L, Kramer W, et al. Clinical prediction models for cardiovascular disease: tufts predictive analytics and comparative effectiveness clinical prediction model database. Circ Cardiovasc Qual Outcomes. 2015;8:368–375.
    1. Califf RM, Pencina MJ. Predictive models in heart failure: who cares? Circ Heart Fail. 2013;6:877–878.
    1. Damen JA, Hooft L, Schuit E, et al. Prediction models for cardiovascular disease risk in the general population: systematic review. BMJ. 2016;353
    1. Siontis GC, Tzoulaki I, Castaldi PJ, Ioannidis JP. External validation of new risk prediction models is infrequent and reveals worse prognostic discrimination. J Clin Epidemiol. 2015;68:25–34.
    1. Thygesen K, Alpert JS, Jaffe AS, et al. Fourth universal definition of myocardial infarction (2018) Circulation. 2018;138:e618–e651.
    1. Anand A, Shah ASV, Beshiri A, Jaffe AS, Mills NL. Global adoption of high-sensitivity cardiac troponins and the universal definition of myocardial infarction. Clin Chem. 2019;65:484–489.
    1. Roffi M, Patrono C, Collet JP, et al. 2015 ESC Guidelines for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation: task force for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation of the European Society of Cardiology (ESC) Eur Heart J. 2016;37:267–315.
    1. Neumann JT, Twerenbold R, Ojeda F, et al. Application of high-sensitivity troponin in suspected myocardial infarction. N Engl J Med. 2019;380:2529–2540.
    1. Lee KK, Noaman A, Vaswani A, et al. Prevalence, determinants, and clinical associations of high-sensitivity cardiac troponin in patients attending emergency departments. Am J Med. 2019;132:e8–21.
    1. Wereski R, Kimenai DM, Taggart C, et al. Cardiac troponin thresholds and kinetics to differentiate myocardial injury and myocardial infarction. Circulation. 2021;144:528–538.

Source: PubMed

Sponsors et collaborateurs

Les conditions médicales

Interventions en matière de drogue

3
S'abonner