Deep learning for cardiovascular medicine: a practical primer

Abstract

Deep learning (DL) is a branch of machine learning (ML) showing increasing promise in medicine, to assist in data classification, novel disease phenotyping and complex decision making. Deep learning is a form of ML typically implemented via multi-layered neural networks. Deep learning has accelerated by recent advances in computer hardware and algorithms and is increasingly applied in e-commerce, finance, and voice and image recognition to learn and classify complex datasets. The current medical literature shows both strengths and limitations of DL. Strengths of DL include its ability to automate medical image interpretation, enhance clinical decision-making, identify novel phenotypes, and select better treatment pathways in complex diseases. Deep learning may be well-suited to cardiovascular medicine in which haemodynamic and electrophysiological indices are increasingly captured on a continuous basis by wearable devices as well as image segmentation in cardiac imaging. However, DL also has significant weaknesses including difficulties in interpreting its models (the 'black-box' criticism), its need for extensive adjudicated ('labelled') data in training, lack of standardization in design, lack of data-efficiency in training, limited applicability to clinical trials, and other factors. Thus, the optimal clinical application of DL requires careful formulation of solvable problems, selection of most appropriate DL algorithms and data, and balanced interpretation of results. This review synthesizes the current state of DL for cardiovascular clinicians and investigators, and provides technical context to appreciate the promise, pitfalls, near-term challenges, and opportunities for this exciting new area.

Keywords: Artificial intelligence; Big data; Cardiovascular medicine; Deep learning; Precision medicine.

Published on behalf of the European Society of Cardiology. All rights reserved. © The Author(s) 2019. For permissions, please email: journals.permissions@oup.com.

Figures

Figure 1

Relationship of deep learning to clinical and translational medicine. Venn diagrams show deep learning as one type of machine learning, within the scope of artificial intelligence. Statistical methods are applied across clinical and translational science, and the form known as statistical learning theory has overlap with machine learning. Automated decision making is often used in clinical practice. Deep learning may extend statistical approaches in some key areas by analysing large multivariate datasets, which often show complex interactions, in which simple hypotheses are difficult to formulate. Deep learning has been successful in medical image recognition (e.g. electrocardiogram, echocardiogram, and magnetic resonance imaging) and holds the promise of enhancing clinic decision making.

Figure 2

Types of machine learning in cardiovascular science. (A) Supervised learning uses inputs (e.g. electrocardiograms) each with a label (‘ground truth’, and a diagnosis of atrial fibrillation or not atrial fibrillation). Machines are iteratively ‘trained’, using direct feedback for multiple inputs, until their output matches the ground truth. Trained machines can then classify unknown (test) electrocardiograms. One misclassification is shown. (B) Unsupervised learning uses unlabelled data, ideally in large quantities, to identify novel patterns. In this example, QRS indices identified novel phenotypes (‘clusters’) for hypertrophic cardiomyopathy with distinct outcomes (Ref: Lyon et al.26). (C) Reinforcement learning uses models developed from psychological training applied to gaming, but infrequently to medicine. An agent, e.g. a clinical decision-making tool, performs an action At (e.g. which therapy for non-ST-segment elevation myocardial infarction best reduces mortality? (1) non-invasive, (2) early invasive, and (3) mixed) that alters the environment (e.g. biomarker response or patient outcomes). A Rt reward is then given (e.g. higher survival rate) that alters the state St. This process is iterated with the intention of moving State St+1 closer to the desired goal (i.e. improved outcomes).

Figure 3

Neural network design to classify atrial fibrillation from the electrocardiogram. Continuous electrocardiogram voltage points (red dots, arrows) are fed to ‘input neurons’ (x0, x1, x2, …, xm), which are coded as software objects. Hidden neurons within this three-layer network (h0, h1, h2, …, hn) connect input and output layer neurons (here, two neurons) by numerical weights (w). Deep learning typically uses multiple hidden layers, as shown here. The output indicates atrial fibrillation (y1; correct, red) or non-atrial fibrillation (y0). If the output is correct for that electrocardiogram input, weights are strengthened; else they are reduced. This process is iterated during training on multiple input electrocardiograms. The trained network can then be tested on new (unseen) electrocardiograms. Other designs could accept categorical variables (age, gender) or mixed data types.

Figure 4

Impact of deep learning design on learning: effect of learning rate. (A) Efficient learning. Cost function (network error) gradually descends (‘Gradient descent’) to achieve the optimal point (called local minimum) as a function of weight. (B) Learning rate is too high, so that the cost function overshoots the minimum and oscillates. This network design may not be trained effectively for this problem. (C) Gradient descent examining two variables on cost function simultaneously.

Figure 5

Classifying complex data. (A) Transforming data to enable linear separation of non-linearly separable raw data. Raw non-linear data are transformed by mapping functions that may include time, frequency, or other operations. This projects them into higher-dimensional parameters space in which they are now linearly separable. One example is classifying patients with heart failure with preserved ejection fraction whose response to beta-blockers may vary due to obesity, atrial fibrillation, left ventricular hypertrophy, diabetes, or other factors. Data transformation to a higher-dimensional space now enables a simple partitioning process. (B) Bias–variance tradeoff. Model with high bias (straight line), when a straight line could not classify appropriately (here, between atrial fibrillation and normal sinus rhythm) in both training dataset (5.B.a) and testing dataset (5.B.b). This leads to prediction errors on other datasets (low variance − frequent errors). In contrast, model with low bias (i.e. due to overtraining) when data is fitted well in training set (5.B.c), but not in testing set (5.B.d), leading to reduced generalization (high variability due to difference between training and validation sets).

Take home figure

Deep learning process flow for cardiovascular medicine. Deep learning has the ability to produce actionable clinical information from diverse datasets. Such data may span (i) comprehensive, traditional clinical data; (ii) non-traditional ‘real-world’ data such as near-continuous streams from wearable devices but also questionnaires or online forms. The deep learning process flow commences with designing the most appropriate model. Deep learning is usually implemented by deep neural networks with convolutional layers defined by specific parameters (e.g. max pooling, activation function, and learning rate). Several algorithms traditionally applied in machine learning (i.e. SVM, RF, KNN, RNN, AE, GAN) can be combined to address complex problems. Data is selected and pre-processed (curated), and missing elements are imputed. Training proceeds until the deep learning machine converges at acceptable accuracy. The deep learning machine is then ready to be applied to unseen test data.