Anatomically informed deep learning on contrast-enhanced cardiac magnetic resonance imaging for scar segmentation and clinical feature extraction

Dan M Popescu, Haley G Abramson, Rebecca Yu, Changxin Lai, Julie K Shade, Katherine C Wu, Mauro Maggioni, Natalia A Trayanova, Dan M Popescu, Haley G Abramson, Rebecca Yu, Changxin Lai, Julie K Shade, Katherine C Wu, Mauro Maggioni, Natalia A Trayanova

Abstract

Background: Visualizing fibrosis on cardiac magnetic resonance (CMR) imaging with contrast enhancement (late gadolinium enhancement; LGE) is paramount in characterizing disease progression and identifying arrhythmia substrates. Segmentation and fibrosis quantification from LGE-CMR is intensive, manual, and prone to interobserver variability. There is an unmet need for automated LGE-CMR image segmentation that ensures anatomical accuracy and seamless extraction of clinical features.

Objective: This study aimed to develop a novel deep learning solution for analysis of contrast-enhanced CMR images that produces anatomically accurate myocardium and scar/fibrosis segmentations and uses these to calculate features of clinical interest.

Methods: Data sources were 155 2-dimensional LGE-CMR patient scans (1124 slices) and 246 synthetic "LGE-like" scans (1360 slices) obtained from cine CMR using a novel style-transfer algorithm. We trained and tested a 3-stage neural network that identified the left ventricle (LV) region of interest (ROI), segmented ROI into viable myocardium and regions of enhancement, and postprocessed the segmentation results to enforce conforming to anatomical constraints. The segmentations were used to directly compute clinical features, such as LV volume and scar burden.

Results: Predicted LV and scar segmentations achieved 96% and 75% balanced accuracy, respectively, and 0.93 and 0.57 Dice coefficient when compared to trained expert segmentations. The mean scar burden difference between manual and predicted segmentations was 2%.

Conclusion: We developed and validated a deep neural network for automatic, anatomically accurate expert-level LGE- CMR myocardium and scar/fibrosis segmentation, allowing direct calculation of clinical measures. Given the training set heterogeneity, our approach could be extended to multiple imaging modalities and patient pathologies.

Keywords: CMR; Contrast-enhanced; Deep learning; Machine learning; Segmentation.

© 2021 Heart Rhythm Society.

Figures

Figure 1
Figure 1
Distribution of enhanced myocardium regions for ground truth data. The spatial distribution of regions of enhanced myocardium is shown for 3 regions of the ventricle: basal (left), middle (center), and apical (right). The x-axes capture septal vs lateral location and the y-axes capture anterior vs posterior. The heat map quantifies the proportion of enhanced myocardium located in the respective region, averaged over all patients.
Figure 2
Figure 2
Conversion process of cine images to “late gadolinium enhancement (LGE)-like” images. (1) The original cine image is cropped/padded to a square and contrast-limited adaptive histogram equalization (CLAHE) is applied. (2) Cine images are further transformed by first generating a pseudo-enhancement (“LGE-like” enhanced myocardium) mask. (3) Pseudo-scar mask is randomly eroded and Gaussian filters are applied to realistically blur the edges. (4) Speckle noise is added to the image to resemble LGE noise. (5) An LGE cardiac magnetic resonance scan is sampled at random and a histogram match is performed.
Figure 3
Figure 3
ACSNet architecture consisting of 3 interconnected deep learning subnetworks. A: The first residual U-Net (ResU-Net) is used to identify and crop around the left ventricle (LV). B: The second network uses the tightly cropped image from panel A and the LV segmentation to further segment the LV into viable and enhanced myocardium. C: The third network is a convolutional autoencoder trained to encode (compress) and decode myocardial segmentation masks. D: Segmentations from the training set are encoded using the third network to form a latent space. The space is modeled as a Gaussian mixture model (GMM) and conditional resampling is performed to populate the space with anatomically correct samples (black dots). Predicted segmentations are encoded and the nearest-neighbors algorithm is used to return a perturbed, anatomically correct version (green dot) of the original (red dot). GMM image adapted from source. Additional details are presented in Methods.
Figure 4
Figure 4
Left ventricle and myocardium segmentation network architecture. The left ventricle region of interest network (LV Net.) identifies the main region of interest. The second network (MYO Net.) segments the myocardium by differentiating between viable and nonviable tissue represented by each of the 2 outputs. The networks differ by the number of filters, input image size, and number of outputs as indicated.
Figure 5
Figure 5
Autoencoder network architecture. The anatomical autoencoder is used as a postprocessing step that takes in myocardial masks and uses a series of convolutions and downsampling layers to create a 16-dimensional latent representation (left side). The decoder piece of the autoencoder (right side) re-creates the original myocardial segmentation from the latent representation.
Figure 6
Figure 6
Left ventricle (LV) region of interest (ROI) and myocardium segmentation results by region. Histograms of per-slice Dice scores are shown for 3 regions of the heart (rows from top to bottom: basal, middle, and apical). Columns represent the left ventricle (LV ROI) segmentation (A–C) and myocardium (MYO) segmentation (D–F). The averages are shown as solid vertical lines, and the dotted lines represent the 5th and 95th percentiles. LGE-CMR = cardiac magnetic resonance imaging with late gadolinium enhancement.
Figure 7
Figure 7
Scar segmentation results. Segmentations of enhancement regions from the myocardium segmentation network represent gray zone (yellow) and scar (red). The first row shows the original scan, the middle row shows the ground truth scar and gray zone segmentations, and the bottom row shows the predicted segmentations. LGE-CMR = cardiac magnetic resonance imaging with late gadolinium enhancement.
Figure 8
Figure 8
Scar and left ventricle (LV) volumes. LV (A) and scar (B) volume error is computed as the absolute error normalized by each respective volume. Each point represents the error in LV volume of a single segmented patient scan. The solid black line shows the mean.

References

    1. Zipes D.P., Wellens H.J.J. Sudden cardiac death. Circulation. 1998;98:2334–2351.
    1. Kuruvilla S., Adenaw N., Katwal A.B., Lipinski M.J., Kramer C.M., Salerno M. Late gadolinium enhancement on cardiac magnetic resonance predicts adverse cardiovascular outcomes in nonischemic cardiomyopathy: a systematic review and meta-analysis. Circ Cardiovasc Imaging. 2014;7:250–258.
    1. Marco A.D., Anguera I., Schmitt M., et al. Late gadolinium enhancement and the risk for ventricular arrhythmias or sudden death in dilated cardiomyopathy: systematic review and meta-analysis. JACC Heart Fail. 2017;5:28–38.
    1. van der Burg A.E.B., Bax J.J., Boersma E., Pauwels E.K.J., Wall E.E., Schalij M.J. Impact of viability, ischemia, scar tissue, and revascularization on outcome after aborted sudden death. Circulation. 2003;108:1954–1959.
    1. Wu K.C. Sudden cardiac death substrate imaged by MRI: from investigational tool to clinical applications. Circ Cardiovasc Imaging. 2017;10:7.
    1. Roes S.D., Borleffs C.J.W., Geest R.J., et al. Infarct tissue heterogeneity assessed with contrast-enhanced MRI predicts spontaneous ventricular arrhythmia in patients with ischemic cardiomyopathy and implantable cardioverter-defibrillator. Circ Cardiovasc Imaging. 2009;2:183–190.
    1. Scott P.A., Morgan J.M., Carroll N., et al. The extent of left ventricular scar quantified by late gadolinium enhancement MRI is associated with spontaneous ventricular arrhythmias in patients with coronary artery disease and implantable cardioverter-defibrillators. Circ Arrhythm Electrophysiol. 2011;4:324–330.
    1. AlJaroudi W.A., Flamm S.D., Saliba W., Wilkoff B.L., Kwon D. Role of CMR imaging in risk stratification for sudden cardiac death. JACC Cardiovasc Imaging. 2013;6:392–406.
    1. Kim R.J., Fieno D.S., Parrish T.B., et al. Relationship of MRI delayed contrast enhancement to irreversible injury, infarct age, and contractile function. Circulation. 1999;100:1992–2002.
    1. Disertori M., Rigoni M., Pace N., et al. Myocardial fibrosis assessment by LGE is a powerful predictor of ventricular tachyarrhythmias in ischemic and nonischemic LV dysfunction: a meta-analysis. JACC Cardiovasc Imaging. 2016;9:1046–1055.
    1. Mordi I., Jhund P.S., Gardner R.S., et al. LGE and NT-ProBNP identify low risk of death or arrhythmic events in patients with primary prevention ICDs. JACC Cardiovasc Imaging. 2014;7:561–569.
    1. Cain M.A., Metzl M.D., Patel A.R., et al. Cardiac sarcoidosis detected by late gadolinium enhancement and prevalence of atrial arrhythmias. Am J Cardiol. 2014;113:1556–1560.
    1. Marra M.P., De Lazzari M., Zorzi A., et al. Impact of the presence and amount of myocardial fibrosis by cardiac magnetic resonance on arrhythmic outcome and sudden cardiac death in nonischemic dilated cardiomyopathy. Heart Rhythm. 2014;11:856–863.
    1. Balaban G., Halliday B.P., Porter B., et al. Late-gadolinium enhancement interface area and electrophysiological simulations predict arrhythmic events in patients with nonischemic dilated cardiomyopathy. JACC Clin Electrophysiol. 2021;7:238–249.
    1. Kucukseymen S., Yavin H., Barkagan M., et al. Discordance in scar detection between electroanatomical mapping and cardiac MRI in an infarct swine model. JACC Clin Electrophysiol. 2020;6:1452–1464.
    1. Arevalo H.J., Vadakkumpadan F., Guallar E., et al. Arrhythmia risk stratification of patients after myocardial infarction using personalized heart models. Nat Commun. 2016;7:11437.
    1. Ashikaga H., Arevalo H., Vadakkumpadan F., et al. Feasibility of image-based simulation to estimate ablation target in human ventricular arrhythmia. Heart Rhythm. 2013;8:1109–1116.
    1. Bello G.A., Dawes T.J.W., Duan J., et al. Deep-learning cardiac motion analysis for human survival prediction. Nat Mach Intell. 2019;1:95–104.
    1. Zheng Q., Delingette H., Duchateau N., Ayache N. 3D consistent and robust segmentation of cardiac images by deep learning with spatial propagation. IEEE Trans Med Imaging. 2018:1.
    1. Yue Q., Luo X., Ye Q., Xu L., Zhuang X. Cardiac segmentation from LGE MRI using deep neural network incorporating shape and spatial priors. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. 2019;11765 LNCS:559–567.
    1. Roth H., Zhu W., Yang D., Xu Z., Xu D. In: Statistical Atlases and Computational Models of the Heart. Multi-Sequence CMR Segmentation, CRT-EPiggy and LV Full Quantification Challenges. Pop M., Sermesant M., Camara O., et al., editors. Springer International Publishing; 2020. Cardiac segmentation of LGE MRI with noisy labels; pp. 228–236.
    1. Chen C., Ouyang C., Tarroni G., et al. International Workshop on Statistical Atlases and Computational Models of the Heart. Springer; 2019. Unsupervised multi-modal style transfer for cardiac MR segmentation; pp. 209–219.
    1. Campello V., Martín-Isla C., Izquierdo C., Petersen S.E., Ballester M.A.G., Lekadir K. International Workshop on Statistical Atlases and Computational Models of the Heart. Springer; 2019. Combining multi-sequence and synthetic images for improved segmentation of late gadolinium enhancement cardiac MRI; pp. 290–299.
    1. Zabihollahy F., Rajchl M., White J.A., Ukwatta E. Fully automated segmentation of left ventricular scar from 3D late gadolinium enhancement magnetic resonance imaging using a cascaded multi-planar U-net (CMPU-net) Med Phys. 2020;47:1645–1655.
    1. Moccia S., Banali R., Martini C., et al. Development and testing of a deep learning-based strategy for scar segmentation on CMR-LGE images. MAGMA. 2019;32:187–195.
    1. Fahmy A.S., Neisius U., Chan R.H., et al. Three-dimensional deep convolutional neural networks for automated myocardial scar quantification in hypertrophic cardiomyopathy: a multicenter multivendor study. Radiology. 2019;294:190737.
    1. Fahmy A.S., Rowin E., Chan R.H., Manning W.J., Maron M.S., Nezafat R. Improved quantification of myocardium scar in late gadolinium enhancement images: deep learning based image fusion approach. J Magn Reson Imaging. 2021;54:303–312.
    1. Wu K.C., Gerstenblith G., Guallar E., et al. Combined cardiac magnetic resonance imaging and C-reactive protein levels identify a cohort at low risk for defibrillator firings and death. Circ Cardiovasc Imaging. 2012;5:178–186.
    1. Bernard O., Lalande A., Zotti C., et al. Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE Trans Med Imaging. 2018;37:2514–2525.
    1. Radau P., Lu Y., Connelly K., Paul G., Dick A., Wright G. Evaluation framework for algorithms segmenting short axis cardiac MRI. The MIDAS Journal. 2009
    1. Pizer S.M., Amburn E.P., Austin J.D., et al. Adaptive histogram equalization and its variations. Computer Vision, Graphics, and Image Processing. 1987;39:355–368.
    1. Ronneberger O., Fischer P., Brox T. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Navab N., Hornegger J., Wells W., Frangi A., editors. Springer; 2015. U-net: convolutional networks for biomedical image segmentation; pp. 234–241.
    1. Salehi S.S.M., Erdogmus D., Gholipour A. International Workshop on Machine Learning in Medical Imaging. Springer; 2017. Tversky loss function for image segmentation using 3D fully convolutional deep networks; pp. 379–387.
    1. Kingma D., Ba J. Adam: a method for stochastic optimization. Int. Conf. on Learn. Represent. 2014
    1. Chollet François, others. Keras 2015. Retrieved from.
    1. Abadi M., Agarwal A., Barham P., et al. TensorFlow: large-scale machine learning on heterogeneous systems. 2015. Software available from.
    1. Birsan T., Tiba D. IFIP Conference on System Modeling and Optimization. Springer; 2005. One hundred years since the introduction of the set distance by Dimitrie Pompeiu; pp. 35–39.
    1. Schmidt A., Azevedo C.F., Cheng A., et al. Infarct tissue heterogeneity by magnetic resonance imaging identifies enhanced cardiac arrhythmia susceptibility in patients with left ventricular dysfunction. Circulation. 2007;115:2006–2014.
    1. Jablonowski R., Chaudhry U., Pals J., et al. Cardiovascular magnetic resonance to predict appropriate implantable cardioverter defibrillator therapy in ischemic and nonischemic cardiomyopathy patients using late gadolinium enhancement border zone: comparison of four analysis methods. Circ Cardiovasc Imaging. 2017;10
    1. Zhuang X. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2016. Multivariate mixture model for cardiac segmentation from multi-sequence MRI; pp. 581–588.
    1. Zhuang X. Multivariate mixture model for myocardial segmentation combining multi-source images. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2019;41:2933–2946.
    1. Klem I., Weinsaft J.W., Bahnson T.D., et al. Assessment of myocardial scarring improves risk stratification in patients evaluated for cardiac defibrillator implantation. J Am Coll Cardiol. 2012;60:408–420.
    1. Prakosa A., Arevalo H.J., Deng D., et al. Personalized virtual-heart technology for guiding the ablation of infarct-related ventricular tachycardia. Nat Biomed Eng. 2018;2:732–740.
    1. Ballistipedia Bivariate. 2020. Retrieved from.
    1. Boyle P.M., Zghaib T., Zahid S., et al. Computationally guided personalized targeted ablation of persistent atrial fibrillation. Nat Biomed Eng. 2019;3:870–879.
    1. Zhuang X., Xu J., Luo X., et al. Cardiac segmentation on late gadolinium enhancement MRI: a benchmarkstudy from multi-sequence cardiac MR segmentation challenge. arXiv preprint arXiv. 2006
    1. Painchaud N., Skandarani Y., Judge T., Olivier B., Lalande A., Jodoin P.M. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2019. CardiacMRI segmentation with strong anatomical guarantees; pp. 632–640.
    1. Larrazabal A.J., Martínez C., Glocker B., Ferrante E. Post-DAE: Anatomically plausible segmentation viapost-processing with denoising autoencoders. IEEE Trans Med Imaging. 2020;39:3813–3820.
    1. Akaike H. A new look at the statistical model identification. IEEE Trans Autom Control. 1974;19:716–723.

Source: PubMed

Sponsoren und Mitarbeiter

Krankheiten

Drogeninterventionen

3
Abonnieren