Fast and accurate view classification of echocardiograms using deep learning
Ali Madani, Ramy Arnaout, Mohammad Mofrad, Rima Arnaout, Ali Madani, Ramy Arnaout, Mohammad Mofrad, Rima Arnaout
Abstract
Echocardiography is essential to cardiology. However, the need for human interpretation has limited echocardiography's full potential for precision medicine. Deep learning is an emerging tool for analyzing images but has not yet been widely applied to echocardiograms, partly due to their complex multi-view format. The essential first step toward comprehensive computer-assisted echocardiographic interpretation is determining whether computers can learn to recognize these views. We trained a convolutional neural network to simultaneously classify 15 standard views (12 video, 3 still), based on labeled still images and videos from 267 transthoracic echocardiograms that captured a range of real-world clinical variation. Our model classified among 12 video views with 97.8% overall test accuracy without overfitting. Even on single low-resolution images, accuracy among 15 views was 91.7% vs. 70.2-84.0% for board-certified echocardiographers. Data visualization experiments showed that the model recognizes similarities among related views and classifies using clinically relevant image features. Our results provide a foundation for artificial intelligence-assisted echocardiographic interpretation.
Conflict of interest statement
Competing interests: The authors declare no competing financial interests.
Figures
References
- Karpathy, A. The Unreasonable Effectiveness of Recurrent Neural Networks. (2015).
- Esteva A, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542:115–118. doi: 10.1038/nature21056.
- Gulshan V, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316:2402–2410. doi: 10.1001/jama.2016.17216.
- Litjens, G. et al. A survey on deep learning in medical image analysis. eprint at (2017).
- Douglas PS, et al. ACCF/ASE/AHA/ASNC/HFSA/HRS/SCAI/SCCM/SCCT/SCMR 2011 appropriate use criteria for echocardiography. J. Am. Coll. Cardiol. 2011;57:1126–1166. doi: 10.1016/j.jacc.2010.11.002.
- Wharton G, et al. A minimum dataset for a standard adult transthoracic echocardiogram: a guideline protocol from the British Society of Echocardiography. Echo Res. Pract. 2015;2:G9–G24. doi: 10.1530/ERP-14-0079.
- Khamis H, et al. Automatic apical view classification of echocardiograms using a discriminative learning dictionary. Med. Image Anal. 2017;36:15–21. doi: 10.1016/j.media.2016.10.007.
- Knackstedt C, et al. Fully automated versus standard tracking of left ventricular ejection fraction and longitudinal strain: the FAST-EFs multicenter study. J. Am. Coll. Cardiol. 2015;66:1456–1466. doi: 10.1016/j.jacc.2015.07.052.
- Narula S, et al. Machine-learning algorithms to automate morphological and functional assessments in 2D echocardiography. J. Am. Coll. Cardiol. 2016;68:2287–2295. doi: 10.1016/j.jacc.2016.08.062.
- Park J, Zhou SK, Simopoulos C, Comaniciu D. AutoGate: fast and automatic Doppler gate localization in B-mode echocardiogram. Med. Image Comput. Comput. Assist. Interv. 2008;11:230–237.
- Sengupta, P. P. et al. Cognitive machine-learning algorithm for cardiac imaging: a pilot study for differentiating constrictive pericarditis from restrictive cardiomyopathy. Circ. Cardiovasc. Imaging9, 10.1161/CIRCIMAGING.115.004330 (2016).
- Gao XH, Li W, Loomes M, Wang LY. A fused deep learning architecture for viewpoint classification of echocardiography. Inf. Fusion. 2017;36:103–113. doi: 10.1016/j.inffus.2016.11.007.
- Penatti OA, et al. Mid-level image representations for real-time heart view plane classification of echocardiograms. Comput. Biol. Med. 2015;66:66–81. doi: 10.1016/j.compbiomed.2015.08.004.
- Abadi, M. et al. TensorFlow: Large-scale machine learning on heterogeneous systems (2015).
- Keras (GitHub, 2015).
- Python Language Reference, version 2.7. Python Software Foundation (2017).
- Russakovsky O, et al. ImageNet large scale visual recognition challenge. Int. J. Comput. Vision. 2015;115:211–252. doi: 10.1007/s11263-015-0816-y.
- van der Maaten L, Hinton G. Visualizing data using t-SNE. J. Mach. Learn. Res. 2008;9:2579–2605.
Source: PubMed