Thyroid Nodule Classification in Ultrasound Images by Fine-Tuning Deep Convolutional Neural Network

Jianning Chi, Ekta Walia, Paul Babyn, Jimmy Wang, Gary Groot, Mark Eramian, Jianning Chi, Ekta Walia, Paul Babyn, Jimmy Wang, Gary Groot, Mark Eramian

Abstract

With many thyroid nodules being incidentally detected, it is important to identify as many malignant nodules as possible while excluding those that are highly likely to be benign from fine needle aspiration (FNA) biopsies or surgeries. This paper presents a computer-aided diagnosis (CAD) system for classifying thyroid nodules in ultrasound images. We use deep learning approach to extract features from thyroid ultrasound images. Ultrasound images are pre-processed to calibrate their scale and remove the artifacts. A pre-trained GoogLeNet model is then fine-tuned using the pre-processed image samples which leads to superior feature extraction. The extracted features of the thyroid ultrasound images are sent to a Cost-sensitive Random Forest classifier to classify the images into "malignant" and "benign" cases. The experimental results show the proposed fine-tuned GoogLeNet model achieves excellent classification performance, attaining 98.29% classification accuracy, 99.10% sensitivity and 93.90% specificity for the images in an open access database (Pedraza et al. 16), while 96.34% classification accuracy, 86% sensitivity and 99% specificity for the images in our local health region database.

Keywords: Computer vision; Convolutional neural network; Deep learning; Fine-tuning; Machine learning; Thyroid nodules; Ultrasonography.

Conflict of interest statement

Funding

This research was funded through a Collaborative Innovation Development Grant from the Saskatchewan Health Research Foundation.

Figures

Fig. 1
Fig. 1
Example of artifacts made by radiologist on the thyroid ultrasound image. a Ultrasound image with artifacts covering the textures. b Details of how the artifact covers the ultrasound textures in the image, bounded by the red rectangle in a
Fig. 2
Fig. 2
The process of the thyroid images classification based on fine-tuned GoogLeNet network
Fig. 3
Fig. 3
Detection of pixels deemed to be ticks by intensity thresholding. a Input thyroid ultrasound image. b Thirty percent right region of the input image where the thresholding was applied. c Binary image showing the pixels deemed likely to be ticks, and red line in d represents the column containing the tick bar
Fig. 4
Fig. 4
Example of removing artifacts from the thyroid image. a Original thyroid image with artifacts on the textures. b Thyroid image after removing the artifacts by the thresholding method. c Thyroid image after restoring the artifact-removed regions with textures similar to neighbourhood
Fig. 5
Fig. 5
The process of fine-tuning GoogLeNet. The new sample images from the target domain are sent to the pre-trained GoogLeNet, changed the three classification layers and run the GoogLeNet, and then the parameters of the network are fine-tuned automatically
Fig. 6
Fig. 6
The ROC curves of classifying image samples from database 1
Fig. 7
Fig. 7
The ROC curves of classifying image samples from database 2

References

    1. Diana Gaitini, Rhodri M Evans, and Gordana Ivanac, Chapter 16: thyroid ultrasound. EFSUMB Course Book, 2011.
    1. Kwak JY, Han KH, Yoon JH, Moon HJ, Son EJ, Park SH, Jung HK, Choi JS, Kim BM, Kim E-K. Thyroid imaging reporting and data system for us features of nodules: a step in establishing better stratification of cancer risk. Radiology. 2011;260(3):892–899. doi: 10.1148/radiol.11110206.
    1. Lingam RK, Qarib MH, Tolley NS. Evaluating thyroid nodules: predicting and selecting malignant nodules for fine-needle aspiration (fna) cytology. Insights Imaging. 2013;4(5):617–624. doi: 10.1007/s13244-013-0256-6.
    1. Chen S-J, Chang C-Y, Chang K-Y, Tzeng J-E, Chen Y-T, Lin C-W, Hsu W-C, Wei C-K. Classification of the thyroid nodules based on characteristic sonographic textural feature and correlated histopathology using hierarchical support vector machines. Ultrasound Med Biol. 2010;36(12):2018–2026. doi: 10.1016/j.ultrasmedbio.2010.08.019.
    1. Chang C-Y, Liu H-Y, Tseng C-H, Shih S-R. Computer-aided diagnosis for thyroid Graves’ disease in ultrasound images. Biomed Eng: Appl Basis Commun. 2010;22(02):91–99.
    1. Katsigiannis S, Keramidas EG, Maroulis D. A contourlet transform feature extraction scheme for ultrasound thyroid texture classification. Int J Eng Intell Syst Electr Eng Commun. 2010;18(3):171.
    1. Rajendra Acharya U, Vinitha Sree S, Swapna G, Gupta S, Filippo M, Garberoglio R, Witkowska A, Suri JS. Effect of complex wavelet transform filter on thyroid tumor classification in three-dimensional ultrasound. Proc Inst Mech Eng H J Eng Med. 2013;227(3):284–292. doi: 10.1177/0954411912472422.
    1. Savelonas MA, Iakovidis DK, Dimitropoulos N, Maroulis D: Computational characterization of thyroid tissue in the radon domain. In Computer-Based Medical Systems, 2007. CBMS’07. Twentieth IEEE International Symposium on. IEEE, 2007, pp 189–192
    1. Iakovidis DK, Keramidas EG, Maroulis D. Fuzzy local binary patterns for ultrasound texture characterization in International Conference Image Analysis and Recognition. Berlin: Springer; 2008. pp. 750–759.
    1. Chang C-Y, Chen S-J, Tsai M-F. Application of support-vector-machine-based method for feature selection and classification of thyroid nodules in ultrasound images. Pattern Recogn. 2010;43(10):3494–3506. doi: 10.1016/j.patcog.2010.04.023.
    1. Rajendra Acharya U, Swapna G, Vinitha Sree S, Molinari F, Gupta S, Bardales RH, Witkowska A, Suri JS. A review on ultrasound based thyroid cancer tissue characterization and automated classification. Technol Cancer Res Treat. 2014;13(4):289–301. doi: 10.7785/tcrt.2012.500381.
    1. Hijazi S, Kumar R, Rowen C: Using convolutional neural networks for image recognition. Technical Report, 2015, available online:
    1. Spanhol FA, Oliveira LS, Petitjean C, Heutte L: Breast cancer histopathological image classification using convolutional neural networks. In Neural Networks (IJCNN), 2016 International Joint Conference on, pp. 2560–2567, Vancouver, BC, Canada, 2016
    1. Li W, Cao P, Zhao D, Wang J: Pulmonary nodule classification with deep convolutional neural networks on computed tomography images. In Computational and Mathematical Methods in Medicine, 2016
    1. Holger R. Roth, Le Lu, Ari Seff, Kevin M. Cherry, Joanne Hoffman, Shijun Wang, Jiamin Liu and Evrim Turkbey, A new 2.5D representation for lymph node detection using random sets of deep convolutional neural network observations. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2014: 17th International Conference, pp. 520–527, Boston, MA, USA, 2014.
    1. Lina Pedraza, Carlos Vargas, Fabián Narváez, Oscar Durán, Emma Muñoz, and Eduardo Romero, An open access thyroid ultrasound image database, 2015
    1. Waqas Ahmed and Mark G. Eramian, Automated detection of grayscale bar and distance scale in ultrasound images, 2010.
    1. N. S. Narayan, P. Marziliano, and C. G. L. Hobbs, Automatic removal of manually induced artefacts in ultrasound images of thyroid gland. In 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2013, pp. 3399–3402.
    1. Youla DC, Webb H. Image restoration by the method of convex projections: part 1 theory. IEEE Trans Med Imaging. 1982;1(2):81–94. doi: 10.1109/TMI.1982.4307555.
    1. Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell, Caffe: convolutional architecture for fast feature embedding. In Proceedings of the 22Nd ACM International Conference on Multimedia, New York, NY, USA, 2014, MM ‘14, pp. 675–678, ACM. 25.
    1. Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich, Going deeper with convolutions. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
    1. Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ, editors. Advances in neural information processing systems 25. New York City: Curran Associates Inc.; 2012. pp. 1097–1105.
    1. Soekhoe D, van der Putten P, Plaat A. On the impact of data set size in transfer learning using deep neural networks. Cham: Springer International Publishing; 2016. pp. 50–60.
    1. Lim KJ, Choi CS, Yoon DY, Chang SK, Kim KK, Han H, Kim SS, Lee J, Jeon YH. Computer-aided diagnosis for the differentiation of malignant from benign thyroid nodules on ultrasonography. Acad Radiol. 2008;15(7):853–858. doi: 10.1016/j.acra.2007.12.022.
    1. Michalis A. Savelonas, Dimitris E. Maroulis, Dimitris K. Iakovidis, Nikos Dimitropoulos, Computer-aided malignancy risk assessment of nodules in thyroid US images utilizing boundary descriptors. Informatics, 2008. PCI'08. Panhellenic Conference on. IEEE, 2008
    1. Iakovidis DK, Keramidas EG, Maroulis D. Fusion of fuzzy statistical distributions for classification of thyroid ultrasound patterns. Artif Intell Med. 2010;50(1):33–41. doi: 10.1016/j.artmed.2010.04.004.
    1. Legakis I, Savelonas MA, Maroulis D, Iakovidis DK. Computer-based nodule malignancy risk assessment in thyroid ultrasound images. Int J Comput Appl. 2011;33(1):29–35.
    1. Luo S, Kim EH, Dighe M, Kim Y. Thyroid nodule classification using ultrasound elastography via linear discriminant analysis. Ultrasonics. 2011;51(4):425–431. doi: 10.1016/j.ultras.2010.11.008.
    1. Wu H, Deng Z, Zhang B, Liu Q, Chen J. Classifier model based on machine learning algorithms: Application to different diagnosis of suspicious thyroid nodules via sonography. Am J Roentgenol. 2016;207(4):859–864. doi: 10.2214/AJR.15.15813.

Source: PubMed

Szponzorok és közreműködők

Egészségi állapot

Kábítószer-beavatkozások

3
Iratkozz fel