A Deep Learning-Based Radiomics Model for Differentiating Benign and Malignant Renal Tumors

Abstract

Objectives: To investigate the effect of transfer learning on computed tomography (CT) images for the benign and malignant classification on renal tumors and to attempt to improve the classification accuracy by building patient-level models.

Methods: One hundred ninety-two cases of renal tumors were collected and identified by pathologic diagnosis within 15 days after enhanced CT examination (66% male, 70% malignant renal tumors, average age of 62.27 ± 12.26 years). The InceptionV3 model pretrained by the ImageNet dataset was cross-trained to perform this classification. Five image-level models were established for each of the Slice, region of interest (ROI), and rectangular box region (RBR) datasets. Then, two patient-level models were built based on the optimal image-level models. The network's performance was evaluated through analysis of the receiver operating characteristic (ROC) and five-fold cross-validation.

Results: In the image-level models, the test results of model trained on the Slice dataset [accuracy (ACC) = 0.69 and Matthews correlation coefficient (MCC) = 0.45] were the worst. The corresponding results on the ROI dataset (ACC = 0.97 and MCC = 0.93) were slightly better than those on the RBR dataset (ACC = 0.93 and MCC = 0.85) when freezing the weights before the mixed6 layer. Compared with the image-level models, both patient-level models could discriminate better (ACC increased by 2%-5%) on the RBR and Slice datasets.

Conclusions: Deep learning can be used to classify benign and malignant renal tumors from CT images. Our patient-level models could benefit from 3D data to improve the accuracy.

Copyright © 2018 The Authors. Published by Elsevier Inc. All rights reserved.

Figures

Figure 1

Flowchart of image preprocessing and datasets setting. A (blue lines) was the flow of image preprocessing. An example of renal/high-attenuation/low-attenuation CT windowing for an axis renal CT slice. We encode the renal/high-attenuation/low-attenuation CT windowing into red/green/blue channels. B (green lines) was the flow of datasets setting. The Slice dataset was made up of axial multichannel renal CT slices. The ROI mask was drawn manually by two experienced radiologists on each image of the Slice dataset. The RBR mask was generated from the bounding box of ROI mask's contour. The ROI/RBR dataset consisted of ROI/RBR images which were gotten from slice images and corresponding ROI/RBR masks.

Figure 2

An overview of two patient-level models (model one: FC method, model two: GRU method). Our approach included three steps. The first step was learning intraimage features (1024 × 1) from one patient's N renal tumor image based on optimal image-level model and concatenating features into two-dimensional vectors(1024 × N) as the input tensors of the patient-level model. In the second step, there were differences between the two models. Model 1 added one Max pooling layer to merge the image sequences into a one-dimensional vector and FC layers to learn interimage features, while model 2 only added GRU layers. In the third step, the softmax activation containing two nodes (benign/malignant) was used to realize the diagnosis on the patient-level.

Figure 3

Traces of training loss and validation loss (blue solid and dash lines) and validation accuracy (orange lines). A, B, and C columns were trained on the ROI dataset, RBR dataset, and Slice dataset, respectively. −1, −2, −3, −4, and − 5 denoted freezing the weights of CNN before mixed0, mixed3, mixed6, mixed9, and mixed10 layers, respectively.

Figure 4

ROC averaged on five-fold cross-validation of the transfer learning with freezing different layers for (A) Slice, (B) ROI, and (C) RBR datasets. (D) The plot of AUC calculated from ROC with freezing different layers for three datasets. ROI and RBR datasets had larger AUCs than Slice dataset with statistical significant (P = .001 and .008, respectively), while the differences between ROI and RBR datasets of AUCs did not reach statistical significant (P = .101).

Figure 5

The plot of SEN, SPEC, PPV, NPV, MCC, and ACC with freezing different layers for three datasets.