Representation learning: a unified deep learning framework for automatic prostate MR segmentation

Abstract

Image representation plays an important role in medical image analysis. The key to the success of different medical image analysis algorithms is heavily dependent on how we represent the input data, namely features used to characterize the input image. In the literature, feature engineering remains as an active research topic, and many novel hand-crafted features are designed such as Haar wavelet, histogram of oriented gradient, and local binary patterns. However, such features are not designed with the guidance of the underlying dataset at hand. To this end, we argue that the most effective features should be designed in a learning based manner, namely representation learning, which can be adapted to different patient datasets at hand. In this paper, we introduce a deep learning framework to achieve this goal. Specifically, a stacked independent subspace analysis (ISA) network is adopted to learn the most effective features in a hierarchical and unsupervised manner. The learnt features are adapted to the dataset at hand and encode high level semantic anatomical information. The proposed method is evaluated on the application of automatic prostate MR segmentation. Experimental results show that significant segmentation accuracy improvement can be achieved by the proposed deep learning method compared to other state-of-the-art segmentation approaches.

Figures

Fig. 1

Schematic illustration of the basic ISA network, where inputs of the basic ISA network are image patches sampled from the training images. The basic ISA network contains two layers. The first layer is consisted of simple units (green circles) to capture the square nonlinearity among the input patches. The second layer is consisted of pooling units (red circles) to group and integrate the responses from the simple units in the first layer to capture the square root nonlinearity relationships. R1,R2,#x2026;,Rm denote the responses from the second layer.

Fig. 2

Typical filters learnt by the ISA network from 30 prostate T2 MR images with patch size 16 × 16 × 2. Each patch is normalized to have zero mean and unit variance before the training process.

Fig. 3

Schematic illustration of the stacked ISA deep learning architecture. We first learn the lower level ISA network with small input patches. Then, for each larger patch, we can represent it as s overlapping small patches, and we can obtain the pooling unit responses of each overlapping small patch based on the previously learnt lower level ISA network. The pooling unit responses of each overlapping small patch are then put through a PCA dimensionality reduction procedure to serve as input to train the higher level ISA network.

Fig. 4

(a) A prostate T2 MR image, with the reference voxel highlighted by the green cross. (b), (c), (d), and (e) denote the color coded difference map obtained by computing the Euclidean distance between the reference voxel and all the other voxels in (a) by using features extracted by Haar wavelet [3], HOG [4], low level ISA network, and the stacked ISA network, respectively.

Fig. 5

Typical segmentation results obtained by the proposed method. Each row shows the segmentation results of a patient. The estimated prostate boundary is highlighted in yellow, and the groundtruth prostate boundary is highlighted in red. The third row shows a failure case.