Automatic segmentation of the spinal cord and intramedullary multiple sclerosis lesions with convolutional neural networks

Abstract

The spinal cord is frequently affected by atrophy and/or lesions in multiple sclerosis (MS) patients. Segmentation of the spinal cord and lesions from MRI data provides measures of damage, which are key criteria for the diagnosis, prognosis, and longitudinal monitoring in MS. Automating this operation eliminates inter-rater variability and increases the efficiency of large-throughput analysis pipelines. Robust and reliable segmentation across multi-site spinal cord data is challenging because of the large variability related to acquisition parameters and image artifacts. In particular, a precise delineation of lesions is hindered by a broad heterogeneity of lesion contrast, size, location, and shape. The goal of this study was to develop a fully-automatic framework - robust to variability in both image parameters and clinical condition - for segmentation of the spinal cord and intramedullary MS lesions from conventional MRI data of MS and non-MS cases. Scans of 1042 subjects (459 healthy controls, 471 MS patients, and 112 with other spinal pathologies) were included in this multi-site study (n = 30). Data spanned three contrasts (T1-, T2-, and T2∗-weighted) for a total of 1943 vol and featured large heterogeneity in terms of resolution, orientation, coverage, and clinical conditions. The proposed cord and lesion automatic segmentation approach is based on a sequence of two Convolutional Neural Networks (CNNs). To deal with the very small proportion of spinal cord and/or lesion voxels compared to the rest of the volume, a first CNN with 2D dilated convolutions detects the spinal cord centerline, followed by a second CNN with 3D convolutions that segments the spinal cord and/or lesions. CNNs were trained independently with the Dice loss. When compared against manual segmentation, our CNN-based approach showed a median Dice of 95% vs. 88% for PropSeg (p ≤ 0.05), a state-of-the-art spinal cord segmentation method. Regarding lesion segmentation on MS data, our framework provided a Dice of 60%, a relative volume difference of -15%, and a lesion-wise detection sensitivity and precision of 83% and 77%, respectively. In this study, we introduce a robust method to segment the spinal cord and intramedullary MS lesions on a variety of MRI contrasts. The proposed framework is open-source and readily available in the Spinal Cord Toolbox.

Trial registration: ClinicalTrials.gov NCT02117375.

Keywords: Convolutional neural networks; MRI; Multiple sclerosis; Segmentation; Spinal cord.

Conflict of interest statement

Declaration of interest:

Charley Gros, Benjamin De Leener, Josefina Maranzano, Dominique Eden, Atef Badji, Govind Nair, Tobias Granberg, Hugh Kearney, Ferran Prados, Russell Ouellette, Daniel S. Reich, Pierre Labauge, Leszek Stawiarz, Anne Kerbrat, Elise Bannier, Shahamat Tauhid and Julien Cohen-Adad have no relevant financial interests to disclose.

Copyright © 2018 Elsevier Inc. All rights reserved.

Figures

Figure 1:. Spinal cord axial slice samples.

(a-f) show the variability of the images in terms of resolution, field of view, and MR contrasts. Images were acquired from 6 different sites, of subjects with different clinical status: healthy control (HC, b), amyotrophic lateral sclerosis (ALS, a), degenerative cervical myelopathy (DCM, c) and multiple sclerosis (MS, d-f). The in-plane resolutions vary across the images. For all images, the spinal cord and lesion voxels represent less than 1% and 0.1%, respectively, of the entire volume. The shape, location, size, and level of contrast differ among MS lesions (d-f). The histograms for spinal cord and lesion voxels of the MS patient (d-f) images are shown at the bottom. Although lesions mostly appear hyperintense in T2- and T2*-weighted, a substantial overlap between spinal cord and lesion intensities is observed, leading to low contrast, especially for T2*-w images (f) with similarities between grey matter and lesion appearance.

Figure 2:. Overview of the data set.

Samples of cross-sectional axial slices of the three MR contrast data sets (T1-weighted, T2-weighted, T2*-weighted) are depicted (top row). Image characteristics in terms of orientation (orient.) and resolution (resol.), grouped by isotropic, anisotropic and with axial (Ax.) orientation or sagittal (Sag.) orientation are presented (middle row). The last row shows the proportion of clinical status among the imaged subjects, including: healthy controls (HC), multiple sclerosis (MS), degenerative cervical myelopathy (DCM), neuromyelitis optica (NMO), traumatic spinal cord injury (SCI), amyotrophic lateral sclerosis (ALS), and syringomyelia (SYR). Imaging parameters across participating sites are detailed in Table A1 (see Appendix).

Figure 3:. Automatic segmentation framework.

(1) detection of the spinal cord by CNN1 which outputs a heatmap (red-to-yellow) of the spinal cord location, (2) computation of the spinal cord centerline (pink) from the spinal cord heatmap (Gros et al., 2018), and extraction of 3D patches in a volume of interest surrounding the spinal cord centerline, (3) segmentation of the spinal cord (red) by CNN2-SC, and/or of lesions (blue) by CNN2-lesion. SC: Spinal cord ; CNN: Convolutional Neural Network ; S: Superior ; I: Inferior ; A: Anterior ; P: Posterior.

Figure 4:

Examples of automatic spinal cord segmentations on T1-w (top), T2-w (middle) and T2*-w (bottom) MRI data. This includes a comparison between manual (green) and automatic (red) delineations, with Dice coefficient indicated just below each comparison. Note that the depicted samples represent a variety of subjects in terms of clinical status, and were scanned at different sites, identified by their ID (e.g. S10_HC23 is the ID of the HC subject #23, from the site #10). Abbreviations: A: Anterior ; P: Posterior ; L: Left ; R: Right ; I: Inferior ; S: Superior ; Auto.: Automatic ; HC: healthy controls ; MS: multiple sclerosis ; DCM: degenerative cervical myelopathy ; NMO: neuromyelitis optica ; ALS: amyotrophic lateral sclerosis.

Figure 5:

Examples of automatic lesion segmentations on Axial T2-w (top left), Axial T2*-w (bottom) and Sagittal T2-w (top, right) MRI data. This includes a comparison between manual (green) and automatic (blue) delineations, with Dice coefficients indicated just below each comparison. Note that the depicted samples were scanned at different sites, identified by their ID (e.g. S1_RRMS17 is the ID of subject #17 from site #1 with relapsing-remitting multiple sclerosis).

Figure 6:. Inter-rater variability.

Comparison between raters and automatic MS lesion segmentation on 10 testing subjects. (A.) shows the Dice coefficient (range of [0-100] with 100% as best possible value) computed between the rater consensus (majority voting) and each individual rater (n=7) segmentation (purple distributions) as well as the automatic method (blue dot). (B.) depicts axial cross-sectional samples with the manual segmentation of the raters and the automatic delineation (blue). The consensus between raters vary from “low agreements” (in blues, mainly on the borders) to “strong agreement” (in reds, mainly on the cores). The green-to-red (see colormap) voxels were considered as part of the majority voting masks. (C.) presents the segmentation time, averaged across subjects, for each rater and the automatic segmentation (iMac i7 4-cores 3.4 GHz 8Gb RAM). Abbreviations: Seg.: Segmentation ; A: Anterior ; P: Posterior ; L: Left ; R: Right ; I: Inferior ; S: Superior ; Auto.: Automatic.

Figure 7:

Visualisation of feature map instances, learnt by different layers of the CNN2-Lesion, applied to an input image (left) leading to a binary segmentation (right). The normalised values represent the responses to filters learnt during the training step, with a colormap from blues (weak filter match) to reds (strong filter match).