Multimodal MRI-Based Whole-Brain Assessment in Patients In Anoxoischemic Coma by Using 3D Convolutional Neural Networks

Giulia Maria Mattia, Benjamine Sarton, Edouard Villain, Helene Vinour, Fabrice Ferre, William Buffieres, Marie-Veronique Le Lann, Xavier Franceries, Patrice Peran, Stein Silva, Giulia Maria Mattia, Benjamine Sarton, Edouard Villain, Helene Vinour, Fabrice Ferre, William Buffieres, Marie-Veronique Le Lann, Xavier Franceries, Patrice Peran, Stein Silva

Abstract

Background: There is an unfulfilled need to find the best way to automatically capture, analyze, organize, and merge structural and functional brain magnetic resonance imaging (MRI) data to ultimately extract relevant signals that can assist the medical decision process at the bedside of patients in postanoxic coma. We aimed to develop and validate a deep learning model to leverage multimodal 3D MRI whole-brain times series for an early evaluation of brain damages related to anoxoischemic coma.

Methods: This proof-of-concept, prospective, cohort study was undertaken at the intensive care unit affiliated with the University Hospital (Toulouse, France), between March 2018 and May 2020. All patients were scanned in coma state at least 2 days (4 ± 2 days) after cardiac arrest. Over the same period, age-matched healthy volunteers were recruited and included. Brain MRI quantification encompassed both "functional data" from regions of interest (precuneus and posterior cingulate cortex) with whole-brain functional connectivity analysis and "structural data" (gray matter volume, T1-weighted, fractional anisotropy, and mean diffusivity). A specifically designed 3D convolutional neuronal network (CNN) was created to allow conscious state discrimination (coma vs. controls) by using raw MRI indices as the input. A voxel-wise visualization method based on the study of convolutional filters was applied to support CNN outcome. The Ethics Committee of the University Teaching Hospital of Toulouse, France (2018-A31) approved the study and informed consent was obtained from all participants.

Results: The final cohort consisted of 29 patients in postanoxic coma and 34 healthy volunteers. Coma patients were successfully discerned from controls by using 3D CNN in combination with different MR indices. The best accuracy was achieved by functional MRI data, in particular with resting-state functional MRI of the posterior cingulate cortex, with an accuracy of 0.96 (range 0.94-0.98) on the test set from 10-time repeated tenfold cross-validation. Even more satisfactory performances were achieved through the majority voting strategy, which was able to compensate for mistakes from single MR indices. Visualization maps allowed us to identify the most relevant regions for each MRI index, notably regions previously described as possibly being involved in consciousness emergence. Interestingly, a posteriori analysis of misclassified patients indicated that they may present some common functional MRI traits with controls, which suggests further favorable outcomes.

Conclusions: A fully automated identification of clinically relevant signals from complex multimodal neuroimaging data is a major research topic that may bring a radical paradigm shift in the neuroprognostication of patients with severe brain injury. We report for the first time a successful discrimination between patients in postanoxic coma patients from people serving as controls by using 3D CNN whole-brain structural and functional MRI data. Clinical Trial Number http://ClinicalTrials.gov (No. NCT03482115).

Keywords: Cardiac arrest; Coma; Convolutional neural networks; Deep learning; Multimodal MRI.

Conflict of interest statement

All authors certify that they have no conflict of interest to report.

© 2022. The Author(s).

Figures

Fig. 1
Fig. 1
Methods overview. Structural and functional magnetic resonance (MR) indices from the set of controls (n = 34) and patients in coma (n = 29) were assessed to perform binary classification by using a 3D convolutional neuronal network (CNN) in a 10-time repeated tenfold cross-validation. The 3D CNN model is schematized with fundamental building blocks. Feeding as input each MR index, we examined their discriminant power by using standard evaluation metrics and visualization maps to discover the most relevant voxels taken into account for CNN prediction. AveragePooling3D, average pooling layer, BN, batch normalization, Conv3D, convolutional layer, Dropout, dropout layer, ELU, exponential linear unit activation, FCL, fully connected layer, Flatten, output from the convolutional part reshaped in a 1D array, Softmax, softmax activation
Fig. 2
Fig. 2
Individual classification according to magnetic resonance imaging (MRI) indices. Analysis of misclassified samples was conducted on the basis of model performance. Controls and patients in coma were associated with their classification label assigned by the 3D convolutional neuronal network (CNN) according to magnetic resonance (MR) index. Majority voting (MajVot) was computed to assess whether the individual MR index performance on each sample could be improved considering the most scored classification output among all MR indices. This was indeed the case for controls, all correctly classified with MajVot. Regarding patients in coma, MajVot was second only to rs-fMRI PCC, totaling only two misclassified patients instead of four. FA, fractional anisotropy, GM, gray matter volume, MD, mean diffusivity, PCC, posterior cingulate cortex, PreCun, precuneus, rs-fMRI, resting-state functional MRI, T1, T1-weighted
Fig. 3
Fig. 3
3D CNN visual interpretation. Visualization maps representing activation values from the learned convolutional filters passed over the images. The absolute difference between maps belonging to correctly classified samples of the training set is shown to highlight the most discriminant voxels. To obtain clearer visualizations, we applied a threshold value (Threshold) equal to half of the maximum value (Max) considering activation values from every magnetic resonance (MR) index. Notice how voxels with greater activation vary according to the MR index. FA, fractional anisotropy, GM, gray matter volume, l, left, MD, mean diffusivity, PCC, posterior cingulate cortex, PreCun, precuneus, r, right, rs-fMRI, resting-state functional MRI, T1, T1-weighted

References

    1. Greer DM, Rosenthal ES, Wu O. Neuroprognostication of hypoxic–ischaemic coma in the therapeutic hypothermia era. Nature Rev. Neurol. 2014;10:190–203. 10.1038/nrneurol.2014.36
    1. Sandroni C, Grippo A, Nolan JP. ERC-ESICM guidelines for prognostication after cardiac arrest: time for an update. Intensive Care Med. 2020;46:1901–3. 10.1007/s00134-020-06224-x
    1. Edlow BL, Barra ME, Zhou DW, Foulkes AS, Snider SB, Threlkeld ZD, et al. Personalized connectome mapping to guide targeted therapy and promote recovery of consciousness in the intensive care unit. Neurocrit Care. 2020;33(2):364–375. doi: 10.1007/s12028-020-01062-7.
    1. Velly L, Perlbarg V, Boulier T, Adam N, Delphine S, Luyt CE, et al. Use of brain diffusion tensor imaging for the prediction of long-term neurological outcomes in patients after cardiac arrest: a multicentre, international, prospective, observational, cohort study. Lancet Neurol. 2018;17:317–26.
    1. Silva S, Peran P, Kerhuel L, Malagurski B, Chauveau N, Bataille B, et al. Brain gray matter mri morphometry for neuroprognostication after cardiac arrest. Crit Care Med 2017;45:e763–71.
    1. Malagurski B, Péran P, Sarton B, Riu B, Gonzalez L, Vardon-Bounes F, et al. Neural signature of coma revealed by posteromedial cortex connection density analysis. Neuroimage Clin; 2017;15:315–24.
    1. Malagurski B, Péran P, Sarton B, Vinour H, Naboulsi E, Riu B, et al. Topological disintegration of resting state functional connectomes in coma. Neuroimage; 2019;195:354–61.
    1. Peran P, Malagurski B, Nemmi F, Sarton B, Vinour H, Ferre F, et al. Functional and structural integrity of frontoparietal connectivity in traumatic and anoxic coma. Crit Care Med; 2020;48:E639–47.
    1. Silva S, de Pasquale F, Vuillaume C, Riu B, Loubinoux I, Geeraerts T, et al. Disruption of posteromedial large-scale neural communication predicts recovery from coma. Neurology 2015;85:2036–44.
    1. Giacino JT, Fins JJ, Laureys S, Schiff ND. Disorders of consciousness after acquired brain injury: the state of the science. Nat Rev Neurol; 2014;10:99–114.
    1. Edlow BL, Claassen J, Schiff ND, Greer DM. Recovery from disorders of consciousness: mechanisms, prognosis and emerging therapies. Nature Rev Neurol 2020 17(3):135–56.
    1. Hubel D, Wiesel T. Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J Physiol. 1962;160:106–154. doi: 10.1113/jphysiol.1962.sp006837.
    1. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436–444. doi: 10.1038/nature14539.
    1. Holzinger A, Biemann C, Pattichis CS, Kell DB. What do we need to build explainable AI systems for the medical domain? 2017 [cited 2021 Nov 20]; Available from:
    1. Gilpin LH, Bau D, Yuan B, Bajwa A, Specter M, Kagal L. Explaining explanations: an overview of interpretability of machine learning, in 2018 IEEE 5th Int Conf on Data Science and Advanced Analytics (DSAA). 2018;80–9.
    1. Sarraf S, DeSouza D, Anderson JAE, Tofighi G. DeepAD: Alzheimer’s Disease Classification via Deep Convolutional Neural Networks using MRI and fMRI. bioRxiv. 2016.
    1. Cole JH, Poudel RPK, Tsagkrasoulis D, Caan MWA, Steves C, Spector TD, et al. Predicting brain age with deep learning from raw imaging data results in a reliable and heritable biomarker. Neuroimage. 2017;163:115–124. doi: 10.1016/j.neuroimage.2017.07.059.
    1. Litjens G, others. A survey on deep learning in medical image analysis. Med Image Anal 2017;42:60–88.
    1. Booth CM, Boone RH, Tomlinson G, Detsky AS. Is this patient dead, vegetative, or severely neurologically impaired? Assessing outcome for comatose survivors of cardiac arrest. JAMA 2004;291:870–9.
    1. Seel RT, Sherer M, Whyte J, Katz DI, Giacino JT, Rosenbaum AM, et al. Assessment scales for disorders of consciousness: evidence-based recommendations for clinical practice and research. Arch Phys Med Rehabil 2010;91:1795–813.
    1. Lundervold AS, Lundervold A. An overview of deep learning in medical imaging focusing on MRI. Z Med Phys. 2019;29:102–127. doi: 10.1016/j.zemedi.2018.11.002.
    1. Lecun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc IEEE. 1998;86:2278–2324. doi: 10.1109/5.726791.
    1. Rumelhart D, Hinton GE, Williams RJ. Learning representations by back-propagating errors. Nature. 1986;323:533–536. doi: 10.1038/323533a0.
    1. Rosenblatt F. The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev. 1958;65:386–408. doi: 10.1037/h0042519.
    1. Goodfellow I, Bengio Y, Courville A. Deep learning. Cambridge: MIT Press; 2016.
    1. Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition. Computing Research Repository. 2015;
    1. Krizhevsky A, Sutskever I, Hinton GE. ImageNet Classification with Deep Convolutional Neural Networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ, editors. Advances in neural information processing systems. Curran Associates, Inc.; 2012.
    1. Ioffe S, Szegedy C. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. 2015;.
    1. Chollet F, others. Keras. GitHub; 2015. Available from:
    1. Srivastava N, Hinton GE, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: A simple way to prevent neural networks from overfitting. J Mach Learn Res. 2014;15:1929–1958.
    1. Kingma DP, Ba J. Adam: A Method for Stochastic Optimization. CoRR. 2015;.
    1. Abadi M, et al. TensorFlow: A system for large-scale machine learning. In: Proceedings of the 12th USENIX symposium on operating systems design and implementation (OSDI). 2016.
    1. Qureshi MI, Oh J, Lee B. 3D-CNN based discrimination of schizophrenia using resting-state fMRI. Artif Intell Med. 2019;98:10–17. doi: 10.1016/j.artmed.2019.06.003.
    1. Villain E, Mattia GM, Nemmi F, Péran P, Franceries X, le Lann MV. Visual interpretation of CNN decision-making process using Simulated Brain MRI. In: 2021 IEEE 34th International Symposium on Computer-Based Medical Systems (CBMS). 2021. pp. 515–20.
    1. Dehaene S, Changeux JP. Experimental and Theoretical Approaches to Conscious Processing. Neuron Cell Press. 2011;70:200–227.
    1. Dehaene S, Changeux JP, Naccache L, Sackur J, Sergent C. Conscious, preconscious, and subliminal processing: a testable taxonomy. Trends Cognit Sci. 2006;10:204–211. doi: 10.1016/j.tics.2006.03.007.
    1. Gusnard DA, Raichle ME. Searching for a baseline: functional imaging and the resting human brain. Nature Rev Neurosci. 2001;2:685–94. 10.1038/35094500
    1. di Perri C, Bahri MA, Amico E, Thibaut A, Heine L, Antonopoulos G, et al. Neural correlates of consciousness in patients who have emerged from a minimally conscious state: a cross-sectional multimodal imaging study. Lancet Neurol. 2016;15:830–42.
    1. Fransson P, Marrelec G. The precuneus/posterior cingulate cortex plays a pivotal role in the default mode network: Evidence from a partial correlation network analysis. NeuroImage. 2008;42:1178–84.
    1. Koch C, Massimini M, Boly M, Tononi G. Posterior and anterior cortex — where is the difference that makes the difference? Nature Rev Neurosci. 2016;17(10):666–666.
    1. Brigato L, Iocchi L. A close look at deep learning with small data. 2020.
    1. Korolev S, Safiullin A, Belyaev M, Dodonova Y. Residual and plain convolutional neural networks for 3D brain MRI classification. In: Proceedings of the international symposium on biomedical imaging. IEEE Comput Soc; 2017;835–8.
    1. Esmaeilzadeh S, Yang Y, Adeli E. End-to-end parkinson disease diagnosis using brain MR-images by 3D-CNN. 2018;.
    1. Wang Z, Sun Y, Shen Q, Cao L. Dilated 3d convolutional neural networks for brain mri data classification. IEEE Access. 2019;7:134388–134398. doi: 10.1109/ACCESS.2019.2941912.
    1. Khosla M, Jamison K, Kuceyeski A, Sabuncu MR. Ensemble learning with 3D convolutional neural networks for functional connectome-based prediction. Neuroimage. 2019;199:651–662. doi: 10.1016/j.neuroimage.2019.06.012.
    1. Mattia GM, Nemmi F, Villain E, le Lann MV, Franceries X, Péran P. Investigating the discrimination ability of 3D convolutional neural networks applied to altered brain MRI parametric maps. TechRxiv Preprint. 2021;

Source: PubMed

Sponsor e collaboratori

Condizioni mediche

Interventi farmacologici

3
Sottoscrivi