Deep Multimodal Learning From MRI and Clinical Data for Early Prediction of Neurodevelopmental Deficits in Very Preterm Infants

Lili He, Hailong Li, Ming Chen, Jinghua Wang, Mekibib Altaye, Jonathan R Dillman, Nehal A Parikh, Lili He, Hailong Li, Ming Chen, Jinghua Wang, Mekibib Altaye, Jonathan R Dillman, Nehal A Parikh

Abstract

The prevalence of disabled survivors of prematurity has increased dramatically in the past 3 decades. These survivors, especially, very preterm infants (VPIs), born ≤ 32 weeks gestational age, are at high risk for neurodevelopmental impairments. Early and clinically effective personalized prediction of outcomes, which forms the basis for early treatment decisions, is urgently needed during the peak neuroplasticity window-the first couple of years after birth-for at-risk infants, when intervention is likely to be most effective. Advances in MRI enable the noninvasive visualization of infants' brains through acquired multimodal images, which are more informative than unimodal MRI data by providing complementary/supplementary depicting of brain tissue characteristics and pathology. Thus, analyzing quantitative multimodal MRI features affords unique opportunities to study early postnatal brain development and neurodevelopmental outcome prediction in VPIs. In this study, we investigated the predictive power of multimodal MRI data, including T2-weighted anatomical MRI, diffusion tensor imaging, resting-state functional MRI, and clinical data for the prediction of neurodevelopmental deficits. We hypothesize that integrating multimodal MRI and clinical data improves the prediction over using each individual data modality. Employing the aforementioned multimodal data, we proposed novel end-to-end deep multimodal models to predict neurodevelopmental (i.e., cognitive, language, and motor) deficits independently at 2 years corrected age. We found that the proposed models can predict cognitive, language, and motor deficits at 2 years corrected age with an accuracy of 88.4, 87.2, and 86.7%, respectively, significantly better than using individual data modalities. This current study can be considered as proof-of-concept. A larger study with external validation is important to validate our approach to further assess its clinical utility and overall generalizability.

Keywords: MRI; brain connectome; deep learning; diffuse white matter abnormality; diffusion tensor imaging; neurodevelopment; resting state functional MRI; very preterm infants.

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2021 He, Li, Chen, Wang, Altaye, Dillman and Parikh.

Figures

Figure 1
Figure 1
A deep multimodal learning model consists of feature extractor and fusion classifier, for the prediction of neurodevelopmental (cognitive, language, and motor) deficits using MRI and clinical data.
Figure 2
Figure 2
Top discriminative region-to-region functional connections for early prediction of cognitive, language, and motor deficits. (A) circos plot visualization; (B) Full names and abbreviations table. Three common connections were identified to be important for the prediction of all three deficits (red); five common connections were identified to be predictive of both cognitive and language deficits (red and green); seven common connections were identified to be predictive of both language and motor deficits (red and blue); and seven common connections were identified to be predictive of both cognitive and motor deficits (red and yellow).
Figure 3
Figure 3
Top discriminative region-to-region structural connections for early prediction of cognitive, language, and motor deficits. (A) circos plot visualization; (B) Full names and abbreviations table. Three common connections were identified to be important for the prediction of all three deficits (red); eight common connections were identified to be predictive of both cognitive and language deficits (red and green); seven common connections were identified to be predictive of both language and motor deficits (red and blue); and four common connections were identified to be predictive of both cognitive and motor deficits (red and yellow).

References

    1. Ambalavanan N., Carlo W. A., Tyson J. E., Langer J. C., Walsh M. C., Parikh N. A., et al. . (2012). Outcome trajectories in extremely preterm infants. Pediatrics 130, e115–e125. 10.1542/peds.2011-3693
    1. Bassett D. S., Brown J. A., Deshpande V., Carlson J. M., Grafton S. T. (2011). Conserved and variable architecture of human white matter connectivity. Neuroimage 54, 1262–1279. 10.1016/j.neuroimage.2010.09.006
    1. Bayley N. (2009). Bayley-III: Bayley Scales of Infant and Toddler Development. San Antonio, TX: Giunti OS; Pearson.
    1. Beauchamp M. H., Thompson D. K., Howard K., Doyle L. W., Egan G. F., Inder T. E., et al. . (2008). Preterm infant hippocampal volumes correlate with later working memory deficits. Brain 131, 2986–2994. 10.1093/brain/awn227
    1. Blencowe H., Cousens S., Oestergaard M. Z., Chou D., Moller A. B., Narwal R., et al. . (2012). National, regional, and worldwide estimates of preterm birth rates in the year 2010 with time trends since 1990 for selected countries: a systematic analysis and implications. Lancet 379, 2162–2172. 10.1016/S0140-6736(12)60820-4
    1. Boardman J. P., Craven C., Valappil S., Counsell S. J., Dyet L. E., Rueckert D., et al. . (2010). A common neonatal image phenotype predicts adverse neurodevelopmental outcome in children born preterm. Neuroimage 52, 409–414. 10.1016/j.neuroimage.2010.04.261
    1. Cao M., Huang H., He Y. (2017). Developmental connectomics from infancy through early childhood. Trends Neurosci. 40, 494–506. 10.1016/j.tins.2017.06.003
    1. Chaddad A., Desrosiers C., Toews M. (2017). Multi-scale radiomic analysis of sub-cortical regions in MRI related to autism, gender and age. Sci. Rep. 7:45639. 10.1038/srep45639
    1. Chen M., Li H., Wang J., Yuan W., Altaye M., Parikh N. A., et al. . (2020). Early prediction of cognitive deficit in very preterm infants using brain structural connectome with transfer learning enhanced deep convolutional neural networks. Front. Neurosci. 14:858. 10.3389/fnins.2020.00858
    1. Cheruel F., Dormont J. F., Amalric M., Schmied A., Farin D. (1994). The role of putamen and pallidum in motor initiation in the cat. I. Timing of movement-related single-unit activity. Exp. Brain Res. 100, 250–266. 10.1007/BF00227195
    1. de Jong L. W., van der Hiele K., Veer I. M., Houwing J. J., Westendorp R. G., Bollen E. L., et al. . (2008). Strongly reduced volumes of putamen and thalamus in Alzheimer's disease: an MRI study. Brain 131, 3277–3285. 10.1093/brain/awn278
    1. den Heijer T., der Lijn F., Vernooij M. W., de Groot M., Koudstaal P. J., van der Lugt A., et al. . (2012). Structural and diffusion MRI measures of the hippocampus and memory performance. Neuroimage 63, 1782–1789. 10.1016/j.neuroimage.2012.08.067
    1. Ekstrom A. D., Ranganath C. (2018). Space, time, and episodic memory: the hippocampus is all over the cognitive map. Hippocampus 28, 680–687. 10.1002/hipo.22750
    1. Fischi-Gómez E., Vasung L., Meskaldji D. E., Lazeyras F., Borradori-Tolsa C., Hagmann P., et al. . (2015). Structural brain connectivity in school-age preterm infants provides evidence for impaired networks relevant for higher order cognitive skills and social cognition. Cereb. Cortex 25, 2793–2805. 10.1093/cercor/bhu073
    1. Fortin J. P., Parker D., Tunc B., Watanabe T., Elliott M. A., Ruparel K., et al. . (2017). Harmonization of multi-site diffusion tensor imaging data. Neuroimage 161, 149–170. 10.1016/j.neuroimage.2017.08.047
    1. Friston K. J. (1994). Statistical Parametric Mapping. Cambridge, MA: Academic Press.
    1. Gerdes J. S., Keller S. S., Schwindt W., Evers S., Mohammadi S., Deppe M. (2012). Progression of microstructural putamen alterations in a case of symptomatic recurrent seizures using diffusion tensor imaging. Seizure 21, 478–481. 10.1016/j.seizure.2012.03.015
    1. Gilmore J. H., Knickmeyer R. C., Gao W. (2018). Imaging structural and functional brain development in early childhood. Nat. Rev. Neurosci. 19:123. 10.1038/nrn.2018.1
    1. Girault J. B., Munsell B. C., Puechmaille D., Goldman B. D., Prieto J. C., Styner M., et al. . (2019). White matter connectomes at birth accurately predict cognitive abilities at age 2. Neuroimage 192, 145–155. 10.1016/j.neuroimage.2019.02.060
    1. Gooijers J., Chalavi S., Beeckmans K., Michiels K., Lafosse C., Sunaert S., et al. . (2016). Subcortical volume loss in the thalamus, putamen, and pallidum, induced by traumatic brain injury, is associated with motor performance deficits. Neurorehabil. Neural Repair 30, 603–614. 10.1177/1545968315613448
    1. Gozdas E., Parikh N. A., Merhar S. L., Tkach J. A., He L., Holland S. K. (2018). Altered functional network connectivity in preterm infants: antecedents of cognitive and motor impairments? Brain Struct. Funct. 223, 3665–3680. 10.1007/s00429-018-1707-0
    1. Guo W., Wang J., Wang S. (2019). Deep multimodal representation learning: a survey. IEEE Access 7, 63373–63394. 10.1109/ACCESS.2019.2916887
    1. Hackman D. A., Farah M. J. (2009). Socioeconomic status and the developing brain. Trends Cogn. Sci. 13, 65–73. 10.1016/j.tics.2008.11.003
    1. Haixiang G., Yijing L., Shang J., Mingyun G., Yuanyue H., Bing G. (2017). Learning from class-imbalanced data: review of methods and applications. Expert Syst. Appl. 73, 220–239. 10.1016/j.eswa.2016.12.035
    1. Hallquist M. N., Hwang K., Luna B. (2013). The nuisance of nuisance regression: spectral misspecification in a common approach to resting-state fMRI preprocessing reintroduces noise and obscures functional connectivity. Neuroimage 82, 208–225. 10.1016/j.neuroimage.2013.05.116
    1. He L., Li H., Holland S., Yuan W., Altaye M., Parikh N. (2018). Early prediction of cognitive deficits in very preterm infants using functional connectome data in an artificial neural network framework. Neuroimage Clin. 18, 290–297. 10.1016/j.nicl.2018.01.032
    1. He L., Li H., Wang J., Chen M., Gozdas E., Dillman J. R., et al. . (2020). A multi-task, multi-stage deep transfer learning model for early prediction of neurodevelopment in very preterm infants. Sci. Rep. 10:15072. 10.1038/s41598-020-71914-x
    1. He L., Parikh N. A. (2013). Atlas-guided quantification of white matter signal abnormalities on term-equivalent age MRI in very preterm infants: findings predict language and cognitive development at two years of age. PLoS One 8:e85475. 10.1371/journal.pone.0085475
    1. He L., Parikh N. A. (2015). Aberrant executive and frontoparietal functional connectivity in very preterm infants with diffuse white matter abnormalities. Pediatr. Neurol. 53, 330–337. 10.1016/j.pediatrneurol.2015.05.001
    1. He L., Parikh N. A. (2016). Brain functional network connectivity development in very preterm infants: the first six months. Early Hum. Dev. 98, 29–35. 10.1016/j.earlhumdev.2016.06.002
    1. Heinsfeld A. S., Franco A. R., Craddock R. C., Buchweitz A., Meneguzzi F. (2018). Identification of autism spectrum disorder using deep learning and the ABIDE dataset. Neuroimage Clin. 17, 16–23. 10.1016/j.nicl.2017.08.017
    1. Hjelm R. D., Calhoun V. D., Salakhutdinov R., Allen E. A., Adali T., Plis S. M. (2014). Restricted Boltzmann machines for neuroimaging: an application in identifying intrinsic networks. Neuroimage 96, 245–260. 10.1016/j.neuroimage.2014.03.048
    1. Hyvärinen J., Poranen A. (1978). Receptive field integration and submodality convergence in the hand area of the post-central gyrus of the alert monkey. J. Physiol. 283, 539–556. 10.1113/jphysiol.1978.sp012518
    1. Ioffe S., Szegedy C. (2015). Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167 [arXiv preprint].
    1. Jarjour I. T. (2015). Neurodevelopmental outcome after extreme prematurity: a review of the literature. Pediatr. Neurol. 52, 143–152. 10.1016/j.pediatrneurol.2014.10.027
    1. Johnston M. V. (2009). Plasticity in the developing brain: implications for rehabilitation. Dev. Disabil. Res. Rev. 15, 94–101. 10.1002/ddrr.64
    1. Kassner A., Thornhill R. (2010). Texture analysis: a review of neurologic MR imaging applications. Am. J. Neuroradiol. 31, 809–816. 10.3174/ajnr.A2061
    1. Kawahara J., Brown C. J., Miller S. P., Booth B. G., Chau V., Grunau R. E., et al. . (2017). BrainNetCNN: convolutional neural networks for brain networks; towards predicting neurodevelopment. Neuroimage 146, 1038–1049. 10.1016/j.neuroimage.2016.09.046
    1. Keller S. S., Ahrens T., Mohammadi S., Möddel G., Kugel H., Ringelstein E. B., et al. . (2011). Microstructural and volumetric abnormalities of the putamen in juvenile myoclonic epilepsy. Epilepsia 52, 1715–1724. 10.1111/j.1528-1167.2011.03117.x
    1. Kesner R. P. (2007). Behavioral functions of the CA3 subregion of the hippocampus. Learn. Mem. 14, 771–781. 10.1101/lm.688207
    1. Kingma D. P., Ba J. (2014). Adam: a method for stochastic optimization. arXiv:1412.6980 [arXiv preprint].
    1. Kwon S. H., Vasung L., Ment L. R., Huppi P. S. (2014). The role of neuroimaging in predicting neurodevelopmental outcomes of preterm neonates. Clin. Perinatol. 41, 257–283. 10.1016/j.clp.2013.10.003
    1. Lee J. C., Nopoulos P. C., Bruce Tomblin J. (2013). Abnormal subcortical components of the corticostriatal system in young adults with DLI: a combined structural MRI and DTI study. Neuropsychologia 51, 2154–2161. 10.1016/j.neuropsychologia.2013.07.011
    1. Li H., Parikh N. A., Wang J., Merhar S., Chen M., Parikh M., et al. . (2019). Objective and automated detection of diffuse white matter abnormality in preterm infants using deep convolutional neural networks. Front. Neurosci. 13:610. 10.3389/fnins.2019.00610
    1. Linsell L., Malouf R., Morris J., Kurinczuk J. J., Marlow N. (2015). Prognostic factors for poor cognitive development in children born very preterm or with very low birth weight: a systematic review. JAMA Pediatr. 169, 1162–1172. 10.1001/jamapediatrics.2015.2175
    1. Linsell L., Malouf R., Morris J., Kurinczuk J. J., Marlow N. (2016). Prognostic factors for cerebral palsy and motor impairment in children born very preterm or very low birthweight: a systematic review. Dev. Med. Child Neurol. 58, 554–569. 10.1111/dmcn.12972
    1. Liu Y., Feng X., Zhou Z. (2016). Multimodal video classification with stacked contractive autoencoders. Signal Process. 120, 761–766. 10.1016/j.sigpro.2015.01.001
    1. Lowe M. J., Sorenson J. A. (1997). Spatially filtering functional magnetic resonance imaging data. Magn. Reson. Med. 37, 723–729. 10.1002/mrm.1910370514
    1. Martin J. A., Hamilton B. E., Osterman M. J. K., Driscoll A. K. (2019). Births: final data for 2018. Natl. Vital Stat. Rep. 68, 1–47.
    1. May A. (2011). Experience-dependent structural plasticity in the adult human brain. Trends Cogn. Sci. 15, 475–482. 10.1016/j.tics.2011.08.002
    1. Menke R. A., Scholz J., Miller K. L., Deoni S., Jbabdi S., Matthews P. M., et al. . (2009). MRI characteristics of the substantia nigra in Parkinson's disease: a combined quantitative T1 and DTI study. Neuroimage 47, 435–441. 10.1016/j.neuroimage.2009.05.017
    1. Moeskops P., Išgum I., Keunen K., Claessens N. H. P., van Haastert I. C., Groenendaal F., et al. . (2017). Prediction of cognitive and motor outcome of preterm infants based on automatic quantitative descriptors from neonatal MR brain images. Sci. Rep. 7:2163. 10.1038/s41598-017-02307-w
    1. Mostapha M., Styner M. (2019). Role of deep learning in infant brain MRI analysis. Magn. Reson. Imaging 64, 171–189. 10.1016/j.mri.2019.06.009
    1. Nordhov S. M., Ronning J. A., Dahl L. B., Ulvund S. E., Tunby J., Kaaresen P. I. (2010). Early intervention improves cognitive outcomes for preterm infants: randomized controlled trial. Pediatrics 126, e1088–e1094. 10.1542/peds.2010-0778
    1. Olden J. D., Jackson D. A. (2002). Illuminating the “black box”: a randomization approach for understanding variable contributions in artificial neural networks. Ecol. Modell. 154, 135–150. 10.1016/S0304-3800(02)00064-9
    1. Olson I. R., Plotzker A., Ezzyat Y. (2007). The Enigmatic temporal pole: a review of findings on social and emotional processing. Brain 130, 1718–1731. 10.1093/brain/awm052
    1. Parikh N. A., He L., Priyanka Illapani V. S., Altaye M., Folger A. T., Yeates K. O. (2020). Objectively diagnosed diffuse white matter abnormality at term is an independent predictor of cognitive and language outcomes in infants born very preterm. J. Pediatr. 220, 56–63. 10.1016/j.jpeds.2020.01.034
    1. Parikh N. A., Lasky R. E., Kennedy K. A., McDavid G., Tyson J. E. (2013). Perinatal factors and regional brain volume abnormalities at term in a cohort of extremely low birth weight infants. PLoS One 8:e62804. 10.1371/journal.pone.0062804
    1. Plis S. M., Hjelm D. R., Salakhutdinov R., Allen E. A., Bockholt H. J., Long J. D., et al. . (2014). Deep learning for neuroimaging: a validation study. Front. Neurosci. 8:229. 10.3389/fnins.2014.00229
    1. Pogribna U., Burson K., Lasky R. E., Narayana P. A., Evans P. W., Parikh N. A. (2014). Role of diffusion tensor imaging as an independent predictor of cognitive and language development in extremely low-birth-weight infants. AJNR Am. J. Neuroradiol. 35, 790–796. 10.3174/ajnr.A3725
    1. Power J. D., Mitra A., Laumann T. O., Snyder A. Z., Schlaggar B. L., Petersen S. E. (2014). Methods to detect, characterize, and remove motion artifact in resting state fMRI. Neuroimage 84, 320–341. 10.1016/j.neuroimage.2013.08.048
    1. Qi J., Peng Y. (2018). Cross-modal bidirectional translation via reinforcement learning, in Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18) (Stockholm: ), 2630–2636.
    1. Rasiwasia N., Costa Pereira J., Coviello E., Doyle G., Lanckriet G. R., Levy R., et al. . (2010). A new approach to cross-modal multimedia retrieval, in Proceedings of the 18th ACM International Conference on Multimedia (Florence: ), 251–260.
    1. Reed S., Akata Z., Yan X., Logeswaran L., Schiele B., Lee H. (2016). Generative adversarial text to image synthesis. arXiv:1605.05396 [arXiv preprint].
    1. Rogers C. E., Smyser T., Smyser C. D., Shimony J., Inder T. E., Neil J. J. (2016). Regional white matter development in very preterm infants: perinatal predictors and early developmental outcomes. Pediatr. Res. 79:87. 10.1038/pr.2015.172
    1. Saha S., Pagnozzi A., Bourgeat P., George J. M., Bradford D., Colditz P. B., et al. . (2020). Predicting motor outcome in preterm infants from very early brain diffusion MRI using a deep learning convolutional neural network (CNN) model. Neuroimage 215:116807. 10.1016/j.neuroimage.2020.116807
    1. Sahiner B., Pezeshk A., Hadjiiski L. M., Wang X., Drukker K., Cha K. H., et al. . (2019). Deep learning in medical imaging and radiation therapy. Med. Phys. 46, e1–e36. 10.1002/mp.13264
    1. Selvaraju R. R., Cogswell M., Das A., Vedantam R., Parikh D., Batra D. (2017). Grad-CAM: visual explanations from deep networks via gradient-based localization, in Proceedings of the IEEE International Conference on Computer Vision (Venice: ), 618–626.
    1. Shi F., Salzwedel A. P., Lin W., Gilmore J. H., Gao W. (2017). Functional brain parcellations of the infant brain and the associated developmental trends. Cereb. Cortex 28, 1358–1368. 10.1093/cercor/bhx062
    1. Shi F., Yap P.-T., Wu G., Jia H., Gilmore J. H., Lin W., et al. . (2011). Infant brain atlases from neonates to 1-and 2-year-olds. PLoS One 6:e18746. 10.1371/journal.pone.0018746
    1. Silk T. J., Vance A., Rinehart N., Bradshaw J. L., Cunnington R. (2009). Structural development of the basal ganglia in attention deficit hyperactivity disorder: a diffusion tensor imaging study. Psychiatry Res. 172, 220–225. 10.1016/j.pscychresns.2008.07.003
    1. Simonyan K., Zisserman A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 [arXiv preprint].
    1. Small S. A., Perera G. M., DeLaPaz R., Mayeux R., Stern Y. (1999). Differential regional dysfunction of the hippocampal formation among elderly with memory decline and Alzheimer's disease. Ann. Neurol. 45, 466–472. 10.1002/1531-8249(199904)45:4<466::AID-ANA8>;2-Q
    1. Snook L., Paulson L. A., Roy D., Phillips L., Beaulieu C. (2005). Diffusion tensor imaging of neurodevelopment in children and young adults. Neuroimage 26, 1164–1173. 10.1016/j.neuroimage.2005.03.016
    1. Sperling R. (2007). Functional MRI studies of associative encoding in normal aging, mild cognitive impairment, Alzheimer's disease. Ann. N. Y. Acad. Sci. 1097, 146–155. 10.1196/annals.1379.009
    1. Srivastava N., Hinton G., Krizhevsky A., Sutskever I., Salakhutdinov R. (2014). Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958.
    1. Stam C. J., van Straaten E. C. W., Van Dellen E., Tewarie P., Gong G., Hillebrand A., et al. . (2016). The relation between structural and functional connectivity patterns in complex brain networks. Int. J. Psychophysiol. 103, 149–160. 10.1016/j.ijpsycho.2015.02.011
    1. Thompson D. K., Chen J., Beare R., Adamson C. L., Ellis R., Ahmadzai Z. M., et al. . (2016). Structural connectivity relates to perinatal factors and functional impairment at 7 years in children born very preterm. Neuroimage 134, 328–337. 10.1016/j.neuroimage.2016.03.070
    1. Thompson D. K., Lee K. J., Egan G. F., Warfield S. K., Doyle L. W., Anderson P. J., et al. . (2014). Regional white matter microstructure in very preterm infants: predictors and 7 year outcomes. Cortex 52, 60–74. 10.1016/j.cortex.2013.11.010
    1. Thompson P. M., Cannon T. D., Narr K. L., Van Erp T., Poutanen V.-P., Huttunen M., et al. . (2001). Genetic influences on brain structure. Nat. Neurosci. 4:1253. 10.1038/nn758
    1. Valizadeh S. A., Liem F., Mérillat S., Hänggi J., Jäncke L. (2018). Identification of individual subjects on the basis of their brain anatomical features. Sci. Rep. 8:5611. 10.1038/s41598-018-23696-6
    1. Wang R., Benner T., Sorensen A. G., Wedeen V. J. (2007). “Diffusion toolkit: a software package for diffusion imaging data processing and tractography,” Proceedings of the International Soceity for Magnetic Resonance in Medicine (Berlin).
    1. Wee C.-Y., Yap P.-T., Zhang D., Denny K., Browndyke J. N., Potter G. G., et al. . (2012). Identification of MCI individuals using structural and functional connectivity networks. Neuroimage 59, 2045–2056. 10.1016/j.neuroimage.2011.10.015
    1. Whitfield-Gabrieli S., Nieto-Castanon A. (2012). Conn: a functional connectivity toolbox for correlated and anticorrelated brain networks. Brain Connect. 2, 125–141. 10.1089/brain.2012.0073
    1. Woolrich M. W., Jbabdi S., Patenaude B., Chappell M., Makni S., Behrens T., et al. . (2009). Bayesian analysis of neuroimaging data in FSL. Neuroimage 45, S173–S186. 10.1016/j.neuroimage.2008.10.055
    1. Wu L., Wang Y., Shao L. (2018). Cycle-consistent deep generative hashing for cross-modal retrieval. IEEE Trans. Image Process. 28, 1602–1612. 10.1109/TIP.2018.2878970
    1. Wu S., Bondugula S., Luisier F., Zhuang X., Natarajan P. (2014). Zero-shot event detection using multi-modal fusion of weakly supervised concepts, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Columbus, OH: ), 2665–2672.
    1. Xu K., Ba J., Kiros R., Courville A., Salakhutdinov R., Zemel R., et al. . (2015). Show, attend and tell: Neural image caption generation with visual attention. arXiv:1502.03044 [arXiv preprint].
    1. Xu T., Zhang H., Huang X., Zhang S., Metaxas D. N. (2016). Multimodal deep learning for cervical dysplasia diagnosis, in International Conference on Medical Image Computing and Computer-Assisted Intervention, eds. Ourselin S., Joskowicz L., Sabuncu M., Unal G., Wells W. (Cham: Springer; ), 115–123.

Source: PubMed

Patrocinadores e Colaboradores

Condições médicas

Intervenções de drogas

3
Se inscrever