Abstract
This work introduces a novel Stacked Multimodal (SM2N2) architecture and assess its performance in classifying whether a patient have or not Myocardial Infarction. Central to this SM2N2 architecture is the use of images and clinical data as input. Comparison studies of Multimodal Neural Network (M2N2) component of SM2N2 with AlexNet3D model demonstrated that on small size dataset, M2N2 is faster, has less trainable parameters and results higher accuracy in this binary classification. In addition to M2N2 we also identify clinical features that are sufficient to classify normal vs pathological cases. We also train statistical models on identified clinical features and use stacking to combine outputs from statistical models and M2N2. Stacking generalizes the results and the new model learns how to best combine the results of the individual base models. One of the potential application of the M2N2 is that because of less parameters the network can be deployed on mobile devices for inference.
This work was supported by the National Science Foundation award CNS-1646566. All opinions, findings, conclusions or recommendations expressed in this work are those of the authors and do not necessarily reflect the views of our sponsors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Lalande, A., et al.: Emidec: a database usable for the automatic evaluation of myocardial infarction from delayed-enhancement cardiac MRI. Data 5(4), 89 (2020)
CDC. https://www.cdc.gov/heartdisease/facts.htm. Last accessed 17 Aug 2020
Chollet, F.: Xception: deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 1251–1258 (2017)
Agarap, A.F.: Deep learning using rectified linear units (ReLU) (2018). arXiv preprint arXiv:1803.08375
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift (2015). arXiv preprint arXiv:1502.03167
Hochreiter, S.: The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int. J. Uncertainty Fuzziness Knowl.-Based Syst. 6(02), 107–116 (1998)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Zhang, Z., Mert S.: Generalized cross entropy loss for training deep neural networks with noisy labels. In: Advances in Neural Information Processing Systems, pp. 8778–8788 (2018)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)
Howard, A.G., et al.: Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)
Polat, H., Danaei Mehr, H.: Classification of pulmonary CT images by using hybrid 3D-deep convolutional neural network architecture. Appl. Sci. 9(5), 940 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Sharma, R., Eick, C.F., Tsekos, N.V. (2021). SM2N2: A Stacked Architecture for Multimodal Data and Its Application to Myocardial Infarction Detection. In: Puyol Anton, E., et al. Statistical Atlases and Computational Models of the Heart. M&Ms and EMIDEC Challenges. STACOM 2020. Lecture Notes in Computer Science(), vol 12592. Springer, Cham. https://doi.org/10.1007/978-3-030-68107-4_35
Download citation
DOI: https://doi.org/10.1007/978-3-030-68107-4_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68106-7
Online ISBN: 978-3-030-68107-4
eBook Packages: Computer ScienceComputer Science (R0)
