Abstract
In recent years, deep learning has become a powerful tool for medical image analysis mainly because of their ability to automatically extract more abstract features from large training data. The current methods used for multiple modalities are mostly conventional machine learning, in which people use the handcrafted feature, which is very difficult to construct for large training sizes. Deep learning which is an advancement in the machine learning automatically extracts relevant features from the data. In this paper, we have used deep learning model for the multimodal data. The basic building blocks of the network are stacked autoencoder for the multiple modalities. The performance of deep learning-based models with and without multimodal fusion and shared learning are compared. The results indicates that the use of multimodal fusion and shared learning help to improve deep learning-based medical image analysis.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural network. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Howard, A.G.: Some Improvements on Deep Convolutional Neural Network Based Image Classification (2013). arXiv preprint arXiv:1312.5402
Szegedy, C., Toshev, A., Erhan, D.: Deep neural networks for object detection. In: Advances in Neural Information Processing Systems, pp. 2553–2561 (2013)
Zhang, J., Zong, C.: Deep Neural Networks in Machine Translation: An Overview (2015)
Edelman, R.R., Warach, S.: Magnetic resonance imaging. New England Journal of Medicine 328(11), 785–791 (1993)
Ogawa, S., Lee, T.M., Kay, A.R., Tank, D.W.: Brain magnetic resonance imaging with contrast dependent on blood oxygenation. Proc. Natl. Acad. Sci. 87(24), 9868–9872 (1990)
van Gerven, M.A., de Lange, F.P., Heskes, T.: Neural decoding with hierarchical generative models. Neural Comput. 22(12), 3127–3142 (2010)
Bailey, D.L., Townsend, D.W., Valk, P.E., Maisey, M.N.: Positron Emission Tomography. Springer, London (2005)
Hsieh, J.: Computed Tomography: Principles, Design, Artifacts, and Recent Advances. SPIE Bellingham, WA (2009)
Chapman, D., Thomlinson, W., Johnston, R. et al.: Diffraction enhanced x-ray imaging. Phys. Med. Biol. 42(11), 2015 (1997)
Gurcan, M.N., Boucheron, L.E., Can, A., Madabhushi, A., Rajpoot, N.M., Yener, B.: Histopathological image analysis: a review. IEEE Rev. Biomed. Eng. 2, 147–171 (2009)
Plis, S.M., Hjelm, D.R., Salakhutdinov, R., Allen, E.A., Bockholt, H.J., Long, J.D., Johnson, H.J., Paulsen, J.S., Turner, J.A., Calhoun, V.D.: Deep learning for neuroimaging: a validation study. Front. Neurosci. 8 (2014)
Hua, K.L., Hsu, C.H., Hidayati, S.C., Cheng, W.H., Chen, Y.J.: Computer-aided classification of lung nodules on computed tomography images via deep learning technique. Onco Targets Ther. 8 (2015)
Ciresan, D., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Deep neural networks segment neuronal membranes in electron microscopy images. In: Advances in Neural Information Processing Systems, pp. 2843–2851 (2012)
Roth, H.R., Lu, L., Farag, A., Shin, H.C., Liu, J., Turkbey, E.B., Summers, R.M.: Deeporgan: multi-level deep convolutional networks for automated pancreas segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 556–564. Springer, Cham (2015)
Xu, J., Xiang, L., Liu, Q., Gilmore, H., Wu, J., Tang, J., Madabhushi, A.: Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images. IEEE Trans. Med. Imaging 35(1), 119–130 (2016)
Koyamada, S., Shikauchi, Y., Nakae, K., Koyama, M., Ishii, S.: Deep Learning of fMRI Big Data: A Novel Approach to Subject-Transfer Decoding (2015). arXiv preprint arXiv:1502.00093
Zhang, D., Shen, D., Alzheimer’s Disease Neuroimaging Initiative: Multi-modal multi-task learning for joint prediction of multiple regression and classification variables in Alzheimer’s disease. NeuroImage 59(2), 895–907 (2012)
Zhang, D., Wang, Y., Zhou, L., Yuan, H., Shen, D., Alzheimer’s Disease Neuroimaging Initiative: Multimodal classification of Alzheimer’s disease and mild cognitive impairment. Neuroimage 55(3), 856–867 (2011)
Sevakula, R.K., Verma, N.K.: Assessing generalization ability of majority vote point classifiers. IEEE Trans. Neural Netw. Learn. Syst. 28(12), 2985–2997 (2017)
Gray, K.R., Aljabar, P., Heckemann, R.A., Hammers, A., Rueckert, D., Alzheimer’s Disease Neuroimaging Initiative: Random forest-based similarity measures for multi-modal classification of Alzheimer’s disease. NeuroImage 65, 167–175 (2013)
Singh, V., Gupta, R.K., Sevakula, R.K., Verma, N.K.: Comparative analysis of Gaussian mixture model, logistic regression and random forest for big data classification using map reduce. In: 11th IEEE International Conference on Industrial and Information Systems (ICIIS), pp. 333–338. IEEE (2016)
Cheng, B., Zhang, D., Chen, S., Kaufer, D.I., Shen, D., Alzheimers Disease Neuroimaging Initiative: Semi-supervised multimodal relevance vector regression improves cognitive performance estimation from imaging and biological biomarkers. Neuroinformatics 11(3), 339–353 (2013)
Young, J., Modat, M., Cardoso, M.J., Mendelson, A., Cash, D., Ourselin, S., Alzheimer’s Disease Neuroimaging Initiative: Accurate multimodal probabilistic prediction of conversion to Alzheimer’s disease in patients with mild cognitive impairment. NeuroImage: Clin. 2, 735–745 (2013)
Ngiam, J., Khosla, A., Kim, M., Nam, J., Lee, H., Ng, A.Y.: Multimodal deep learning. In: Proceedings of the 28th International Conference on Machine Learning (ICML-11), pp. 689–696 (2011)
Srivastava, N., Salakhutdinov, R.R.: Multimodal learning with deep boltzmann machines. In: Advances in Neural Information Processing Systems, pp. 2222–2230 (2012)
Suk, H.I., Lee, S.W., Shen, D., Alzheimer’s Disease Neuroimaging Initiative: Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis. NeuroImage 101, 569–582 (2014)
Cao, Y., Steffey, S., He, J., Xiao, D., Tao, C., Chen, P., Mller, H.: Medical image retrieval: a multimodal approach. Cancer Inf. 13(Suppl 3), 125 (2014)
Singh, V., Baranwal, N., Sevakula, R.K., Verma, N.K., Cui, Y.: Layerwise feature selection in Stacked Sparse Auto-Encoder for tumor type prediction. In: 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 15421548. IEEE (2016)
Singh, V., Verma, N.K.: Deep learning architecture for high-level feature generation using stacked auto encoder for business intelligence. Complex Systems: Solutions and Challenges in Economics, Management and Engineering. Springer International Publishing, Basel (2017) (Accepted)
Sevakula, R.K., Thirukovalluru, R., Verma, N.K., Cui, Y.: Deep neural networks for transcriptome based cancer classification. BMC Bioinformatics (2017) (Accepted)
Rajurkar, S., Singh, V., Verma, N.K., Cui, Y.: Deep stacked auto-encoder with deep fuzzy network for transcriptome based tumor type classification. BMC Bioinformatics (2017) (Accepted)
Sevakula, R.K., Singh, V., Verma, N.K., Kumar, C., Cui, Y.: Transfer learning for molecular cancer classification using deep neural networks. IEEE/ACM Trans. Comput. Biol. Bioinform. (1), 1–1 (2018)
Hinton, G.E., Osindero, S., Teh, Y.-W.: A fast learning algorithm for deep belief nets. Neural comput. 18(7), 1527–1554 (2006)
Broyden, C.G.: The convergence of a class of double-rank minimization algorithms. J. Inst. Maths. Appl. 6, 76–90 (1970)
Fletcher, R.: A new approach to variable metric algorithms. Comput. J. 13, 317–322 (1970)
Goldfarb, D.: A family of variable metric updates derived by variational means. Math. Comput. 24, 23–26 (1970)
Shanno, D.F.: Conditioning of quasi-Newton methods for function minimization. Math. Comput. 24, 647–656 (1970)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Singh, V., Verma, N.K., Ul Islam, Z., Cui, Y. (2019). Feature Learning Using Stacked Autoencoder for Shared and Multimodal Fusion of Medical Images. In: Verma, N., Ghosh, A. (eds) Computational Intelligence: Theories, Applications and Future Directions - Volume I. Advances in Intelligent Systems and Computing, vol 798. Springer, Singapore. https://doi.org/10.1007/978-981-13-1132-1_5
Download citation
DOI: https://doi.org/10.1007/978-981-13-1132-1_5
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-1131-4
Online ISBN: 978-981-13-1132-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)