Abstract
Stacking layers of denoising autoencoders, which are trained to reconstruct corrupted versions of their inputs, results in a type of deep neural network architecture called stacked denoising autoencoders. This paper introduces a model of complex-valued stacked denoising autoencoders, which can be used to build complex-valued deep neural networks. Experiments done using the MNIST and FashionMNIST datasets show superior performance of the complex-valued stacked denoising autoencoders with respect to their real-valued counterparts, both in terms of reconstruction error, and in terms of classification error.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Arjovsky, M., Shah, A., Bengio, Y.: Unitary evolution recurrent neural networks. In: ICML 2016 (2016). https://arxiv.org/abs/1511.06464
Baldi, P., Lu, Z.: Complex-valued autoencoders. Neural Netw. 33, 136–147 (2012)
Bengio, Y.: Learning deep architectures for AI. Found. Trends Mach. Learn. 2(1), 1–127 (2009)
Bengio, Y., LeCun, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015)
Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H.: Greedy layer-wise training of deep networks. In: NIPS 2006, pp. 153–160 (2006)
Bruna, J., Mallat, S.: Invariant scattering convolution networks. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1872–1886 (2013)
Erhan, D., Bengio, Y., Courville, A., Manzagol, P.A., Vincent, P., Bengio, S.: Why does unsupervised pre-training help deep learning? J. Mach. Learn. Res. 11, 625–660 (2010)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)
Guberman, N.: On complex valued convolutional neural networks. Master’s thesis, School of Computer Science and Engineering, The Hebrew University of Jerusalem (2016). https://arxiv.org/abs/1602.09046
Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)
Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)
Jing, L., Shen, Y., Dubcek, T., Peurifoy, J., Skirlo, S., LeCun, Y., Tegmark, M., Soljacic, M.: Tunable efficient unitary neural networks (EUNN) and their application to RNNs. In: ICML 2017 (2017). https://arxiv.org/abs/1612.05231
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR 2015 (2015)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Popa, C.A.: Complex-valued convolutional neural networks for real-valued image classification. In: 2017 International Joint Conference on Neural Networks (IJCNN). IEEE, May 2017
Sarroff, A.M., Shepardson, V., Casey, M.A.: Learning representations using complex-valued nets (2015). https://arxiv.org/abs/1511.06351
Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)
Tygert, M., Bruna, J., Chintala, S., LeCun, Y., Piantino, S., Szlam, A.: A mathematical motivation for complex-valued convolutional networks. Neural Comput. 28(5), 815–825 (2016)
Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.A.: Extracting and composing robust features with denoising autoencoders. In: ICML 2008 (2008)
Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.A.: Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371–3408 (2010)
Wisdom, S., Powers, T., Hershey, J.R., Roux, J.L., Atlas, L.: Full-capacity unitary recurrent neural networks. In: NIPS 2016 (2016). https://arxiv.org/abs/1611.00035
Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine and Learning Algorithms (2017). https://arxiv.org/abs/1708.07747
Acknowledgement
This work was supported by research grant no. PCD-TC-2017-41 of the Polytechnic University Timişoara, Romania.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Popa, CA. (2018). Complex-Valued Stacked Denoising Autoencoders. In: Huang, T., Lv, J., Sun, C., Tuzikov, A. (eds) Advances in Neural Networks – ISNN 2018. ISNN 2018. Lecture Notes in Computer Science(), vol 10878. Springer, Cham. https://doi.org/10.1007/978-3-319-92537-0_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-92537-0_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-92536-3
Online ISBN: 978-3-319-92537-0
eBook Packages: Computer ScienceComputer Science (R0)