Abstract
Networks used in Deep Learning generally have feedforward architectures, and they can not use top-down information for recognition. In this paper, we propose Bayesian AutoEncoder (BAE) in order to use top-down information for recognition. BAE constructs a generative model represented as a Bayesian Network, and the networks constructed by BAE behave as Bayesian Networks. The network can execute inference for each stochastic variable through belief propagation, using both bottom-up information and top-down information. We confirmed that BAE can construct small networks with one latent layer and extract features in 3\(\,\times \,\)3 pixel input data as latent variables.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Hinton, G.: Boltzmann machines. Scholarpedia
Hinton, G., Salakhutdinov, R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105. Curran Associates, Inc. (2012). http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
Le, Q.: Building high-level features using large scale unsupervised learning. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8595–8598, May 2013
Lee, T.S., Mumford, D.: Hierarchical bayesian inference in the visual cortex (2002)
Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996)
Pearl, J.: Bayesian networks: a model of self-activated memory for evidential reasoning. In: Cognitive Science Society, pp. 329–334 (1985)
Ranzato, M., Poultney, C.S., Chopra, S., LeCun, Y.: Efficient learning of sparse representations with an energy-based model. In: NIPS, pp. 1137–1144. MIT Press (2006)
Shon, A.P., Rao, R.P.: Implementing belief propagation in neural circuits. Neurocomputing 65–66, 393–399 (2005). Computational Neuroscience: Trends in Research 2005
Yedidia, J.S., Freeman, W., Weiss, Y.: Constructing free-energy approximations and generalized belief propagation algorithms. IEEE Trans. Inf. Theory 51(7), 2282–2312 (2005)
Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H.: Greedy layer-wise training of deep networks, pp. 153–160 (2007)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Nishino, K., Inaba, M. (2015). Feature Extraction Based on Generating Bayesian Network. In: Arik, S., Huang, T., Lai, W., Liu, Q. (eds) Neural Information Processing. ICONIP 2015. Lecture Notes in Computer Science(), vol 9492. Springer, Cham. https://doi.org/10.1007/978-3-319-26561-2_31
Download citation
DOI: https://doi.org/10.1007/978-3-319-26561-2_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26560-5
Online ISBN: 978-3-319-26561-2
eBook Packages: Computer ScienceComputer Science (R0)