Abstract
This paper proposes an improved add-if-silent rule, which is suited for training intermediate layers of a multi-layered convolutional network, such as a neocognitron. By the add-if-silent rule, a new cell is generated if all postsynaptic cells are silent. The generated cell learns the activity of the presynaptic cells in one-shot, and its input connections will never be modified afterward. To use this learning rule for a convolutional network, it is required to decide at which retinotopic location this rule is to be applied. In the conventional add-if-silent rule, we chose the location where the activity of presynaptic cells is the largest. In the proposed new learning rule, a negative feedback is introduced from postsynaptic cells to presynaptic cells, and a new cell is generated at the location where the presynaptic activity fails to be suppressed by the feedback. We apply this learning rule to a neocognitron for hand-written digit recognition, and demonstrate the decrease in the recognition error.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Hinton, G.E., Osindero, S., Teh, Y.: A Fast Learning Algorithm for Deep Belief Nets. Neural Computation 18, 1527–1554 (2006)
Fukushima, K.: Training Multi-layered Neural Network Neocognitron. Neural Networks 40, 18–31 (2013)
Fukushima, K.: Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position. Biol. Cybernetics 36(4), 193–202 (1980)
Fukushima, K.: Artificial Vision by Multi-layered Neural Networks: Neocognitron and its Advances. Neural Networks 37, 103–119 (2013)
Fukushima, K.: Interpolating Vectors for Robust Pattern Recognition. Neural Networks 20(8), 904–916 (2007)
ETL1 database, http://projects.itri.aist.go.jp/etlcdb/
MNIST database, http://yann.lecun.com/exdb/mnist/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Fukushima, K. (2014). One-Shot Learning with Feedback for Multi-layered Convolutional Network. In: Wermter, S., et al. Artificial Neural Networks and Machine Learning – ICANN 2014. ICANN 2014. Lecture Notes in Computer Science, vol 8681. Springer, Cham. https://doi.org/10.1007/978-3-319-11179-7_37
Download citation
DOI: https://doi.org/10.1007/978-3-319-11179-7_37
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11178-0
Online ISBN: 978-3-319-11179-7
eBook Packages: Computer ScienceComputer Science (R0)