Add-if-Silent Rule for Training Multi-layered Convolutional Network Neocognitron

Abstract

The neocognitron is a multi-layered convolutional network that can be trained to recognize visual patterns robustly. This paper discusses a new neocognitron, which uses the add-if-silent rule for training intermediate layers and the method of interpolating-vector for classifying patterns at the highest stage of the hierarchical network. By the add-if-silent rule, a new cell is generated when all postsynaptic cells are silent. The generated cell learns the activity of the presynaptic cells in one-shot, and its input connections will never be modified afterward. Thus the training process is very simple, and does not require time-consuming calculation such as the gradient descent process. This paper analyzes how the size of training set affects the performance of the neocognitron and show that the add-if-silent rule can produce feature-extracting cells efficiently even with a small number of training patterns.