Greedy Training

Abstract

The algorithm for stacking RBMs is surprisingly simple. Roughly stated, you train the bottom-most RBM, the one whose input is the training data. Once that’s trained, you run the training cases through this model and use its hidden-layer activations as inputs to the next RBM, which you then train. When it is trained, you run the training data through the first and second RBMs and use the second’s hidden activation as inputs for the third, which is then trained, and so on.