Improving Deep Learning Accuracy with Noisy Autoencoders Embedded Perturbative Layers

Abstract

Autoencoder has been successfully used as an unsupervised learning framework to learn some useful representations in deep learning tasks. Based on it, a wide variety of regularization techniques have been proposed such as early stopping, weight decay and contraction. This paper presents a new training principle for autoencoder based on denoising autoencoder and dropout training method. We extend denoising autoencoder by both partial corruption of the input pattern and adding noise to its hidden units. This kind of noisy autoencoder can be stacked to initialize deep learning architectures. Moreover, we show that in the full noisy network the activations of hidden units are sparser. Furthermore, the method significantly improves learning accuracy when conducting classification experiments on benchmark data sets.