Improving Deep Neural Network Performance by Reusing Features Trained with Transductive Transference
Transfer Learning is a paradigm in machine learning to solve a target problem by reusing the learning with minor modifications from a different but related source problem. In this paper we propose a novel feature transference approach, especially when the source and the target problems are drawn from different distributions. We use deep neural networks to transfer either low or middle or higher-layer features for a machine trained in either unsupervised or supervised way. Applying this feature transference approach on Convolutional Neural Network and Stacked Denoising Autoencoder on four different datasets, we achieve lower classification error rate with significant reduction in computation time with lower-layer features trained in supervised way and higher-layer features trained in unsupervised way for classifying images of uppercase and lowercase letters dataset.
KeywordsFeature Transference Deep Neural Network
Unable to display preview. Download preview PDF.
- 1.Vapnik, V.N.: An overview of statistical learning theory. IEEE Transactions on Neural Networks / a Publication of the IEEE Neural Networks Council 10(5), 988–999 (1999)Google Scholar
- 2.Bengio, Y., Courville, A., Vincent, P.: Representation learning: A review and new perspectives (2013)Google Scholar
- 3.Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.A.: Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371–3408 (2010); Cited by 0083Google Scholar
- 4.LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11), 2278–2324 (1998); Cited by 2008Google Scholar
- 6.Amaral, T., Silva, L.M., Alexandre, L.A., Kandaswamy, C., Santos, J.M., de Sá, J.M.: Using different cost functions to train stacked auto-encoders. In: Proceedings of the 12th Mexican International Conference on Artificial Intelligence. IEEE (2013)Google Scholar
- 8.de Campos, T., Babu, B.R., Varma, M.: Character recognition in natural images (2009)Google Scholar
- 9.Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., Warde-Farley, D., Bengio, Y.: Theano: a CPU and GPU math expression compiler. In: Proceedings of the Python for Scientific Computing Conference (SciPy), vol. 4 (2010)Google Scholar