Abstract
Many learning algorithms can suffer from a performance bias for classification with imbalanced data. This paper proposes the pre-training the deep structure neural network by restricted Boltzmann machine (RBM) learning algorithm, which is pre-sampled with standard SMOTE methods for imbalanced data classification. Firstly, a new training data set can be generated by a pre-sampling method from original examples; secondly the deep neural network structure is trained on the sampled data and all unlabelled data sets by RBM greedy algorithm, which is called “coarse tuning”. Then the neural networks are fined tuned by BP algorithm. The effectiveness of the RBM pre-training neural network (RBMPT) classifier is demonstrated on a number of benchmark data sets. Compared with only BP classifier, pre-sampling BP classifier and RBMPT classifier, it has shown that pre-training procedure can learn more representations of data better with unlabelled data and has better classification performance for classification with imbalanced data sets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Krizhevsky, A., Sutskever, I., Hinton, G.: ImageNet classification with deep convolutional neural networks. In: Proceedings of the Advances in Neural Information Processing Systems, vol. 25, pp. 1090–1098 (2012)
Hinton, G., et al.: Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Process. Mag. 29, 82–97 (2012)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323, 533–536 (1986)
Hinton, G.E., Osindero, S., Teh, Y.-W.: A fast learning algorithm for deep belief nets. Neural Comput. 18, 1527–1554 (2006)
Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H.: Greedy layer-wise training of deep networks. In: Proceedings of the Advances in Neural Information Processing Systems, vol. 19, pp. 153–160 (2006)
Hinton, G.E., Salakhutdinov, R.: Reducing the dimensionality of data with neural networks. Science 313, 504–507 (2006)
Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35, 1798–1828 (2013)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)
Salakhutdinov, R., Hinton, G.: Deep Boltzmann machines. In: Proceedings of the International Conference on Artificial Intelligence and Statistics, pp. 448–455 (2009)
Munder, S., Gavrila, D.: An experimental study on pedestrian classification. IEEE Trans. Pattern Anal. Mach. Intell. 28(11), 1863–1868 (2006)
Japkowicz, N., Stephen, S.: The class imbalance problem: A systematic study. Intell. Data Anal. 6(5), 429–449 (2002)
Barandela, R., Sanchez, J.S., Garcia, V., Rangel, E.: Strategies for learning in class imbalance problems. Pattern Recogn. 36(3), 849–851 (2003)
Chawla, N.V., Bowyer, K.W., Hall, L.O., Keglmeyer, W.P.: SMOTE: Synthetic minority over-sampling technique. J. Artif. Intell. Res. 16(1), 321–357 (2002)
Fu, X., Zhang, S.: Evolving neural network ensembles using variable string genetic algorithm for pattern classification. In: Proceedings of the 6th International Conference on Advanced Computational Intelligence, pp. 81–85 (2013)
Chen, H., Yao, X.: Regularized negative correlation learning for neural network ensembles. IEEE Trans. Neural Netw. 20(12), 1962–1979 (2009)
Asuncion, A., Newman, J.: UCI Machine learning Repository (2007). http://www.ics.uci.edu/~learn/MLRespository.html
Leung, M.K., Xiong, H.Y., Lee, L.J., Frey, B.J.: Deep learning of the tissue-regulated splicing code. Bioinformatics 30, i121–i129 (2014)
Collobert, R., et al.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)
Bhowan, U., Johnston, M., Zhang, M., Yao, X.: Evolving diverse ensembles using genetic programming for classification with unbalanced data. IEEE Trans. Evol. Comput. 17(3), 368–386 (2013)
Weiss, G.M., Provost, F.: Learning when training data are costly: The effect of class distribution on tree induction. J. Artif. Intell. Res. 19, 315–354 (2003)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Fu, X. (2017). Unsupervised Pre-training Classifier Based on Restricted Boltzmann Machine with Imbalanced Data. In: Qiu, M. (eds) Smart Computing and Communication. SmartCom 2016. Lecture Notes in Computer Science(), vol 10135. Springer, Cham. https://doi.org/10.1007/978-3-319-52015-5_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-52015-5_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-52014-8
Online ISBN: 978-3-319-52015-5
eBook Packages: Computer ScienceComputer Science (R0)