Abstract
Recently, Restricted Boltzmann Machine (RBM) has demonstrated excellent capacity of modelling vector variable. A variant of RBM, Matrix-variate Restricted Boltzmann Machine (MVRBM), extends the ability of RBM and is able to model matrix-variate data directly without vectorized process. However, MVRBM is still an unsupervised generative model, and is usually used to feature extraction or initialization of deep neural network. When MVRBM is used to classify, additional classifiers are necessary. This paper proposes a Matrix-variate Restricted Boltzmann Machine Classification Model (ClassMVRBM) to classify 2D data directly. In the novel ClassMVRBM, classification constraint is introduced to MVRBM. On one hand, the features extracted by MVRBM are more discriminative, on the other hand, the proposed model can be directly used to classify. Experiments on some publicly available databases demonstrate that the classification performance of ClassMVRBM has been largely improved, resulting in higher image classification accuracy than conventional unsupervised RBM, its variants and Restricted Boltzmann Machine Classification Model (ClassRBM).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Wang, H., Wang, J.: 2DPCA with L1-norm for simultaneously robust and sparse modelling. Neural Netw. 46(10), 190–198 (2013)
Ju, F., Sun, Y., Gao, J., Hu, Y., Yin, B.: Image outlier detection and feature extraction via L1-norm-based 2D probabilistic PCA. IEEE Trans. Image Process. 24(12), 4834–4846 (2015)
Li, M., Yuan, B.: 2D-LDA: a statistical linear discriminant analysis for image matrix. Pattern Recogn. Lett. 26(5), 527–532 (2005)
Wang, J., Wang, W., Wang, R., Gao, W.: Image classification using RBM to encode local descriptors with group sparse learning. In: Proceedings of International Conference on Image Processing, pp. 912–916. IEEE, Canada (2015)
Dahl, G.E., Dong, Y., Li, D., Acero, A.: Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. IEEE Trans. Audio Speech Lang. Process. 20(1), 30–42 (2011)
Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning, Israel, pp. 807–814 (2010)
Cho, K., Ilin, A., Raiko, T.: Improved learning of Gaussian-Bernoulli restricted Boltzmann machines. In: Honkela, T., Duch, W., Girolami, M., Kaski, S. (eds.) ICANN 2011. LNCS, vol. 6791, pp. 10–17. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21735-7_2
Nguyen, T., Tran, T., Phung, D., Venkatesh, S.: Tensor-variate restricted Boltzmann machines. In: Proceedings of the Twenty-Ninth National Conference on Artificial Intelligence, pp. 2887–2893. AAAI, USA (2015)
Larochelle, H., Mandel, M., Pascanu, R., et al.: Learning algorithms for the classification restricted Boltzmann machine. J. Mach. Learn. Res. 13(1), 643–669 (2012)
Peng, X., Gao, X., Li, X.: An infinite classification RBM model for radar HRRP recognition. In: International Joint Conference on Neural Networks, pp. 1442–1448, IEEE, USA (2017)
Qi, G., Sun, Y., Gao, J., Hu, Y., Li, J.: Matrix variate restricted Boltzmann machine. In: The proceeding of 2016 International Joint Conference on Neural Networks, pp. 389–395. IEEE, Canada (2016)
Liu, S., Sun, Y., Hu, Y., Gao, J., Ju, F., Yin, B.: Matrix variate RBM model with Gaussian distributions. In: The proceeding of 2017 International Joint Conference on Neural Networks, pp. 808–815. IEEE, USA (2017)
Gao, J., Guo, Y., Wang, Z.: Matrix neural networks. In: Cong, F., Leung, A., Wei, Q. (eds.) ISNN 2017. LNCS, vol. 10261, pp. 313–320. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59072-1_37
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Wang, Y., Mori, G.: Human action recognition by semilatent topic models. IEEE Trans. Pattern Anal. Mach. Intell. 31(10), 1762 (2009)
Leibe, B., Schiele, B.: Analyzing appearance and contour based methods for object categorization. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1–7. IEEE, USA (2003)
Tenenbaum, J., Silva, V., Langford, J.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)
Nene, S., Nayar, S., Murase, H.: Columbia object image library (COIL-20). Technical report CUCS-005-96, USA (1996)
Qi, N., Shi, Y., Sun, X., Wang, J., Yin, B., Gao, J.: Multi-dimensional sparse models. IEEE Trans. Pattern Anal. Mach. Intell. 40(1), 163–178 (2018)
Acknowledgments
This research is supported by NSFC (No.61772049, 61602486), BJNSF (No.4162009), Beijing Educational Committee (No. KM201710005022) and Beijing Key Laboratory of Computational Intelligence and Intelligent System.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Li, J., Tian, P., Kong, D., Wang, L., Wang, S., Yin, B. (2019). Matrix-Variate Restricted Boltzmann Machine Classification Model. In: Song, H., Jiang, D. (eds) Simulation Tools and Techniques. SIMUtools 2019. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 295. Springer, Cham. https://doi.org/10.1007/978-3-030-32216-8_47
Download citation
DOI: https://doi.org/10.1007/978-3-030-32216-8_47
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32215-1
Online ISBN: 978-3-030-32216-8
eBook Packages: Computer ScienceComputer Science (R0)