Abstract
General vector machine (GVM) is one of supervised learning machine, which is based on three-layer neural network. It is capable of constructing a learning model with limited amount of data. Generally, it employs Monte Carlo algorithm (MC) to adjust weights of the underlying network. However, GVM is time-consuming at training and is not efficient when compared with other learning algorithm based on gradient descent learning. In this paper, we present a derivative-based Monte Carlo algorithm (DMC) to accelerate the training of GVM. Our experimental results indicate that DMC algorithm is faster than the original MC method. Specifically, the training time of our DMC algorithm in GVM for function fitting is also less than some gradient descent-based methods, in which we compare DMC with back-propagation neural network. Experimental results indicate that our algorithm is promising for training GVM.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abdalla A, Buckley JJ (2007) Monte carlo methods in fuzzy linear regression. Soft Comput 11(10):991–996
Abdalla A, Buckley JJ (2008) Monte carlo methods in fuzzy linear regression ii. Soft Comput 12(5):463–468. doi:10.1007/s00500-007-0179-6
Chen H, Zhao H, Shen J, Zhou R, Zhou Q (2015) Supervised machine learning model for high dimensional gene data in colon cancer detection. In: IEEE BigData congress, pp 134–141
Freitas JFGD, Niranjan MA, Gee AH, Doucet A (2000) Sequential monte carlo methods to train neural network models. Neural Comput 12(4):955
Hagan MT, Demuth HB, Beale MH (1995) Neural network design. PWS Publishing Company, Boston, MA
İçen D, Cattaneo MEGV (2016) Different distance measures for fuzzy linear regression with Monte Carlo methods. Soft Comput. doi:10.1007/s00500-016-2218-7
Ji S, Xu W, Yang M, Yu K (2013) 3D convolutional neural networks for human action recognition. IEEE Trans Pattern Anal Mach Intell 35(1):221–231. doi:10.1109/TPAMI.2012.59
Kreinovich V, Sirisaengtaksin O (1993) 3-layer neural networks are universal approximators for functionals and for control strategies. Neural Parallel Sci Comput 1(3):325–346
Krizhevsky A, Sutskever I, Hinton GE (2012a) Imagenet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in neural information processing systems, vol 25. Curran Associates, Inc., Red Hook, pp 1097–1105
Krizhevsky A, Sutskever I, Hinton GE (2012b) Imagenet classification with deep convolutional neural networks. In: International conference on neural information processing systems, pp 1097–1105
Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324
Liang F (2007) Annealing stochastic approximation monte carlo algorithm for neural network training. Mach Learn 68(3):201–233
Wang H, Raj B (2017) On the origin of deep learning. arXiv preprint
Wang L, Shen J, Zhou Q, Shang Z, Chen H, Zhao H (2016) An evaluation of the dynamics of diluted neural network. Int J Comput Intell Syst 9(6):1191–1199
Yong B, Xu Z, Shen J, Chen H, Tian Y, Zhou Q (2017) Neural network model with monte carlo algorithm for electricity demand forecasting in queensland. In: Australasian computer science week multiconference, p 47
Zhao H (2016) General vector machine. arXiv preprint
Zhou Q, Chen H, Zhao H, Zhang G, Yong J, Shen J (2016) A local field correlated and monte carlo based shallow neural network model for non-linear time series prediction. EAI Endorsed Trans Scalable Inf Syst 3(8):e5-1–e5-7
Acknowledgements
This work was supported by Dongguan’s Recruitment of Innovation and entrepreneurship talent program, National Natural Science Foundation of China under Grant Nos. 61402210 and 60973137, Program for New Century Excellent Talents in University under Grant No. NCET-12-0250, Strategic Priority Research Program of the Chinese Academy of Sciences with Grant No. XDA03030100, Gansu Sci. and Tech. Program under Grant Nos. 1104GKCA049, 1204GKCA061 and 1304GKCA018, Google Research Awards and Google Faculty Award, China. This research has also been conducted with the support of the Australian Government Research Training Program Scholarship.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
All authors declare that they have no conflicts of interest regarding the publication of this manuscript.
Additional information
Communicated by V. Loia.
Rights and permissions
About this article
Cite this article
Yong, B., Li, F., Lv, Q. et al. Derivative-based acceleration of general vector machine. Soft Comput 23, 987–995 (2019). https://doi.org/10.1007/s00500-017-2808-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-017-2808-z