Abstract
We show how to implement a simple procedure for support vector machine training as a recurrent neural network. Invoking the fact that support vector machines can be trained using Frank-Wolfe optimization which in turn can be seen as a form of reservoir computing, we obtain a model that is of simpler structure and can be implemented more easily than those proposed in previous contributions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Anguita, D., Boni, A.: Improved neural network for SVM learning. IEEE Trans. Neural Netw. 13(2), 1243–1244 (2002)
Bauckhage, C.: A neural network implementation of Frank-Wolfe optimization. In: Lintas, A., Rovetta, S., Verschure, P., Villa, A. (eds.) ICANN 2017. LNCS, vol. 10613, pp. 219–226. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68600-4_26
Bauckhage, C.: The dual problem of \(L_2\) SVM training. Technical report, ResearchGate (2018)
Bishop, C.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)
Clarkson, K.: Coresets, sparse greedy approximation, and the Frank-Wolfe algorithm. ACM Trans. Algorithms 6(4), 63:1–63:30 (2010)
Cortes, C., Vapnik, V.: Support vector networks. Mach. Learn. 20(3), 273–297 (1995)
Duch, W.: Support vector neural training. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 67–72. Springer, Heidelberg (2005). https://doi.org/10.1007/11550907_11
Frank, M., Wolfe, P.: An algorithm for quadratic programming. Nav. Res. Logist. Q. 3(1–2), 95–110 (1956)
Jäger, H., Haas, H.: Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304(5667), 78–80 (2004)
Jaggi, M.: Revisiting Frank-Wolfe: projection-free sparse convex optimization. J. Mach. Learn. Res. 28(1), 427–435 (2013)
Jändel, M.: Biologically relevant neural network architectures for support vector machines. Neural Netw. 49, 39–50 (2014)
Koshiba, Y., Abe, S.: Comparison of L1 and L2 support vector machines. In: Proceedings IJCNN (2003)
Perfetti, R., Ricci, E.: Analogue neural network for support vector machine learning. IEEE Trans. Neural Netw. 17(4), 1085–1091 (2006)
Sifa, R.: An overview of Frank-Wolfe optimization for stochasticity constrained interpretable matrix and tensor factorization. In: ICANN 2018 (2018)
Tang, Y.: Deep Learning using Linear Support Vector Machines. arXiv:1306.0239 [cs.LG] (2013)
Vincent, P., Bengio, Y.: A neural support vector network architecture with adaptive kernels. In: Proceedings IJCNN (2000)
Xia, Y.: A new neural network for solving linear and quadratic programming problems. IEEE Trans. Neural Netw. 7(6), 1544–1547 (1996)
Yang, Y., He, Q., Hu, X.: A compact neural network for training support vector machines. Neurocomputing 86, 193–198 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Sifa, R., Paurat, D., Trabold, D., Bauckhage, C. (2018). Simple Recurrent Neural Networks for Support Vector Machine Training. In: Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I. (eds) Artificial Neural Networks and Machine Learning – ICANN 2018. ICANN 2018. Lecture Notes in Computer Science(), vol 11141. Springer, Cham. https://doi.org/10.1007/978-3-030-01424-7_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-01424-7_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-01423-0
Online ISBN: 978-3-030-01424-7
eBook Packages: Computer ScienceComputer Science (R0)