Abstract
Multi-label support vector machine (Rank-SVM) is an effective algorithm for multi-label classification, which is formulated as a quadratic programming problem with q equality constraints and lots of box constraints for a q-class multi-label data set. So far, Rank-SVM is solved by Frank-Wolfe method (FWM), where a large-scale linear programming problem needs to be dealt with at each iteration. In this paper, we propose a random block coordinate descent method (RBCDM) for Rank-SVM, in which a small-scale quadratic programming problem with at least (q+1) variables randomly is solved at each iteration. Experiments on three data sets illustrate that our RBCDM runs much faster than FWM for Rank-SVM, and Rank-SVM is a powerful candidate for multi-label classification.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Boutell, M.R., Lu, J., Shen, X., Brown, C.M.: Learning multi-label scene classification. Pattern Recognition 37(9), 1757–1771 (2004)
Tsoumakas, G., Katakis, I.: Multi-label classification: an overview. International Journal of Data Warehousing and Mining 3(3), 1–13 (2007)
Tsoumakas, G., Katakis, I., Vlahavas, I.: Mining multi-label data. In: Maimon, O., Rokach, L. (eds.) Data Mining and Knowledge Discovery Handbook, 2nd edn., pp. 667–685. Springer, Heidelberg (2010)
Zhang, M.L., Zhou, Z.H.: A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering (in press, 2013), doi:10.1109/TKDE.2013.39
Elisseeff, A., Weston, J.: A kernel method for multi-labelled classification. In: 14th Annual Conference on Neural Information Processing Systems, pp. 681–687. MIT Press, Cambridge (2002)
Elisseeff, A., Weston, J.: Kernel methods for multi-labelled classification and categorical regression problems. Technical Report, BIOwulf Technologies (2001), http://www.kyb.tuebingen.mpg.de/bs/people/weston/publications
Frank, M., Wolfe, P.: An algorithm for quadratic programming. Naval Research Logistic Quarterly 3(1-2), 95–110 (1956)
Guelat, J., Marcotte, P.: Some comments on Wolfe’s away step. Mathematical Programming 34(1), 110–119 (1986)
Wen, Z.W., Goldfarb, D., Scheinberg, K.: Block coordinate descent methods for seimdefinate programming. In: Anjos, M.F., Lasserre, J.B. (eds.) Handbook on Semidefinite, Conic and Polynomial Optimization, pp. 533–564. Springer, Heidelberg (2012)
Platt, J.: Sequential minimal optimization: a fast algorithm for training support vector machines. In: Scholkopf, B., Burges, C.J.C., Smola, A.J. (eds.) Advances in Kernel Methods - Support Vector Learning, pp. 185–208. MIT Press, Cambridge (1999)
Fan, R.E., Chen, P.H., Lin, C.J.: Working set selection using second order information for training support vector machines. Journal of Machine Learning Research 6, 1889–1918 (2005)
Tseng, P., Yun, S.: A coordinate gradient descent method for linearly constrained smooth optimization and support vetcor machines training. Computational Optimization and Applications 47(2), 179–206 (2010)
Necoara, I., Patrascu, A.: A random coordinate descent algorithm for optimization problems with composite objective function and linear coupled constraints. Technical Report, Department of Automatic Control and Systems Engineering, University Politehnica Bucharest, Romania (2013), http://arXiv.1302.3074v1
Zhang, M.L., Zhou, Z.H.: Multilabel neural networks with application to function genomics and text categorization. IEEE Transactions on Knowledge and Data Engineering 18(10), 1338–1351 (2006)
Zhang, M.L.: ML-RBF: RBF neural networks for multi-label learning. Neural Processing Letters 29(2), 61–74 (2009)
Zhang, M.L., Pena, J.M., Robles, V.: Feature selection for multi-label naive Bayes classification. Information Sciences 179(19), 3218–3229 (2009)
Zhang, M.L., Zhou, Z.H.: ML-kNN: a lazy learning approach to multi-label learning. Pattern Recognition 40(5), 2038–2048 (2007)
Multi-label data sets, http://mulan.sourceforge.net/datasets.html
Multi-label suport vector machine C/C++ software of MLC-SVM, http://computer.njnu.edu.cn/Lab/LABIC/LABIC_software.html
Linear programming solver: LPSOL5.5, http://lpsolve.soureforge.net
Multi-label Matlab software of BP-MLL, ML-RBF, ML-NB and ML-kNN, http://cse.seu.edu.cn/people/zhangml
Brazdil, P.B., Soares, C.: A comparison of ranking methods for classification algorithm selection. In: Lopez de Mantaras, R., Plaza, E. (eds.) ECML 2000. LNCS (LNAI), vol. 1810, pp. 63–75. Springer, Heidelberg (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Xu, J. (2013). A Random Block Coordinate Descent Method for Multi-label Support Vector Machine. In: Lee, M., Hirose, A., Hou, ZG., Kil, R.M. (eds) Neural Information Processing. ICONIP 2013. Lecture Notes in Computer Science, vol 8227. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-42042-9_36
Download citation
DOI: https://doi.org/10.1007/978-3-642-42042-9_36
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-42041-2
Online ISBN: 978-3-642-42042-9
eBook Packages: Computer ScienceComputer Science (R0)