Abstract
We propose a direct approach to learning sparse Support Vector Machine (SVM) prediction models for Multi-Instance (MI) classification. The proposed sparse SVM is based on a “label-mean” formulation of MI classification which takes the average of predictions of individual instances for bag-level prediction. This leads to a convex optimization problem, which is essential for the tractability of the optimization problem arising from the sparse SVM formulation we derived subsequently, as well as the validity of the optimization strategy we employed to solve it. Based on the “label-mean” formulation, we can build sparse SVM models for MI classification and explicitly control their sparsities by enforcing the maximum number of expansions allowed in the prediction function. An effective optimization strategy is adopted to solve the formulated sparse learning problem which involves the learning of both the classifier and the expansion vectors. Experimental results on benchmark data sets have demonstrated that the proposed approach is effective in building very sparse SVM models while achieving comparable performance to the state-of-the-art MI classifiers.
Chapter PDF
Similar content being viewed by others
Keywords
- Support Vector Machine
- Support Vector Machine Model
- Support Vector Machine Method
- Multiple Kernel Learn
- Music Genre
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Dietterich, T.G., Lathrop, R.H., Lozano-perez, T.: Solving the multiple-instance problem with axis-parallel rectangles. Artificial Intelligence 89, 31–71 (1997)
Gartner, T., Flach, A., Kowalczyk, A., Smola, A.J.: Multi-instance kernels. In: Intl. Conf. Machine Learning, pp. 179–186 (2002)
Andrews, S., Tsochantaridis, I., Hofmann, T.: Support vector machines for multiple-instance learning. In: Advances in Neural Information Processing Systems, pp. 561–568 (2003)
Cheung, P.M., Kwok, J.T.Y.: A regularization framework for multiple-instance learning. In: Intl. Conf. Machine Learning (2006)
Chen, Y., Bi, J., Wang, J.: Miles: Multiple-instance learning via embedded instance selection. IEEE Trans. on Pattern Analysis and Machine Intelligence 28(12), 1931–1947 (2006)
Zhou, Z.H., Xu, J.M.: On the relation between multi-instance learning and semi-supervised learning. In: Intl. Conf. Machine Learning (2007)
Li, Y.F., Kwok, J.T., Tsang, I., Zhou, Z.H.: A convex method for locating regions of interest with multi-instance learning. In: European Conference on Machine Learning (2009)
Smola, A.J., Scholkopf, B.: Sparse greedy matrix approximation for machine learning. In: Intl. Conf. Machine Learning, pp. 911–918 (2000)
Lee, Y.J., Mangasarian, O.L.: Rsvm: Reduced support vector machines. In: SIAM Conf. Data Mining (2001)
Scholkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. The MIT Press, Cambridge (2002)
Keerthi, S., Chapelle, O., DeCoste, D.: Building support vector machines with reduced classifier complexity. Journal of Machine Learning Research 7, 1493–1515 (2006)
Wu, M., Scholkopf, B., Bakir, G.: A direct method for building sparse kernel learning algorithms. Journal of Machine Learning Research 7, 603–624 (2006)
Bezdek, J.C., Harthaway, R.J.: Convergence of alternating optimization. Journal Neural, Parallel & Scientific Computations 11(4) (2003)
Rakotomamonjy, A., Bach, F.R., Canu, S., Grandvalet, Y.: Simplemkl. Journal of Machine Learning Research 9, 2491–2521 (2008)
Bonnans, J.F., Shapiro, A.: Optimization problems with pertubation: A guided tour. SIAM Review 40(2), 202–227 (1998)
Tzanetakis, G., Cook, P.: Music genre classification of audio signals. IEEE Trans. Speech and Audio Processing 10(5), 293–302 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Fu, Z., Lu, G., Ting, K.M., Zhang, D. (2011). Building Sparse Support Vector Machines for Multi-Instance Classification. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2011. Lecture Notes in Computer Science(), vol 6911. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23780-5_40
Download citation
DOI: https://doi.org/10.1007/978-3-642-23780-5_40
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23779-9
Online ISBN: 978-3-642-23780-5
eBook Packages: Computer ScienceComputer Science (R0)