Building Sparse Support Vector Machines for Multi-Instance Classification
We propose a direct approach to learning sparse Support Vector Machine (SVM) prediction models for Multi-Instance (MI) classification. The proposed sparse SVM is based on a “label-mean” formulation of MI classification which takes the average of predictions of individual instances for bag-level prediction. This leads to a convex optimization problem, which is essential for the tractability of the optimization problem arising from the sparse SVM formulation we derived subsequently, as well as the validity of the optimization strategy we employed to solve it. Based on the “label-mean” formulation, we can build sparse SVM models for MI classification and explicitly control their sparsities by enforcing the maximum number of expansions allowed in the prediction function. An effective optimization strategy is adopted to solve the formulated sparse learning problem which involves the learning of both the classifier and the expansion vectors. Experimental results on benchmark data sets have demonstrated that the proposed approach is effective in building very sparse SVM models while achieving comparable performance to the state-of-the-art MI classifiers.
KeywordsSupport Vector Machine Support Vector Machine Model Support Vector Machine Method Multiple Kernel Learn Music Genre
Unable to display preview. Download preview PDF.
- 2.Gartner, T., Flach, A., Kowalczyk, A., Smola, A.J.: Multi-instance kernels. In: Intl. Conf. Machine Learning, pp. 179–186 (2002)Google Scholar
- 3.Andrews, S., Tsochantaridis, I., Hofmann, T.: Support vector machines for multiple-instance learning. In: Advances in Neural Information Processing Systems, pp. 561–568 (2003)Google Scholar
- 4.Cheung, P.M., Kwok, J.T.Y.: A regularization framework for multiple-instance learning. In: Intl. Conf. Machine Learning (2006)Google Scholar
- 6.Zhou, Z.H., Xu, J.M.: On the relation between multi-instance learning and semi-supervised learning. In: Intl. Conf. Machine Learning (2007)Google Scholar
- 7.Li, Y.F., Kwok, J.T., Tsang, I., Zhou, Z.H.: A convex method for locating regions of interest with multi-instance learning. In: European Conference on Machine Learning (2009)Google Scholar
- 8.Smola, A.J., Scholkopf, B.: Sparse greedy matrix approximation for machine learning. In: Intl. Conf. Machine Learning, pp. 911–918 (2000)Google Scholar
- 9.Lee, Y.J., Mangasarian, O.L.: Rsvm: Reduced support vector machines. In: SIAM Conf. Data Mining (2001)Google Scholar
- 10.Scholkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. The MIT Press, Cambridge (2002)Google Scholar
- 13.Bezdek, J.C., Harthaway, R.J.: Convergence of alternating optimization. Journal Neural, Parallel & Scientific Computations 11(4) (2003)Google Scholar