Building Sparse Support Vector Machines for Multi-Instance Classification

  • Zhouyu Fu
  • Guojun Lu
  • Kai Ming Ting
  • Dengsheng Zhang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6911)


We propose a direct approach to learning sparse Support Vector Machine (SVM) prediction models for Multi-Instance (MI) classification. The proposed sparse SVM is based on a “label-mean” formulation of MI classification which takes the average of predictions of individual instances for bag-level prediction. This leads to a convex optimization problem, which is essential for the tractability of the optimization problem arising from the sparse SVM formulation we derived subsequently, as well as the validity of the optimization strategy we employed to solve it. Based on the “label-mean” formulation, we can build sparse SVM models for MI classification and explicitly control their sparsities by enforcing the maximum number of expansions allowed in the prediction function. An effective optimization strategy is adopted to solve the formulated sparse learning problem which involves the learning of both the classifier and the expansion vectors. Experimental results on benchmark data sets have demonstrated that the proposed approach is effective in building very sparse SVM models while achieving comparable performance to the state-of-the-art MI classifiers.


Support Vector Machine Support Vector Machine Model Support Vector Machine Method Multiple Kernel Learn Music Genre 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Dietterich, T.G., Lathrop, R.H., Lozano-perez, T.: Solving the multiple-instance problem with axis-parallel rectangles. Artificial Intelligence 89, 31–71 (1997)CrossRefzbMATHGoogle Scholar
  2. 2.
    Gartner, T., Flach, A., Kowalczyk, A., Smola, A.J.: Multi-instance kernels. In: Intl. Conf. Machine Learning, pp. 179–186 (2002)Google Scholar
  3. 3.
    Andrews, S., Tsochantaridis, I., Hofmann, T.: Support vector machines for multiple-instance learning. In: Advances in Neural Information Processing Systems, pp. 561–568 (2003)Google Scholar
  4. 4.
    Cheung, P.M., Kwok, J.T.Y.: A regularization framework for multiple-instance learning. In: Intl. Conf. Machine Learning (2006)Google Scholar
  5. 5.
    Chen, Y., Bi, J., Wang, J.: Miles: Multiple-instance learning via embedded instance selection. IEEE Trans. on Pattern Analysis and Machine Intelligence 28(12), 1931–1947 (2006)CrossRefGoogle Scholar
  6. 6.
    Zhou, Z.H., Xu, J.M.: On the relation between multi-instance learning and semi-supervised learning. In: Intl. Conf. Machine Learning (2007)Google Scholar
  7. 7.
    Li, Y.F., Kwok, J.T., Tsang, I., Zhou, Z.H.: A convex method for locating regions of interest with multi-instance learning. In: European Conference on Machine Learning (2009)Google Scholar
  8. 8.
    Smola, A.J., Scholkopf, B.: Sparse greedy matrix approximation for machine learning. In: Intl. Conf. Machine Learning, pp. 911–918 (2000)Google Scholar
  9. 9.
    Lee, Y.J., Mangasarian, O.L.: Rsvm: Reduced support vector machines. In: SIAM Conf. Data Mining (2001)Google Scholar
  10. 10.
    Scholkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. The MIT Press, Cambridge (2002)Google Scholar
  11. 11.
    Keerthi, S., Chapelle, O., DeCoste, D.: Building support vector machines with reduced classifier complexity. Journal of Machine Learning Research 7, 1493–1515 (2006)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Wu, M., Scholkopf, B., Bakir, G.: A direct method for building sparse kernel learning algorithms. Journal of Machine Learning Research 7, 603–624 (2006)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Bezdek, J.C., Harthaway, R.J.: Convergence of alternating optimization. Journal Neural, Parallel & Scientific Computations 11(4) (2003)Google Scholar
  14. 14.
    Rakotomamonjy, A., Bach, F.R., Canu, S., Grandvalet, Y.: Simplemkl. Journal of Machine Learning Research 9, 2491–2521 (2008)MathSciNetzbMATHGoogle Scholar
  15. 15.
    Bonnans, J.F., Shapiro, A.: Optimization problems with pertubation: A guided tour. SIAM Review 40(2), 202–227 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Tzanetakis, G., Cook, P.: Music genre classification of audio signals. IEEE Trans. Speech and Audio Processing 10(5), 293–302 (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Zhouyu Fu
    • 1
  • Guojun Lu
    • 1
  • Kai Ming Ting
    • 1
  • Dengsheng Zhang
    • 1
  1. 1.Monash UniversityChurchillAustralia

Personalised recommendations