Fast Image Classification with Reduced Multiclass Support Vector Machines

  • Marco Melis
  • Luca PirasEmail author
  • Battista Biggio
  • Giorgio Giacinto
  • Giorgio Fumera
  • Fabio Roli
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9280)


Image classification is intrinsically a multiclass, nonlinear classification task. Support Vector Machines (SVMs) have been successfully exploited to tackle this problem, using one-vs-one or one-vs-all learning schemes to enable multiclass classification, and kernels designed for image classification to handle nonlinearities. To classify an image at test time, an SVM requires matching it against a small subset of the training data, namely, its support vectors (SVs). In the multiclass case, though, the union of the sets of SVs of each binary SVM may almost correspond to the full training set, potentially yielding an unacceptable computational complexity at test time. To overcome this limitation, in this work we propose a well-principled reduction method that approximates the discriminant function of a multiclass SVM by jointly optimizing the full set of SVs along with their coefficients. We show that our approach is capable of reducing computational complexity up to two orders of magnitude without significantly affecting recognition accuracy, by creating a super-sparse, budgeted set of virtual vectors.


Support Vector Machine Recognition Rate Test Time Spatial Pyramid Match Binary SVMs 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Akata, Z., Perronnin, F., Harchaoui, Z., Schmid, C.: Good practice in large-scale learning for image classification. IEEE Trans. Pattern Anal. Mach. Intell. 36(3), 507–520 (2014)CrossRefGoogle Scholar
  2. 2.
    Barla, A., Franceschi, E., Odone, F., Verri, A.: Image kernels. In: Lee, S.-W., Verri, A. (eds.) SVM 2002. LNCS, vol. 2388, pp. 83–96. Springer, Heidelberg (2002) CrossRefGoogle Scholar
  3. 3.
    Barla, A., Odone, F., Verri, A.: Histogram intersection kernel for image classification. In: Int’l Conf. Image Processing (ICIP), pp. 513–516 (2003)Google Scholar
  4. 4.
    Biggio, B., Melis, M., Fumera, G., Roli, F.: Sparse support faces. In: Int’l Conf. Biometrics (ICB), pp. 1–6 (2015)Google Scholar
  5. 5.
    Bosch, A., Muñoz, X., Marti, R.: Which is the best way to organize/classify images by content? Image Vision Comput. 25(6), 778–791 (2007)CrossRefGoogle Scholar
  6. 6.
    Chapelle, O.: Training a support vector machine in the primal. Neural Comput. 19(5), 1155–1178 (2007)CrossRefMathSciNetzbMATHGoogle Scholar
  7. 7.
    Chapelle, O., Haffner, P., Vapnik, V.: Support vector machines for histogram-based image classification. IEEE Trans. on Neural Networks 10(5), 1055–1064 (1999)CrossRefGoogle Scholar
  8. 8.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 886–893 (2005)Google Scholar
  9. 9.
    Felzenszwalb, P.F., Girshick, R.B., McAllester, D.A., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Patt. Anal. Mach. Intell. 32(9), 1627–1645 (2010)CrossRefGoogle Scholar
  10. 10.
    Keerthi, S.S., Chapelle, O., DeCoste, D.: Building support vector machines with reduced classifier complexity. J. Mach. Learn. Res. 7, 1493–1515 (2006)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 2169–2178 (2006)Google Scholar
  12. 12.
    Lee, Y.J., Mangasarian, O.L.: RSVM: reduced support vector machines. SDM 1, 325–361 (2001)Google Scholar
  13. 13.
    Li, F.F., Perona, P.: A bayesian hierarchical model for learning natural scene categories. In: IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 524–531 (2005)Google Scholar
  14. 14.
    Lin, Y., Lv, F., Zhu, S., Yang, M., Cour, T., Yu, K., Cao, L., Huang, T.S.: Large-scale image classification: fast feature extraction and SVM training. In: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 1689–1696 (2011)Google Scholar
  15. 15.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int’l Journal of Computer Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  16. 16.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: A holistic representation of the spatial envelope. Int’l Journal of Computer Vision 42(3), 145–175 (2001)CrossRefzbMATHGoogle Scholar
  17. 17.
    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Rifkin, R.M., Klautau, A.: In defense of one-vs-all classification. J. Mach. Learn. Res. 5, 101–141 (2004)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Schölkopf, B., Mika, S., Burges, C.J.C., Knirsch, P., Muller, K.R., Rätsch, G., Smola, A.J.: Input space versus feature space in kernel-based methods. IEEE Trans. Neural Networks 10(5), 1000–1017 (1999)CrossRefGoogle Scholar
  20. 20.
    Steinwart, I.: Sparseness of support vector machines. J. Mach. Learn. Res. 4, 1071–1105 (2003)MathSciNetGoogle Scholar
  21. 21.
    Suhr, J.K., Jung, H.G.: Sensor fusion-based vacant parking slot detection and tracking. IEEE Trans. on Intelligent Transportation Systems 15(1), 21–36 (2014)CrossRefGoogle Scholar
  22. 22.
    Tang, B., Mazzoni, D.: Multiclass reduced-set support vector machines. In: Proc. Int’l Conf. Machine Learning, ICML 2006, pp. 921–928. ACM, New York (2006)Google Scholar
  23. 23.
    Vapnik, V.N.: Statistical Learning Theory. Wiley, New York (1998)zbMATHGoogle Scholar
  24. 24.
    Xiao, J., Hays, J., Ehinger, K.A., Oliva, A., Torralba, A.: SUN database: large-scale scene recognition from abbey to zoo. In: IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 3485–3492 (2010)Google Scholar
  25. 25.
    Zhang, L., Lin, F., Zhang, B.: Support vector machine learning for image retrieval. In: Int’l Conf. Image Processing (ICIP), pp. 721–724 (2001)Google Scholar
  26. 26.
    Zhou, L., Zhou, Z., Hu, D.: Scene classification using multi-resolution low-level feature combination. Neurocomputing 122, 284–297 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Marco Melis
    • 1
  • Luca Piras
    • 1
    Email author
  • Battista Biggio
    • 1
  • Giorgio Giacinto
    • 1
  • Giorgio Fumera
    • 1
  • Fabio Roli
    • 1
  1. 1.Department of Electrical and Electronic EngineeringUniversity of CagliariCagliariItaly

Personalised recommendations