A Sequential Minimal Optimization Algorithm for the All-Distances Support Vector Machine

  • Diego Candel
  • Ricardo Ñanculef
  • Carlos Concha
  • Héctor Allende
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6419)

Abstract

The All-Distances SVM is a single-objective light extension of the binary μ-SVM for multi-category classification that is competitive against multi-objective SVMs, such as One-against-the-Rest SVMs and One-against-One SVMs. Although the model takes into account considerably less constraints than previous formulations, it lacks of an efficient training algorithm, making its use with medium and large problems impracticable. In this paper, a Sequential Minimal Optimization-like algorithm is proposed to train the All-Distances SVM, making large problems abordable. Experimental results with public benchmark data are presented to show the performance of the AD-SVM trained with this algorithm against other single-objective multi-category SVMs.

Keywords

Kernel Machines Multi-category Classification Support Vector Machines Sequential Minimal Optimization 

References

  1. 1.
    Blake, C., Merz, C.: UCI repository of machine learning databases (1998), http://mlr.cs.umass.edu/ml/index.html
  2. 2.
    Candel, D.: Source code of the smo algorithm for the ad-svm, http://git.inf.utfsm.cl/?p=dcontard.git;a=summary
  3. 3.
    Candel, D.: Algoritmo tipo SMO para la AD-SVM aplicado a Clasificación Multicategoría. Master’s thesis, Universidad Técnica Federico Santa María, Valparaíso, Chile (2010), http://www.alumnos.inf.utfsm.cl/~dcontard/tesis.pdf
  4. 4.
    Chang, C.C., Lin, C.J.: Libsvm data: Classification, multi-class (2001), http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/multiclass.html
  5. 5.
    Chen, P.H., Fan, R.E., Lin, C.J.: A study on SMO-type decomposition methods for support vector machines. IEEE transactions on neural networks / a publication of the IEEE Neural Networks Council 17(4), 893–908 (2006)CrossRefGoogle Scholar
  6. 6.
    Coleman, T.F., Li, Y.: A reflective newton method for minimizing a quadratic function subject to bounds on some of the variables. SIAM J. on Optimization 6(4), 1040–1058 (1996)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Crammer, K., Singer, Y.: On the algorithmic implementation of multiclass kernel-based vector machines. Journal of Machine Learning Research (2), 265–292 (2001)Google Scholar
  8. 8.
    Crammer, K., Singer, Y.: On the learnability and design of output codes for multiclass problems. Machine Learning 47(2-3), 201–233 (2002)CrossRefMATHGoogle Scholar
  9. 9.
    Crisp, D., Burges, C.: A Geometric Interpretation of ν-SVM Classifiers. In: Advances in Neural Information, vol. (12), pp. 244–250. MIT, Cambridge (2000)Google Scholar
  10. 10.
    Fan, R.e., Chen, P.h., Lin, C.j.: Working Set Selection Using Second Order Information for Training Support Vector Machines. Journal of Machine Learning Research 6, 1889–1918 (2005)MathSciNetMATHGoogle Scholar
  11. 11.
    Hull, J.J.: A database for handwritten text recognition research. IEEE Trans. Pattern Anal. Mach. Intell. 16(5), 550–554 (1994)CrossRefGoogle Scholar
  12. 12.
    Joachims, T.: Making large-scale support vector machine learning practical (1998)Google Scholar
  13. 13.
    Keerthi, S., Gilbert, E.: Convergence of a Generalized SMO Algorithm for SVM Classifier Design. Machine Learning 46(1), 351–360 (2002)CrossRefMATHGoogle Scholar
  14. 14.
    Keerthi, S., Shevade, S., Murthy, K., Bhattacharyya, C.: Improvements to Platt’s SMO Algorithm for SVM Classifier Design. Neural Computation 13(3), 637–649 (2001)CrossRefMATHGoogle Scholar
  15. 15.
    Kressel, U.: Pairwise classification and support vector machines. In: Advances in kernel methods: support vector learning, pp. 255–268. MIT Press, Cambridge (1999)Google Scholar
  16. 16.
    Lee, Y., Li, Y., Wahba, G.: Multicategory support vector machines: Theory and application to the classification of microarray data and satellite radiance data. Journal of the American Statistical Association 99(465), 67–81 (2004)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Ñanculef, R., Concha, C., Allende, H., Candel, D., Moraga, C.: Ad-svms: A light extension of svms for multicategory classification. International Journal of Hybrid Intelligent Systems 6(2), 69–79 (2009)CrossRefMATHGoogle Scholar
  18. 18.
    Platt, J.C.: Fast training of support vector machines using sequential minimal optimization, pp. 185–208 (1999)Google Scholar
  19. 19.
    Scholkopf, B., Smola, A., Williamson, R., Bartlett, P.: New support vector algorithms. Neural computation 12(5), 1207–1245 (2000)CrossRefGoogle Scholar
  20. 20.
    Vapnik, V.N.: The nature of statistical learning theory. Springer, Heidelberg (1995)CrossRefMATHGoogle Scholar
  21. 21.
    Weston, J., Watkins, C.: Support vector machines for multiclass pattern recognition. In: Proceedings of the Seventh ESSAN, pp. 219–224 (1999)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Diego Candel
    • 1
  • Ricardo Ñanculef
    • 1
  • Carlos Concha
    • 1
  • Héctor Allende
    • 1
    • 2
  1. 1.Departamento de InformáticaUniversidad Técnica Federico Santa MaríaValparaísoChile
  2. 2.Facultad de Ingeniería y CienciaUniversidad Adolfo IbáñezSantiagoChile

Personalised recommendations