Multiobjective Multiclass Soft-Margin Support Vector Machine and Its Solving Technique Based on Benson’s Method

  • Keiji Tatsumi
  • Ryo Kawachi
  • Kenji Hayashida
  • Tetsuzo Tanino
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5861)


In this paper, we focus on the all together model, which is one of the support vector machine (SVM) using a piece-wise linear function for multiclass classification. We already proposed a multiobjective hard-margin SVM model as a new all together model for piecewise linearly separable data, which maximizes all of the geometric margins simultaneously for the generalization ability. In addition, we derived a single-objective convex problem and showed that a Pareto optimal solution for the proposed multiobjective SVM is obtained by solving single-objective problems. However, in the real-world classification problem the data are often piecewise linearly inseparable. Therefore, in this paper we extend the hard-margin SVM for the data by using penalty functions for the margin slack variables between outliers and the corresponding discriminant hyperplane. Those functions are incorporated into the objective functions. Moreover, we derive a single-objective second-order cone programming (SOCP) problem based on Benson’s method and some techniques, and show that a Pareto optimal solution for the proposed soft-margin SVM is obtained by solving the SOCP iteratively. Furthermore through numerical experiments we verify that the proposed iterative method maximizes the geometric margins and constructs a classifier with a high generalization ability.


Support Vector Machine Penalty Function Multiobjective Optimization Pareto Optimal Solution Support Vector Machine Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alizadeh, F., Goldfarb, D.: Second-order cone programming. Mathematical Programming, Ser. B 95, 3–51 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Andersen, E.D., Roos, C., Terlaky, T.: On implementing a primal-dual interior-point method for conic quadratic optimization. Mathematical Programming, Ser. B 95, 249–277 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    Bottou, L., Cortes, C., Denker, J., Drucker, H., Guyon, I., Jackel, L., LeCun, Y., Muller, U., Sackinger, E., Simard, P., Vapnik, V.: Comparison of classifier methods: A case study in handwriting digit recognition. In: Proc. Int. Conf. Pattern Recognition, pp. 77–87 (1994)Google Scholar
  4. 4.
    Bredensteiner, E.J., Bennett, K.P.: Multicategory classification by support vector machines. Computational Optimization and Applications 12, 53–79 (1999)zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Ehrgott, M.: Multicriteria optimization. Springer, Berlin (2005)Google Scholar
  6. 6.
    Guermeur, Y.: Combining discriminant models with new multi-class SVMs. Neuro COLT2 Technical Report Series (2000)Google Scholar
  7. 7.
    Hsh, C.W., Lin, C.J.: A comparison of methods for multiclass support vector machines. IEEE Transactions on Neural Networks 13(2), 181–201 (2002)Google Scholar
  8. 8.
    Kressel, U.: Pairwise classification and support vector machines. In: Schölkopf, B., Burges, C., Smola, A.J. (eds.) Advances in kernel methods – Support vector learning, pp. 255–268. MIT Press, Cambridge (1999)Google Scholar
  9. 9.
    Mittelmann, H.D.: An independent benchmarking of SDP and SOCP solvers. Mathematical Programming, Ser. B 95, 407–430 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Tatsumi, K., Hayashida, K., Higashi, H., Tanino, T.: Multi-objective multiclass support vector machine for pattern recognition. In: Proceedings of SICE Annual Conference 2007, pp. 1095–1098 (2007)Google Scholar
  11. 11.
    Tatsumi, K., Tanino, T., Hayashida, K.: Multiobjective Multiclass Support Vector Machines Maximizing Geometric Margins. Pacific Journal of Optimization (to appear)Google Scholar
  12. 12.
    Vapnik, Y.: Statistical learning theory. A Wiley-Interscience Publication, Hoboken (1998)zbMATHGoogle Scholar
  13. 13.
    Weston, J., Watkins, C.: Multi-class support vector machines. Technical report CSD-TR-98-04, Univ. London, Royal Holloway (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Keiji Tatsumi
    • 1
  • Ryo Kawachi
    • 1
  • Kenji Hayashida
    • 1
  • Tetsuzo Tanino
    • 1
  1. 1.Osaka UniversityOsakaJapan

Personalised recommendations