Multiobjective Multiclass Soft-Margin Support Vector Machine and Its Solving Technique Based on Benson’s Method
- 632 Downloads
In this paper, we focus on the all together model, which is one of the support vector machine (SVM) using a piece-wise linear function for multiclass classification. We already proposed a multiobjective hard-margin SVM model as a new all together model for piecewise linearly separable data, which maximizes all of the geometric margins simultaneously for the generalization ability. In addition, we derived a single-objective convex problem and showed that a Pareto optimal solution for the proposed multiobjective SVM is obtained by solving single-objective problems. However, in the real-world classification problem the data are often piecewise linearly inseparable. Therefore, in this paper we extend the hard-margin SVM for the data by using penalty functions for the margin slack variables between outliers and the corresponding discriminant hyperplane. Those functions are incorporated into the objective functions. Moreover, we derive a single-objective second-order cone programming (SOCP) problem based on Benson’s method and some techniques, and show that a Pareto optimal solution for the proposed soft-margin SVM is obtained by solving the SOCP iteratively. Furthermore through numerical experiments we verify that the proposed iterative method maximizes the geometric margins and constructs a classifier with a high generalization ability.
KeywordsSupport Vector Machine Penalty Function Multiobjective Optimization Pareto Optimal Solution Support Vector Machine Model
Unable to display preview. Download preview PDF.
- 3.Bottou, L., Cortes, C., Denker, J., Drucker, H., Guyon, I., Jackel, L., LeCun, Y., Muller, U., Sackinger, E., Simard, P., Vapnik, V.: Comparison of classifier methods: A case study in handwriting digit recognition. In: Proc. Int. Conf. Pattern Recognition, pp. 77–87 (1994)Google Scholar
- 5.Ehrgott, M.: Multicriteria optimization. Springer, Berlin (2005)Google Scholar
- 6.Guermeur, Y.: Combining discriminant models with new multi-class SVMs. Neuro COLT2 Technical Report Series (2000)Google Scholar
- 7.Hsh, C.W., Lin, C.J.: A comparison of methods for multiclass support vector machines. IEEE Transactions on Neural Networks 13(2), 181–201 (2002)Google Scholar
- 8.Kressel, U.: Pairwise classification and support vector machines. In: Schölkopf, B., Burges, C., Smola, A.J. (eds.) Advances in kernel methods – Support vector learning, pp. 255–268. MIT Press, Cambridge (1999)Google Scholar
- 10.Tatsumi, K., Hayashida, K., Higashi, H., Tanino, T.: Multi-objective multiclass support vector machine for pattern recognition. In: Proceedings of SICE Annual Conference 2007, pp. 1095–1098 (2007)Google Scholar
- 11.Tatsumi, K., Tanino, T., Hayashida, K.: Multiobjective Multiclass Support Vector Machines Maximizing Geometric Margins. Pacific Journal of Optimization (to appear)Google Scholar
- 13.Weston, J., Watkins, C.: Multi-class support vector machines. Technical report CSD-TR-98-04, Univ. London, Royal Holloway (1998)Google Scholar