Robust Support Vector Machines

  • Yong Shi
  • Yingjie Tian
  • Gang Kou
  • Yi Peng
  • Jianping Li
Part of the Advanced Information and Knowledge Processing book series (AI&KP)


In real world applications, the training data are not usually assumed to be known exactly due to measurement and statistical errors. Since the solutions to optimization problems are typically sensitive to training data perturbations, errors in the input data tend to get amplified in the decision function, often resulting in far from optimal solutions. So it will be useful to explore formulations that can yield robust discriminants to such estimation errors. In this chapter, we first established robust versions of SVORM, which are represented as a second order cone programming (SOCP). And as the theoretical foundation, we study the relationship between the solutions of the SOCP and its dual problem. Here the second order cone in Hilbert space is involved. Then, we also establish a multi-class algorithm based on the above robust SVORM for general multi-class classification problem with perturbations. Furthermore, we construct a robust unsupervised and semi-supervised SVC for the problems with uncertainty information.


Dual Problem Training Point Second Order Cone Programming Robust Counterpart Perturbation Region 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 2.
    Alizadeh, F., Goldfarb, D.: Second order cone programming. Math. Program., Ser. B, 95, 3–51 (2003) MathSciNetMATHCrossRefGoogle Scholar
  2. 8.
    Angulo, C., Parr, X., Català, A.: K-SVCR, a support vector machine for multi-class classification. Neurocomputing 55, 57–77 (2003) CrossRefGoogle Scholar
  3. 9.
    Angulo, C., Ruiz, F.J., Gonzalez, L., Ortega, J.A.: Multi-classification by using tri-class SVM. Neural Process. Lett. 23, 89–101 (2006) CrossRefGoogle Scholar
  4. 15.
    Ben-Tal, A., Nemirovski, A.: Robust convex optimization. Math. Oper. Res. 23(4), 769–805 (1998) MathSciNetMATHCrossRefGoogle Scholar
  5. 16.
    Ben-Tal, A., Nemirovski, A.: Robust solutions to uncertain programs. Oper. Res. Lett. 25, 1–13 (1999) MathSciNetMATHCrossRefGoogle Scholar
  6. 21.
    Blake, C.L., Merz, C.J.: UCI Repository of Machine Learning Databases. University of California, Irvine. (1998)
  7. 22.
    Borwein, J.M.: Optimization with respect to partial orderings. Ph.D. Thesis, Oxford University, Jesus College (1974) Google Scholar
  8. 39.
    Chopra, V.K., Ziemba, W.T.: The effect of errors in means, variances and covariances on optimal portfolio choice. J. Portf. Manag. 19, 6–11 (1993) CrossRefGoogle Scholar
  9. 42.
    Chu, W., Keerthi, S.S.: New approaches to support vector ordinal regression. In: Proc. of International Conference on Machine Learning (ICML-05), pp. 145–152 (2005) CrossRefGoogle Scholar
  10. 44.
    Cohen, W.W., Schapire, R.E., Singer, Y.: Learning to order things. J. Artif. Intell. Res. 10, 243–270 (1999) MathSciNetMATHGoogle Scholar
  11. 50.
    Crammer, K., Singer, Y.: Pranking with ranking. In: Proceedings of the Conference on Neural Information Processing Systems (NIPS) (2001) Google Scholar
  12. 65.
    El-Ghaoui, L., Lebret, H.: Robust solutions to least-square problems to uncertain data matrices. SIAM J. Matrix Anal. Appl. 18, 1035–1064 (1997) MathSciNetMATHCrossRefGoogle Scholar
  13. 66.
    El-Ghaoui, L., Oustry, F.L., Lebret, H.: Robust solutions to uncertain semidefinite programs. SIAM J. Matrix Anal. Appl. 9, 33–52 (1998) MathSciNetMATHGoogle Scholar
  14. 71.
    Faybusovich, L., Tsuchiya, T.: Primal-dual algorithms and infinite-dimensional Jordan algebras of finite rank. Math. Program., Ser. B 97, 471–493 (2003) MathSciNetMATHCrossRefGoogle Scholar
  15. 74.
    Frank, E., Hall, M.: A simple approach to ordinal classification. In: Lecture Notes in Computer Science, vol. 2167, pp. 145–156. Springer, Berlin (2001) Google Scholar
  16. 89.
    Goldfarb, D., Iyengar, G.: Robust portfolio selection problems. Math. Oper. Res. 28, 1 (2003) MathSciNetMATHCrossRefGoogle Scholar
  17. 104.
    Har-Peled, S., Roth, D., Zimak, D.: Constraint classification: A new approach to multiclass classification and ranking. In: Advances in Neural Information Processing Systems, vol. 15 (2002) Google Scholar
  18. 129.
    Kramer, S., Widmer, G., Pgahringer, B., DeGroeve, M.: Prediction of ordinal classes using regression trees. Fundam. Inform. 47, 1–13 (2001) MATHGoogle Scholar
  19. 130.
    Krebel, U.: Pairwise classification and support vector machines. In: Schölkopf, B., Burges, C.J.C., Smola, A.J. (eds.) Advances in Kernel Methods: Support Vector Learning, pp. 255–268. MIT Press, Cambridge (1999) Google Scholar
  20. 186.
    Sim, M.: Robust optimization. Ph.D. Thesis (2004) Google Scholar
  21. 191.
    Sturm, J.F.: Using sedumi1.02, a Matlab toolbox for optimization over symmetric cones. Optim. Methods Softw. 11-12, 625–653 (1999) MathSciNetCrossRefGoogle Scholar
  22. 215.
    Weston, J., Watkins, C.: Multi-class Support Vector Machines. University of London Press, Egham (1998) Google Scholar
  23. 225.
    Xu, B.L., Neufeld, J., Schuurmans, D.: Maximum margin clustering. In: Advances in Neural Information Processing Systems (NIPS-04), vol. 17 (2004) Google Scholar
  24. 230.
    Yang, Z.X.: Support vector ordinal regression machines and multi-class classification. Ph.D. Thesis, China Agricultural University (2007) Google Scholar
  25. 232.
    Yang, Z.X., Deng, N.Y., Tian, Y.J.: A multi-class classification algorithm based on ordinal regression machine. In: International Conference on CIMCA2005 & IAWTIC 2005, Vienna, Austria, vol. 2, pp. 810–814 (2005) Google Scholar
  26. 244.
    Zhao, K., Tian, Y.J., Deng, N.Y.: Robust unsupervised and semi-supervised bounded C-support vector machines. In: Proceedings of the 7th IEEE ICDM 2007 Workshop, pp. 331–336 (2007) Google Scholar
  27. 250.
    Zhong, P., Fukushima, M.: Second order cone programming formulations for robust multi-class classification. Neural Comput. 19(1), 258–282 (2007) MathSciNetMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  • Yong Shi
    • 1
    • 2
  • Yingjie Tian
    • 1
  • Gang Kou
    • 3
  • Yi Peng
    • 3
  • Jianping Li
    • 4
  1. 1.Research Center on Fictitious Economy and Data ScienceChinese Academy of SciencesBeijingChina
  2. 2.College of Information Science & TechnologyUniversity of Nebraska at OmahaOmahaUSA
  3. 3.School of Management and EconomicsUniversity of Electronic Science and Technology of ChinaChengduChina
  4. 4.Institute of Policy and ManagementChinese Academy of SciencesBeijingChina

Personalised recommendations