SVM Regularizer Models on RKHS vs. on Rm

  • Yinli Dong
  • Shuisheng Zhou
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7389)


There are two types of regularizer for SVM. The most popular one is that the classification function is norm-regularized on a Reproduced Kernel Hilbert Space(RKHS), and another important model is generalized support vector machine(GSVM), in which the coefficients of the classification function is norm-regularized on a Euclidean space R m . In this paper, we analyze the difference between them on computing stability, computational complexity and the efficiency of the Newton-type algorithms. Many typical loss functions are considered. The results show that the model of GSVM has more advantages than the other model. Some experiments support our analysis.


representer theorem regularizer newton-type algorithm 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, NY (2000)zbMATHGoogle Scholar
  2. 2.
    Steinwart, I., Christmann, A.: Support Vector Machines. Springer (2008)Google Scholar
  3. 3.
    Steinwart, I.: Sparseness of Support Vector Machines. JMLR 4, 1071–1105 (2003)MathSciNetGoogle Scholar
  4. 4.
    Keerthi, S.S., Chapelle, O., Decoste, D.: Building Support Vector Machines with reduced classifier complexity. JMLR 7, 1493–1515 (2006)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Schölkopf, B., Smola, A., Müller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 10(5), 1299–1319 (1998)CrossRefGoogle Scholar
  6. 6.
    Schölkopf, B., Herbrich, R., Smola, A.J.: A Generalized Representer Theorem. In: Helmbold, D.P., Williamson, B. (eds.) COLT 2001 and EuroCOLT 2001. LNCS (LNAI), vol. 2111, pp. 416–426. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  7. 7.
    Chapelle, O.: Training a Support Vector Machine in the primal. Neural Computation 19(5), 1155–1178 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Joachims, T.: Support Vector Machine (1998),
  9. 9.
    Platt, J.C.: Fast training of Support Vector Machines using Sequential Minimal Optimization. In: Schölkopf, B., et al. (eds.) Advances in Kernel Method-Support Vector Learning, pp. 185–208. MIT, Cambridge (1999)Google Scholar
  10. 10.
    Suykens, J., Vandewalle, J.: Least Square Support Vector Machine Classifiers. Neural Processing Letters 9(3), 293–300 (1999)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Mangasarian, O.L.: Generalized Support Vector Machine. In: Smola, A.J., et al. (eds.) Advances in Large Margin Classifiers, pp. 135–146. MIT Press (2000)Google Scholar
  12. 12.
    Lee, Y., Mangasarian, O.L.: RSVM: Reduced Support Vector Machines. Data Mining Institute, Computer Sciences Department, University of Wisconsin, pp. 1–7 (2001)Google Scholar
  13. 13.
    Fung, G., Mangasarian, O.L.: Proximal Support Vector Machine classifiers. In: Provost, Srikant, R. (eds.) Proceedings KDD 2001: Knowledge Discovery and Data Mining, San Francisco, CA, August 26-29, pp. 77–86. ACM, New York (2001)Google Scholar
  14. 14.
    Boyd, S.P., Vandenberghe, L.: Convex Optimization, 7th edn. Cambridge University Press, Cambridge (2009)Google Scholar
  15. 15.
    Zhou, S., Liu, H., Zhou, L., Ye, F.: Semi-smooth Newton Support Vector Machine. Pattern Recognition Letters 28, 2054–2062 (2007)CrossRefGoogle Scholar
  16. 16.
    Lin, K.M., Lin, C.J.: A Study on Reduced Support Vector Machines. IEEE Trans. on Neural Networks 14(6), 1449–1459 (2003)CrossRefGoogle Scholar
  17. 17.
    Ye, F., Liu, H., Zhou, S., Liu, S.: A Smoothing Trust-region Newton-CG method for Minimax Problem. Applied Mathematics and Computation 199(2), 581–589 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Golub, G.H., Loan, C.F.V.: Matrix Computations. The John Hopkins University Press, Baltimore (1996)zbMATHGoogle Scholar
  19. 19.
    Blake, C.L., Merz, C.J.: UCI Repository of Machine Learning Databases (1998),
  20. 20.
    Rätsch, G.: Benchmark Repository (2003),

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Yinli Dong
    • 1
  • Shuisheng Zhou
    • 2
  1. 1.Foundation DepartmentXian Eurasia UniversityP.R. China
  2. 2.School of ScienceXidian UniversityP.R. China

Personalised recommendations