Advertisement

Analyzing l1-loss and l2-loss Support Vector Machines Implemented in PERMON Toolbox

  • Marek PechaEmail author
  • David Horák
Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 554)

Abstract

This paper deals with investigating l1-loss and l2-loss l2-regularized Support Vector Machines implemented in PermonSVM – a part of our PERMON toolbox. The loss functions quantify error between predicted and correct classifications of samples in cases of non-perfectly linearly separable classifications. In numerical experiments, we study properties of Hessians related to performance score of models and analyze convergence rate on 4 public available datasets. The Modified Proportioning and Reduced Gradient Projection algorithm is used as a solver for the dual Quadratic Programming problem resulting from Support Vector Machines formulations.

Keywords

Support Vector Machines SVM PermonSVM Hinge loss functions Quadratic Programming QP MPRGP 

Notes

Acknowledgments

This work has been supported by The Ministry of Education, Youth and Sports from the National Programme of Sustainability (NPU II) project IT4Innovations excellence in science - LQ1602; by Grant of SGS No. SP2018/165, VŠB - Technical University of Ostrava, Czech Republic and by the grant of the Czech Science Foundation (GACR) project no. GA17-22615S. The work has been also performed under the Project HPC-EUROPA3 (INFRAIA-2016-1-730897), with the support of the EC Research Innovation Action under the H2020 Programme; in particular, the author gratefully acknowledges the support of School of Mathematics, The University of Edinburgh, United Kingdom and the computer resources and technical support provided by Edinburgh Parallel Computing Centre (EPCC).

References

  1. 1.
  2. 2.
    Balay, S., Abhyankar, S., Adams, M.F., Brown, J., Brune, P., Buschelman, K., Eijkhout, V., Gropp, W.D., Kaushik, D., Knepley, M.G., McInnes, L.C., Rupp, K., Smith, B.F., Zhang, H.: PETSc – Portable, Extensible Toolkit for Scientific Computation. http://www.mcs.anl.gov/petsc
  3. 3.
    Brown, M., Grundy, W., Lin, D., Cristianini, N., Sugnet, C., Furey, T., Ares Jr., M., Haussler, D.: Knowledge-based analysis of microarray gene expression data by using support vector machines. Proc. Nat. Acad. Sci. U.S.A. 97(1), 262–267 (2000)CrossRefGoogle Scholar
  4. 4.
    Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)zbMATHGoogle Scholar
  5. 5.
    Dostál, Z., Pospíšil, L.: Minimizing quadratic functions with semidefinite hessian subject to bound constraints. Comput. Math. Appl. 70(8), 2014–2028 (2015)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Dostál, Z.: Optimal Quadratic Programming Algorithms, with Applications to Variational Inequalities, vol. 23. SOIA. Springer, New York (2009)Google Scholar
  7. 7.
    Foody, G.M., Mathur, A.: The use of small training sets containing mixed pixels for accurate hard image classification: training on mixed spectral responses for classification by a SVM. Remote Sens. Environ. 103(2), 179–189 (2006)CrossRefGoogle Scholar
  8. 8.
    Hapla, V., Horák, D., Pecha, M.: PermonSVM (2017). http://permon.vsb.cz/permonsvm.htm
  9. 9.
    Hapla, V., Horák, D., Čermák, M., Kružík, J., Pospíšil, L., Sojka, R.: PermonQP (2015). http://permon.it4i.cz/qp/
  10. 10.
    Hapla, V., et al.: PERMON (Parallel, Efficient, Robust, Modular, Object-oriented, Numerical). http://permon.vsb.cz
  11. 11.
    Hsieh, C.J., Chang, K.W., Lin, C.J., Keerthi, S.S., Sundararajan, S.: A dual coordinate descent method for large-scale linear SVM. In: Proceedings of the 25th International Conference on Machine Learning, pp. 408–415. ACM (2008)Google Scholar
  12. 12.
    Kružík, J., Pecha, M., Hapla, V., Horák, D., Čermák, M.: Investigating Convergence of Linear SVM Implemented in PermonSVM Employing MPRGP Algorithm. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11087 LNCS, pp. 115–129 (2018)Google Scholar
  13. 13.
    Pecha, M., Hapla, V., Horák, D., Čermák, M.: Notes on the preliminary results of a linear two-class classifier in the permon toolbox. In: AIP Conference Proceedings, vol. 1978 (2018)Google Scholar
  14. 14.
    Shi, J., Lee, W.J., Liu, Y., Yang, Y., Wang, P.: Forecasting power output of photovoltaic systems based on weather classification and support vector machines. IEEE Trans. Ind. Appl. 48(3), 1064–1069 (2012)CrossRefGoogle Scholar
  15. 15.
    Vishnu, A., Narasimhan, J., Holder, L., Kerbyson, D., Hoisie, A.: Fast and accurate support vector machines on large scale systems. In: 2015 IEEE International Conference on Cluster Computing, pp. 110–119, September 2015Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Department of Applied MathematicsVŠB – Technical University of OstravaOstravaCzech Republic
  2. 2.Institute of Geonics CASOstravaCzech Republic

Personalised recommendations