Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

DC programming and DCA for parametric-margin ν-support vector machine

  • 45 Accesses

Abstract

As a development of ν-support vector machine (ν-SVM), parametric-margin ν-support vector machine (Par-ν-SVM) can be useful in many cases, especially heteroscedastic noise classification problems. The present article proposes a novel and fast method to solve the primal problem of Par-ν-SVM (named as DC-Par-ν-SVM), while Par-ν-SVM maximizes the parametric-margin by solving a dual quadratic programming problem. In fact, the primal non-convex problem is converted into an unconstrained problem to express the objective function as the difference of convex functions (DC). The DC-Algorithm (DCA) based on generalized Newton’s method is proposed to solve the unconstrained problem cited. Numerical experiments performed on several artificial, real-life, UCI and NDC data sets showed the superiority of the DC-Par-ν-SVM in terms of both accuracy and learning speed.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. 1.

    Angulo C, Parra X, Catala A (2003) K-SVCR. A support vector machine for multi-class classification. Neurocomputing 55(1-2):57–77

  2. 2.

    Artacho FJA, Fleming RM, Vuong PT (2018) Accelerating the DC algorithm for smooth functions. Math Program 169(1):95–118

  3. 3.

    Amin M, Ali A (2018) Performance evaluation of supervised machine learning classifiers for predicting healthcare operational decisions. Wavy AI Research Foundation, Lahore

  4. 4.

    Ayres-de-Campos D, Bernardes J, Garrido A, Marques-de-Sa J, Pereira-Leite L (2000) SisPorto 2.0: a program for automated analysis of cardiotocograms. Journal of Maternal-Fetal Medicine 9(5):311–318

  5. 5.

    Bennett KP, Bredensteiner EJ (2000) Duality and geometry in SVM classifiers. In: ICML, pp 57–64

  6. 6.

    Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge University Press, Cambridge

  7. 7.

    Bradley PS, Mangasarian OL (2000) Massive data discrimination via linear support vector machines. Optimization methods and software 13(1):1–10

  8. 8.

    Burges CJ (1998) A tutorial on support vector machines for pattern recognition. Data mining and knowledge discovery 2(2):121–167

  9. 9.

    Belghiti MT, Tao PD (2007) A new efficient algorithm based on DC programming and DCA for clustering. J Glob Optim 37(4):593–608

  10. 10.

    Chen X, Yang J, Liang J (2012) A flexible support vector machine for regression. Neural Comput Appl 21(8):2005–2013

  11. 11.

    Cherkassky V, Mulier FM (2007) Learning from data: Concepts, theory, and methods. Wiley, New York

  12. 12.

    Clarke FH (1990) Optimization and nonsmooth analysis, Siam

  13. 13.

    Daniel WW (1990) Friedman two-way analysis of variance by ranks, Applied nonparametric statistics, 262–274

  14. 14.

    Ding S, An Y, Zhang X, Wu F, Xue Y (2017) Wavelet twin support vector machines based on glowworm swarm optimization. Neurocomputing 225:157–163

  15. 15.

    Friedman M (1940) A comparison of alternative tests of significance for the problem of m rankings. The Annals of Mathematical Statistics 11(1):86–92

  16. 16.

    Hao PY (2010) New support vector algorithms with parametric insensitive/margin model. Neural Netw 23 (1):60–73

  17. 17.

    Hiriart-Urruty JB, Strodiot JJ, Nguyen VH (1984) Generalized Hessian matrix and second-order optimality conditions for problems with C 1,1 data. Appl Math Optim 11(1):43–56

  18. 18.

    Horst R, Pardalos PM, Van Thoai N (2000) Introduction to global optimization, Springer Science & Business Media

  19. 19.

    Hong ZQ, Yang JY (1991) Optimal discriminant plane for a small number of samples and design method of classifier on the plane. Pattern Recogn 24(4):317–324

  20. 20.

    Iman RL, Davenport JM (1980) Approximations of the critical region of the fbietkan statistic. Communications in Statistics-Theory and Methods 9(6):571–595

  21. 21.

    Karasuyama M, Harada N, Sugiyama M, Takeuchi I (2012) Multi-parametric solution-path algorithm for instance-weighted support vector machines. Mach Learn 88(3):297–330

  22. 22.

    Ketabchi S, Moosaei H, Razzaghi M, Pardalos PM (2019) An improvement on parametric ν-support vector algorithm for classification. Ann Oper Res 276(1-2):155–168

  23. 23.

    Ketabchi S, Moosaei H (2012) Minimum norm solution to the absolute value equation in the convex case. J Optim Theory Appl 154(3):1080–1087

  24. 24.

    Khozeimeh F, Alizadehsani R, Roshanzamir M, Khosravi A, Layegh P, Nahavandi S (2017) An expert system for selecting wart treatment method. Comput Bio Med 81:167–175

  25. 25.

    Khozeimeh F, Jabbari Azad F, Mahboubi Oskouei Y, Jafari M, Tehranian S, Alizadehsani R, Layegh P (2017) Intralesional immunotherapy compared to cryotherapy in the treatment of warts. Int J Dermatology 56 (4):474–478

  26. 26.

    Koczkodaj WW, Kakiashvili T, Szymanska A, Montero-Marin J, Araya R, Garcia-Campayo J, Rutkowski K, Strzalka D (2017) How to reduce the number of rating scale items without predictability loss. Scientometrics 111(2):581–593

  27. 27.

    Le Thi HA, Dinh TP, Yen ND (2011) Properties of two DC algorithms in quadratic programming. J Glob Optim 49(3):481–495

  28. 28.

    Le Thi HA, Dinh TP (2018) DC programming and DCA: thirty years of developments. Math Program 169(1):5–68

  29. 29.

    Lichman M (2013) UCI machine learning repository. University of California, School of Information and Computer Science , Irvine

  30. 30.

    Lima MD, Costa NL, Barbosa R (2018) Improvements on least squares twin multi-class classification support vector machine. Neurocomputing 313:196–205

  31. 31.

    Mayoraz E, Alpaydin E (1999) Support vector machines for multi-class classification. In: International work-conference on artificial neural networks. Springer, Berlin

  32. 32.

    Melki G, Kecman V, Ventura S, Cano A (2018) OLLAWV:online learning algorithm using worst-violators. Appl Soft Comput 66:384–393

  33. 33.

    Musicant DR (1998) NDC: Normally distributed clustered data sets. Computer Sciences Department, University of Wisconsin

  34. 34.

    Pardalos PM, Ketabchi S, Moosaei H (2014) Minimum norm solution to the positive semidefinite linear complementarity problem. Optimization 63(3):359–369

  35. 35.

    Patricio M, Pereira J, Crisostomo J, Matafome P, Gomes M, Seica R, Caramelo F (2018) Using Resistin, glucose, age and BMI to predict the presence of breast cancer. BMC Cancer 18 (1):29

  36. 36.

    Peng X (2011) TPMSVM: A novel twin parametric-margin support vector machine for pattern recognition. Pattern Recogn 44(10-11):2678–2692

  37. 37.

    Schölkopf B, Smola AJ, Bach F (2002) Learning with kernels:support vector machines, regularization, optimization, and beyond. MIT Press, Cambridge

  38. 38.

    Schölkopf B, Smola AJ, Williamson RC, Bartlett PL (2000) New support vector algorithms. Neural Comput 12(5):1207–1245

  39. 39.

    Tanveer M, Khan MA, Ho SS (2016) Robust energy-based least squares twin support vector machines. Appl Intell 45(1):174– 186

  40. 40.

    Tao PD, Muu LD (1996) Numerical solution for optimization over the efficient set by dc optimization algorithms. Oper Res Lett 19(3):117–128

  41. 41.

    Vapnik V, Chervonenkis A (1974) Theory of pattern recognition. Nauka, Moscow. [in Russian]

  42. 42.

    Vapnik V (2013) The nature of statistical learning theory, Springer science & business media

  43. 43.

    Wang H, Zhou Z, Xu Y (2018) An improved ν-twin bounded support vector machine. Appl Intell 48(4):1041–1053

  44. 44.

    Xu Y (2016) K-nearest neighbor-based weighted multi-class twin support vector machine. Neurocomputing 205:430–438

Download references

Author information

Correspondence to Saeed Ketabchi.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Bazikar, F., Ketabchi, S. & Moosaei, H. DC programming and DCA for parametric-margin ν-support vector machine. Appl Intell (2020). https://doi.org/10.1007/s10489-019-01618-x

Download citation

Keywords

  • Support vector machine
  • Non-convex optimization
  • Generalized Newton’s method
  • DC programming
  • DCA