Asymmetric ν-twin support vector regression

Original Article
  • 97 Downloads

Abstract

Twin support vector regression (TSVR) aims at finding 𝜖-insensitive up- and down-bound functions for the training points by solving a pair of smaller-sized quadratic programming problems (QPPs) rather than a single large one as in the conventional SVR. So TSVR works faster than SVR in theory. However, TSVR gives equal emphasis to the points above the up-bound and below the down-bound, which leads to the same influences on the regression function. In fact, points in different positions have different effects on the regressor. Inspired by it, we propose an asymmetric ν-twin support vector regression based on pinball loss function (Asy- ν-TSVR). The new algorithm can effectively control the fitting error by tuning the parameters ν and p. Therefore, it enhances the generalization ability. Moreover, we study the distribution of samples and give the upper bounds for the samples locating in different positions. Numerical experiments on one artificial dataset, eleven benchmark datasets and a real wheat dataset demonstrate the validity of our proposed algorithm.

Keywords

Support vector regression Twin support vector regression Pinball loss Asymmetric 

Notes

Acknowledgments

The authors gratefully acknowledge the helpful comments and suggestions of the reviewers, which have improved the presentation. This work was supported in part by the National Natural Science Foundation of China (No. 11671010) and Natural Science Foundation of Beijing Municipality (No. 4172035).

Compliance with ethical standards

Conflict of interests

The authors declare that they have no conflict of interest.

References

  1. 1.
    Steinwart I, Christmann A (2008) Support vector machines. Springer, New YorkMATHGoogle Scholar
  2. 2.
    Vapnik V (1995) The nature of statistical learning theory. Springer, New YorkCrossRefMATHGoogle Scholar
  3. 3.
    Shawe-Taylor J, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge University PressGoogle Scholar
  4. 4.
    Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University PressGoogle Scholar
  5. 5.
    Schölkopf B, Smola A, Williamson RC, Bartlett PL (2000) New support vector algorithms. Neural Comput 12:1207–1245CrossRefGoogle Scholar
  6. 6.
    Bi J, Bennett KP (2003) A geometric approach to support vector regression. Neurocomputing 55:79–108CrossRefGoogle Scholar
  7. 7.
    Peng XJ (2010) TSVR: an efficient twin support vector machine for regression. Neural Netw 23:365–372CrossRefGoogle Scholar
  8. 8.
    Peng XJ (2012) Efficient twin parametric insensitive support vector regression model. Neurocomputing 79:26–38CrossRefGoogle Scholar
  9. 9.
    Peng XJ, Xu D, Shen JD (2014) A twin projection support vector machine for data regression. Neurocomputing 138:131–141CrossRefGoogle Scholar
  10. 10.
    Santanu G, Mukherjee A, Dutta PK (2009) Nonparallel plane proximal classifier. Signal Process 89:510–522CrossRefMATHGoogle Scholar
  11. 11.
    Jayadeva KR, Chandra S (2007) Khemchandani Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29:905–910CrossRefGoogle Scholar
  12. 12.
    Xu YT, Xi WW, Lv X (2012) An improved least squares twin support vector machine. J Info Comput Sci 9:1063–1071Google Scholar
  13. 13.
    Singh M, Chadha J, Ahuja P, Jayadeva S (2011) Chandra Reduced twin support vector regression. Neurocomputing 74:1474– 1477CrossRefGoogle Scholar
  14. 14.
    Zhao YP, Zhao J, Zhao M (2013) Twin least squares support vector regression. Neurocomputing 118:225–236CrossRefGoogle Scholar
  15. 15.
    Kumar MA, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36:7535–7543CrossRefGoogle Scholar
  16. 16.
    Tomar D, Agarwal S (2015) Twin Support Vector Machine: a review from 2007 to 2014. Egyptian Info J 16:55–69CrossRefGoogle Scholar
  17. 17.
    Huang XL, Shi L, Suykens JAK (2014) Support vector machine classifier with pinball loss. IEEE Trans Pattern Anal Mach Intell 36:984–997CrossRefGoogle Scholar
  18. 18.
    Huang XL, Shi L, Pelckmans K, Suykens JAK (2014) Asymmetric ν-tube support vector regression. Comput Stat Data Anal 77:371–382MathSciNetCrossRefGoogle Scholar
  19. 19.
    Xu YT, Yang ZJ, Zhang YQ, Pan XL, Wang LS (2016) A maximum margin and minimum volume hyper-spheres machine with pinball loss for imbalanced data classification. Knowl-Based Syst 95:75–85CrossRefGoogle Scholar
  20. 20.
    Xu YT, Yang ZJ, Pan XL (2016) A novel twin support vector machine with pinball loss. IEEE Trans Neural Netw Learn Syst 28(2):359–370MathSciNetCrossRefGoogle Scholar
  21. 21.
    Hao PY (2010) New support vector algorithms with parametric insensitive/margin model. Neural Netw 23:60–73CrossRefGoogle Scholar
  22. 22.
    Le Masne Q, Pothier H, Birge NO, Urbina C, Esteve D (2009) Asymmetric noise probed with a josephson junction. Phys Rev Lett 102:067002CrossRefGoogle Scholar
  23. 23.
    Yu K, Moyeed RA (2001) Bayesian quantile regression. Stat Prob Lett 54:437–447MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Sengupta RN (2008) Use of asymmetric loss functions in sequential estimation problems for multiple linear regression. J Appl Stat 35:245–261MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    Xu YT, Guo R (2014) An improved ν-twin support vector machine. Appl Intell 41:42–54CrossRefGoogle Scholar
  26. 26.
    Xu YT, Wang L, Zhong P (2012) A rough margin-based ν-twin support vector machine. Neural Comput Applic 21:1307–1317CrossRefGoogle Scholar
  27. 27.
    Steinwart I, Christmann A (2011) Estimating conditional quantiles with the help of the pinball loss. Bernoulli 17:211–225MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    Suykens JAK, Tony VG, Jos DB et al (2002) Least squares support vector machines. World Scientific Pub Co, SingaporeCrossRefMATHGoogle Scholar
  29. 29.
    Xu YT (2012) A rough margin-based linear ν support vector regression. Stat Prob Lett 82:528–534MathSciNetCrossRefMATHGoogle Scholar
  30. 30.
    Xu YT, Wang LS (2012) A weighted twin support vector regression. Knowl-Based Syst 33:92–101CrossRefGoogle Scholar
  31. 31.
    Navia-Vzquez F, Prez-Cruz A, Arts-Rodrguezand A, Figueiras-Vidal R (2001) Weighted least squares training of support vectors classifiers which leads to compact and adaptive schemes. IEEE Trans Neural Netw 12 (5):1047–1059CrossRefGoogle Scholar
  32. 32.
    Prez-Cruz J, Herrmann DJL, Scholkopf B (2003) Weston Weston Extension of the nu-SVM range for classification. In: Prez-Cruz J, Herrmann DJL, Scholkopf B (eds) Advances in learning theory: methods, models and applications. IOS Press, pp 179–196Google Scholar
  33. 33.
    Scholkopf B, Smola A, Bartlett P, Williamson R (2000) New support vector algorithms. Neural Comput 12(5):1207–1245CrossRefGoogle Scholar
  34. 34.
    Xu YT, Wang LS (2014) k-nearest neighbor-based weighted twin support vector regression. Appl Intell 41:299–309CrossRefGoogle Scholar
  35. 35.
    Gibbons JD, Chakraborti S (2011) Nonparametric statistical inference, 5th Ed. Chapman & Hall CRC Press, Taylor & Francis Group, Boca RatonMATHGoogle Scholar
  36. 36.
    Dems̆ar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30MathSciNetGoogle Scholar
  37. 37.
    García S, Fernández A, Luengo J, Herrera F (2010) Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining. Experimental analysis of power. Info Sci 180:2044–2064Google Scholar

Copyright information

© The Natural Computing Applications Forum 2017

Authors and Affiliations

  1. 1.College of ScienceChina Agricultural UniversityBeijingChina
  2. 2.Donlinks School of Economics and ManagementUniversity of Science and Technology BeijingBeijingChina

Personalised recommendations