Skip to main content
Log in

Accelerating improved twin support vector machine with safe screening rule

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Improved twin support vector machine (ITSVM) is a binary classification model with strong theoretical interpretation. Compared with Twin bound support vector machine (TWBSVM), it avoids the matrix inverse operation. However, the disadvantage of slow speed is exposed during the training process. Thus authors are motivated to employ safe screening method to reduce the scale of dual ITSVM and accelerate its computational speed. More specifically, the proposed method is to identify redundant points in advance and eliminate them before actually solving the problem. If safe screening method is directly used to accelerate ITSVM, it will be inevitable to bring matrix inverse operation during the screening process. In this case, a screening method which is distinct from existing safe screening method is devised to avoid calculating inverse matrix. Meanwhile an improved dual coordinate descent method (DCDM) is employed to accelerate ITSVM. Experiments on eleven real data sets are conducted to demonstrate the effectiveness of the proposed acceleration method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  2. Steinwart I, Christmann A (2008) Support vector machines. Springer, New York

    MATH  Google Scholar 

  3. Schölkop B, Smola AJ (2002) Learning with kernels. MIT Press, Cambridge

    Google Scholar 

  4. Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge

    Book  Google Scholar 

  5. Deng N, Tian Y, Zhang C (2013) Support vector machines: optimization based theory, algorithms, and extensions. Chapman and Hall/CRC, New York

    MATH  Google Scholar 

  6. Jayadeva RK, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    Article  Google Scholar 

  7. Jayadeva RK, Chandra S (2017) Twin support vector machines. Springer, Cham

    Book  Google Scholar 

  8. Brugger D (2006) Parallel support vector machines. Universität Tübingen, Tübingen

  9. Tian Y, Qi Z, Ju X, Shi Y, Liu X (2014) Nonparallel support vector machines for pattern classification. IEEE Trans Cybern 44(7):1067–1079

    Article  Google Scholar 

  10. Qi Z, Tian Y, Shi Y (2013) Structural twin support vector machine for classification. Knowl Based Syst 43:74–81

    Article  Google Scholar 

  11. Shao Y, Chen W, Wang Z, Li C, Deng N (2015) Weighted linear loss twin support vector machine for large-scale classification. Knowl Based Syst 73(1):276–288

    Article  Google Scholar 

  12. Peng X, Xu D (2013) A twin-hypersphere support vector machine classifier and the fast learning algorithm. Inf Sci 221:12–27

    Article  MathSciNet  Google Scholar 

  13. Tomar D, Agarwal S (2015) Twin support vector machine: a review from 2007 to 2014. Egypt Inf J 16(1):55–69

    Article  Google Scholar 

  14. Qi Z, Tian Y, Shi Y (2013) Robust twin support vector machine for pattern classification. Pattern Recognit 46(1):305–316

    Article  Google Scholar 

  15. Cevikalp H (2017) Best fitting hyperplanes for classification. IEEE Trans Pattern Anal Mach Intell 39(6):1076–1088

    Article  Google Scholar 

  16. Yan H, Ye Q, Zhang T, Yu D, Yuan X, Xu Y, Fu L (2018) Least squares twin bounded support vector machines based on L1-norm distance metric for classification. Pattern Recognit 74:434–447

    Article  Google Scholar 

  17. Ye Q, Zhao H, Li Z, Yang X, Gao S, Yin T, Ye N (2018) L1-norm distance minimization-based fast robust twin support vector k-plane clustering. IEEE Trans Neural Netw Learn Syst 29:4494–4503

    Article  Google Scholar 

  18. Xu Y, Yang Z, Pan X (2017) A novel twin support-vector machine with pinball loss. IEEE Trans Neural Netw Learn Syst 28(2):359–370

    Article  MathSciNet  Google Scholar 

  19. Arun Kumar M, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543

    Article  Google Scholar 

  20. Yao X, Wang Z, Zhang H (2018) Identification method for a class of periodic discrete-time dynamic nonlinear systems based on Sinusoidal ESN. Neurocomputing 275:1511–1521

    Article  Google Scholar 

  21. El Ghaoui L, Viallon V, Rabbani T (2010) Safe feature elimination in sparse supervised learning. Pac J Optim 8(4):667–698

    MathSciNet  MATH  Google Scholar 

  22. Ogawa K, Suzuki Y, Takeuchi I (2013) Safe screening of mon-support vectors in pathwise SVM computation. In: International conference on machine learning, pp 1382–1390

  23. Wang J, Zhou J, Wonka P, Ye J (2013) Lasso screening rules via dual polytope projection. In: Neural information processing systems, pp 1070–1078

  24. Wang J, Wonka P, Ye J (2014) Scaling svm and least absolute deviations via exact data reduction. In: International conference on machine learning, pp 523–531

  25. Wang J, Zhou J, Liu J, Wonka P, Ye J (2014) A safe screening rule for sparse logistic regression. In: Neural information processing systems, vol 2, pp 1053–1061

  26. Yang T, Wang J, Sun Q, Hibar DP, Jahanshad N, Liu L, Wang Y, Zhan L, Thompson PM, Ye J (2015) Detecting genetic risk factors for Alzheimer's disease in whole genome sequence data via lasso screening. In: IEEE international symposium on biomedical imaging, pp 985–989

  27. Pan X, Yang Z, Xu Y, Wang L (2018) Safe screening rules for accelerating twin support vector machine classification. IEEE Trans Neural Netw Learn Syst 29(5):1876–1887

    Article  MathSciNet  Google Scholar 

  28. Yang Z, Xu Y (2018) A safe screening rule for Laplacian support vector machine. Eng Appl Artif Intell 67:309–316

    Article  Google Scholar 

  29. Pan X, Pang X, Wang H, Xu Y (2018) A safe screening based framework for support vector regression. Neurocomputing 287:163–172

    Article  Google Scholar 

  30. Pang X, Xu C, Xu Y (2018) Scaling KNN multi-class twin support vector machine via safe instance reduction. Knowl Based Syst 148:17–30

    Article  Google Scholar 

  31. Shao Y, Zhang C, Wang X, Deng N (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw Learn Syst 22(6):962–968

    Article  Google Scholar 

  32. Tian Y, Ju X, Qi Z, Shi Y (2014) Improved twin support vector machine. Sci China Math 57(2):417–432

    Article  MathSciNet  Google Scholar 

  33. Güler O (2010) Foundations of optimization. Springer, New York

    Book  Google Scholar 

  34. Hsieh C, Chang K, Lin C, Keerthi SS, Sundararajan S (2008) A dual coordinate descent method for large-scale linear svm. In: International conference on machine learning, pp 408–415

  35. Luo Z, Tseng P (1992) On the convergence of the coordinate descent method for convex differentiable minimization. J Optim Theory Appl 72(1):7–35

    Article  MathSciNet  Google Scholar 

  36. Lichman M (2013) UCI machine learning repository. http://archive.ics.uci.edu/ml/index.php

Download references

Acknowledgements

The authors would like to thank the reviewers for the helpful comments and suggestions, which have improved the presentation. This work was supported in part by National Natural Science Foundation of China (No. 11671010) and Beijing Natural Science Foundation (No.4172035).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yitian Xu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, W., Xu, Y. Accelerating improved twin support vector machine with safe screening rule. Int. J. Mach. Learn. & Cyber. 10, 3587–3600 (2019). https://doi.org/10.1007/s13042-019-00946-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-019-00946-1

Keywords

Navigation