Accelerating improved twin support vector machine with safe screening rule

  • Weichen Wu
  • Yitian XuEmail author
Original Article


Improved twin support vector machine (ITSVM) is a binary classification model with strong theoretical interpretation. Compared with Twin bound support vector machine (TWBSVM), it avoids the matrix inverse operation. However, the disadvantage of slow speed is exposed during the training process. Thus authors are motivated to employ safe screening method to reduce the scale of dual ITSVM and accelerate its computational speed. More specifically, the proposed method is to identify redundant points in advance and eliminate them before actually solving the problem. If safe screening method is directly used to accelerate ITSVM, it will be inevitable to bring matrix inverse operation during the screening process. In this case, a screening method which is distinct from existing safe screening method is devised to avoid calculating inverse matrix. Meanwhile an improved dual coordinate descent method (DCDM) is employed to accelerate ITSVM. Experiments on eleven real data sets are conducted to demonstrate the effectiveness of the proposed acceleration method.


Improved twin support vector machine Safe screening rule Binary classification Matrix inverse operation 



The authors would like to thank the reviewers for the helpful comments and suggestions, which have improved the presentation. This work was supported in part by National Natural Science Foundation of China (No. 11671010) and Beijing Natural Science Foundation (No.4172035).


  1. 1.
    Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297zbMATHGoogle Scholar
  2. 2.
    Steinwart I, Christmann A (2008) Support vector machines. Springer, New YorkzbMATHGoogle Scholar
  3. 3.
    Schölkop B, Smola AJ (2002) Learning with kernels. MIT Press, CambridgeGoogle Scholar
  4. 4.
    Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, CambridgeCrossRefzbMATHGoogle Scholar
  5. 5.
    Deng N, Tian Y, Zhang C (2013) Support vector machines: optimization based theory, algorithms, and extensions. Chapman and Hall/CRC, New YorkzbMATHGoogle Scholar
  6. 6.
    Jayadeva RK, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910CrossRefzbMATHGoogle Scholar
  7. 7.
    Jayadeva RK, Chandra S (2017) Twin support vector machines. Springer, ChamCrossRefzbMATHGoogle Scholar
  8. 8.
    Brugger D (2006) Parallel support vector machines. Universität Tübingen, TübingenGoogle Scholar
  9. 9.
    Tian Y, Qi Z, Ju X, Shi Y, Liu X (2014) Nonparallel support vector machines for pattern classification. IEEE Trans Cybern 44(7):1067–1079CrossRefGoogle Scholar
  10. 10.
    Qi Z, Tian Y, Shi Y (2013) Structural twin support vector machine for classification. Knowl Based Syst 43:74–81CrossRefGoogle Scholar
  11. 11.
    Shao Y, Chen W, Wang Z, Li C, Deng N (2015) Weighted linear loss twin support vector machine for large-scale classification. Knowl Based Syst 73(1):276–288CrossRefGoogle Scholar
  12. 12.
    Peng X, Xu D (2013) A twin-hypersphere support vector machine classifier and the fast learning algorithm. Inf Sci 221:12–27MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Tomar D, Agarwal S (2015) Twin support vector machine: a review from 2007 to 2014. Egypt Inf J 16(1):55–69CrossRefGoogle Scholar
  14. 14.
    Qi Z, Tian Y, Shi Y (2013) Robust twin support vector machine for pattern classification. Pattern Recognit 46(1):305–316CrossRefzbMATHGoogle Scholar
  15. 15.
    Cevikalp H (2017) Best fitting hyperplanes for classification. IEEE Trans Pattern Anal Mach Intell 39(6):1076–1088CrossRefGoogle Scholar
  16. 16.
    Yan H, Ye Q, Zhang T, Yu D, Yuan X, Xu Y, Fu L (2018) Least squares twin bounded support vector machines based on L1-norm distance metric for classification. Pattern Recognit 74:434–447CrossRefGoogle Scholar
  17. 17.
    Ye Q, Zhao H, Li Z, Yang X, Gao S, Yin T, Ye N (2018) L1-norm distance minimization-based fast robust twin support vector k-plane clustering. IEEE Trans Neural Netw Learn Syst 29:4494–4503CrossRefGoogle Scholar
  18. 18.
    Xu Y, Yang Z, Pan X (2017) A novel twin support-vector machine with pinball loss. IEEE Trans Neural Netw Learn Syst 28(2):359–370MathSciNetCrossRefGoogle Scholar
  19. 19.
    Arun Kumar M, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543CrossRefGoogle Scholar
  20. 20.
    Yao X, Wang Z, Zhang H (2018) Identification method for a class of periodic discrete-time dynamic nonlinear systems based on Sinusoidal ESN. Neurocomputing 275:1511–1521CrossRefGoogle Scholar
  21. 21.
    El Ghaoui L, Viallon V, Rabbani T (2010) Safe feature elimination in sparse supervised learning. Pac J Optim 8(4):667–698MathSciNetzbMATHGoogle Scholar
  22. 22.
    Ogawa K, Suzuki Y, Takeuchi I (2013) Safe screening of mon-support vectors in pathwise SVM computation. In: International conference on machine learning, pp 1382–1390Google Scholar
  23. 23.
    Wang J, Zhou J, Wonka P, Ye J (2013) Lasso screening rules via dual polytope projection. In: Neural information processing systems, pp 1070–1078Google Scholar
  24. 24.
    Wang J, Wonka P, Ye J (2014) Scaling svm and least absolute deviations via exact data reduction. In: International conference on machine learning, pp 523–531Google Scholar
  25. 25.
    Wang J, Zhou J, Liu J, Wonka P, Ye J (2014) A safe screening rule for sparse logistic regression. In: Neural information processing systems, vol 2, pp 1053–1061Google Scholar
  26. 26.
    Yang T, Wang J, Sun Q, Hibar DP, Jahanshad N, Liu L, Wang Y, Zhan L, Thompson PM, Ye J (2015) Detecting genetic risk factors for Alzheimer's disease in whole genome sequence data via lasso screening. In: IEEE international symposium on biomedical imaging, pp 985–989Google Scholar
  27. 27.
    Pan X, Yang Z, Xu Y, Wang L (2018) Safe screening rules for accelerating twin support vector machine classification. IEEE Trans Neural Netw Learn Syst 29(5):1876–1887MathSciNetCrossRefGoogle Scholar
  28. 28.
    Yang Z, Xu Y (2018) A safe screening rule for Laplacian support vector machine. Eng Appl Artif Intell 67:309–316CrossRefGoogle Scholar
  29. 29.
    Pan X, Pang X, Wang H, Xu Y (2018) A safe screening based framework for support vector regression. Neurocomputing 287:163–172CrossRefGoogle Scholar
  30. 30.
    Pang X, Xu C, Xu Y (2018) Scaling KNN multi-class twin support vector machine via safe instance reduction. Knowl Based Syst 148:17–30CrossRefGoogle Scholar
  31. 31.
    Shao Y, Zhang C, Wang X, Deng N (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw Learn Syst 22(6):962–968CrossRefGoogle Scholar
  32. 32.
    Tian Y, Ju X, Qi Z, Shi Y (2014) Improved twin support vector machine. Sci China Math 57(2):417–432MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Güler O (2010) Foundations of optimization. Springer, New YorkCrossRefzbMATHGoogle Scholar
  34. 34.
    Hsieh C, Chang K, Lin C, Keerthi SS, Sundararajan S (2008) A dual coordinate descent method for large-scale linear svm. In: International conference on machine learning, pp 408–415Google Scholar
  35. 35.
    Luo Z, Tseng P (1992) On the convergence of the coordinate descent method for convex differentiable minimization. J Optim Theory Appl 72(1):7–35MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Lichman M (2013) UCI machine learning repository.

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.College of ScienceChina Agricultural UniversityBeijingChina

Personalised recommendations