Entropy-Based Fuzzy Least Squares Twin Support Vector Machine for Pattern Classification

  • Sugen ChenEmail author
  • Junfeng Cao
  • Fenglin Chen
  • Bingbing Liu


Least squares twin support vector machine (LSTSVM) is a new machine learning method, as opposed to solving two quadratic programming problems in twin support vector machine (TWSVM), which generates two nonparallel hyperplanes by solving a pair of linear system of equations. However, LSTSVM obtains the resultant classifier by giving same importance to all training samples which may be important for classification performance. In this paper, by considering the fuzzy membership value for each sample, we propose an entropy-based fuzzy least squares twin support vector machine where fuzzy membership values are assigned based on the entropy values of all training samples. The proposed method not only retains the superior characteristics of LSTSVM which is simple and fast algorithm, but also implements the structural risk minimization principle to overcome the possible over- fitting problem. Experiments are performed on several synthetic as well as benchmark datasets and the experimental results illustrate the effectiveness of our method.


Pattern classification Information entropy Least squares twin support vector machine Fuzzy membership 



This work was partially supported by the National Natural Science Foundation of China (Grant No. 61702012), the University Outstanding Young Talent Support Project of Anhui Province of China (Grant No. gxyq2017026), the University Natural Science Research Project of Anhui Province of China (Grant Nos. KJ2016A431, KJ2017A361 and KJ2017A368) and the Program for Innovative Research Team in Anqing Normal University.

Compliance with Ethical Standards

Conflict of interest

The authors declare that there is no conflict of interests regarding the publication of this paper.


  1. 1.
    Cortes C, Vapnik VN (1995) Support-vector networks. Mach Learn 20(3):273–297zbMATHGoogle Scholar
  2. 2.
    Osuna E, Freund R, Girosi F (1997) Training support vector machines: an application to face detection. In: Proceedings of computer vision and pattern recognition, pp 130–136Google Scholar
  3. 3.
    Noble WS (2004) Kernel methods in computational biology. In: Schölkopf B, Tsuda K, Vert JP (eds) Support vector machine applications in computational biology. MIT, Cambridge, pp 71–92Google Scholar
  4. 4.
    Isa D, Lee LH, Kallimani VP, Rajkumar R (2008) Text document preprocessing with the Bayes formula for classification using the support vector machine. IEEE Trans Knowl Data Eng 20(9):1264–1272CrossRefGoogle Scholar
  5. 5.
    Khan L, Awad M, Thuraisingham B (2007) A new intrusion detection system using support vector machines and hierarchical clustering. VLDB J 16(4):507–521CrossRefGoogle Scholar
  6. 6.
    Schmidt M, Gish H (1996) Speaker identification via support vector classifiers and hierarchical clustering. In: Proceedings of IEEE international conference on acoustics, speech, and signal processing, pp 105–108Google Scholar
  7. 7.
    Zhang J, Liu Y (2004) Cervical cancer detection using SVM-based feature screening. In: Proceedings of international conference on medical image computing and computer-assisted intervention, pp 873–880Google Scholar
  8. 8.
    Fung G, Mangasarian OL (2001) Proximal support vector machine classifiers. In: Proceedings KDD-2001, knowledge discovery and data mining, pp 77–86Google Scholar
  9. 9.
    Mangasarian OL, Wild EW (2006) Multisurface proximal support vector machine classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69–74CrossRefGoogle Scholar
  10. 10.
    Jayadeva Khemchandai R, Chandra S (2007) Twin support vector machine classification for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910CrossRefGoogle Scholar
  11. 11.
    Kumar MA, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543CrossRefGoogle Scholar
  12. 12.
    Shao YH, Zhang CH, Wang XB, Deng NY (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22(6):962–968CrossRefGoogle Scholar
  13. 13.
    Peng XJ (2011) TPMSVM: a novel twin parametric-margin support vector machine for pattern recognition. Pattern Recognit 44(10):2678–2692CrossRefzbMATHGoogle Scholar
  14. 14.
    Qi ZQ, Tian YJ, Shi Y (2013) Robust twin support vector machine for pattern classification. Pattern Recognit 46(1):305–316CrossRefzbMATHGoogle Scholar
  15. 15.
    Tian YJ, Qi ZQ, Ju XC, Shi Y, Liu XH (2014) Nonparallel support vector machines for pattern classification. IEEE Trans Cybern 44(7):1067–1079CrossRefGoogle Scholar
  16. 16.
    Rastogi R, Saigal P, Chandra S (2018) Angle-based twin parametric-margin support vector machine for pattern classification. Knowl Based Syst 139:64–77CrossRefGoogle Scholar
  17. 17.
    Mehrkanoon S, Huang XL, Suykens JAK (2014) Non-parallel support vector classifiers with different loss functions. Neurocomputing 143:294–301CrossRefGoogle Scholar
  18. 18.
    Chen SG, Wu XJ, Zhang RF (2016) A novel twin support vector machine for binary classification problems. Neural Process Lett 44(3):795–811CrossRefGoogle Scholar
  19. 19.
    Ding SF, An YX, Zhang XK, Wu FL, Xue Y (2017) Wavelet twin support vector machine based on glowworm swarm optimization. Neurocomputing 225:157–163CrossRefGoogle Scholar
  20. 20.
    Xu YT, Yang ZJ, Pan XL (2017) A novel twin support vector machine with pinball loss. IEEE Trans Neural Netw Learn Syst 28(2):359–370MathSciNetCrossRefGoogle Scholar
  21. 21.
    Lin CF, Wang SD (2002) Fuzzy support vector machines. IEEE Trans Neural Netw 13(2):464–471CrossRefGoogle Scholar
  22. 22.
    Batuwita R, Palade V (2010) FSVM-CIL: fuzzy support vector machines for class imbalance learning. IEEE Trans Fuzzy Syst 18(3):558–571CrossRefGoogle Scholar
  23. 23.
    Wang Y, Wang S, Lai KK (2005) A new fuzzy support vector machine to evaluate credit risk. IEEE Trans Fuzzy Syst 13(6):820–831CrossRefGoogle Scholar
  24. 24.
    Tao Q, Wang J (2004) A new fuzzy support vector machine based on the weighted margin. Neural Process Lett 20(3):139–150CrossRefGoogle Scholar
  25. 25.
    An WJ, Liang MG (2013) Fuzzy support vector machine based on within-class scatter for classification problems with outliers or noises. Neurocomputing 110(7):101–110CrossRefGoogle Scholar
  26. 26.
    Li K, Ma HY (2013) A fuzzy twin support vector machine algorithm. Int J Appl Innov Eng Manag 2(3):459–465Google Scholar
  27. 27.
    Chen SG, Wu XJ (2017) A new fuzzy twin support vector machine for pattern classification. Int J Mach Learn Cybern 9(9):1553–1564CrossRefGoogle Scholar
  28. 28.
    Shen H, Li F, Wu ZG, Park JH, Sreeram V (2018) Fuzzy-model-based nonfragile control for nonlinear singularly perturbed systems with semi-Markov jump parameters. IEEE Trans Fuzzy Syst 26(6):3428–3439CrossRefGoogle Scholar
  29. 29.
    Shen H, Men YZ, Wu ZG, Cao JD, Lu GP (2019) Network-based quantized control for fuzzy singularly perturbed semi-Markov jump systems and its application. IEEE Trans Circuits Syst I Regul Pap 66(3):1130–1140CrossRefGoogle Scholar
  30. 30.
    Shen H, Li F, Yan HC, Karimi HR, Lam HK (2018) Finite-time event-triggered H control for T-S fuzzy Markov jump systems. IEEE Trans Fuzzy Syst 26(5):3122–3135CrossRefGoogle Scholar
  31. 31.
    Fan Q, Wang Z, Li DD, Gao DQ, Zha HY (2017) Entropy-based fuzzy support vector machine for imbalance datasets. Knowl Based Syst 115:87–99CrossRefGoogle Scholar
  32. 32.
    Gupta D, Richhariya B, Borah P (2018) A fuzzy twin support vector machine based on information entropy for class imbalance learning. Neural Comput Appl. Google Scholar
  33. 33.
    Suykens JAK, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300CrossRefGoogle Scholar
  34. 34.
    Shannon C (2001) A mathematical theory of communication. ACM SIGMOBILE Mobile Comput Commun 5(1):3–55MathSciNetCrossRefGoogle Scholar
  35. 35.
    Benkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7(11):2399–2434MathSciNetzbMATHGoogle Scholar
  36. 36.
    Chen SG, Wu XJ, Xu J (2016) Locality preserving projection twin support vector machine and its application in classification. J Algorithms Comput Technol 10(2):65–72MathSciNetCrossRefGoogle Scholar
  37. 37.
    Ripley BD (2008) Pattern recognition and neural networks. Cambridge University Press, CambridgezbMATHGoogle Scholar
  38. 38.
    Muphy PM, Aha DW (1992) UCI repository of machine learning databases. University of California, Irvine. Accessed 10 Jan 2018
  39. 39.
    Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30MathSciNetzbMATHGoogle Scholar
  40. 40.
    Iman RL, Davenport JM (1980) Approximations of the critical region of the Friedman statistic. Commun Stat Theory Methods 9(6):571–595CrossRefzbMATHGoogle Scholar
  41. 41.
    The USPS Database. Accessed 15 Nov 2018
  42. 42.
    Nene SA, Nayar SK, Murase H (1996) Columbia object image library (COIL-20). Technical report CUCS-005096, FebruaryGoogle Scholar
  43. 43.
    Martinez AM, Benavente R (1998) The AR face database. CVC technical report #24, JuneGoogle Scholar
  44. 44.
    Musicant DR (1998) NDC: Normally distributed clustered datasets. Computer Science Department, University of Wisconsin, Madison, USA. Accessed 16 Feb 2018
  45. 45.
    Golub GH, Van Loan CF (2012) Matrix computations, vol 3. JHU Press, BaltimorezbMATHGoogle Scholar
  46. 46.
    Nasiri JA, Charkari NM, Jalili S (2015) Least squares twin multi-class classification support vector machine. Pattern Recognit 48(3):984–992CrossRefzbMATHGoogle Scholar
  47. 47.
    Chen SG, Wu XJ (2017) Multiple birth least squares support vector machine for multi-class classification. Int J Mach Learn Cybern 8(6):1731–1742CrossRefGoogle Scholar
  48. 48.
    Tang JJ, Li DW, Tian YJ, Liu DL (2018) Multi-view learning based on nonparallel support vector machine. Knowl Based Syst 158:94–108CrossRefGoogle Scholar
  49. 49.
    Houthuys L, Langone R, Suykens JAK (2018) Multi-view least squares support vector machines classification. Neurocomputing 282:78–88CrossRefGoogle Scholar
  50. 50.
    Qi ZQ, Tian YJ, Shi Y (2012) Laplacian twin support vector machine for semi-supervised classification. Neural Netw 35:46–53CrossRefzbMATHGoogle Scholar
  51. 51.
    Chen WJ, Shao YH, Deng NY, Feng ZL (2014) Laplacian least squares twin support vector machine for semi-supervised classification. Neurocomputing 145:465–476CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Mathematics and Computational ScienceAnqing Normal UniversityAnqingPeople’s Republic of China
  2. 2.Key Laboratory of Modeling, Simulation and Control of Complex Ecosystem in Dabie Mountains of Anhui Higher Education InstitutesAnqing Normal UniversityAnqingChina
  3. 3.School of ScienceJiangnan UniversityWuxiPeople’s Republic of China

Personalised recommendations