A Fuzzy Universum Support Vector Machine Based on Information Entropy

  • B. RichhariyaEmail author
  • M. Tanveer
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 748)


Universum-based support vector machines (USVMs) are known to give better generalization performance than standard SVM methods by incorporating prior information about the data. In datasets involving noise and outliers, this universum-based scheme is not so effective because the generated universum data points do not lie in between the two classes. In this paper, we propose a fuzzy universum support vector machine (FUSVM) by introducing the weights to the universum data points based on their information entropy. Since there is no standard approach of selecting the universum, our information entropy based approach is helpful in giving less weight to the outlier universum points and thus gives prior information about the data in an appropriate manner. In addition, we also propose a fuzzy-based approach for universum twin support vector machine named as fuzzy universum twin support vector machine (FUTSVM). Experimental results on several benchmark datasets indicate that, comparing to SVM, USVM, TWSVM and UTSVM our proposed FUSVM and FUTSVM have shown better generalization performance.


Universum Fuzzy membership Information entropy K-nearest neighbour (KNN) 



This work was supported by Science and Engineering Research Board (SERB) as Early Career Research Award grant no. ECR/2017/000053 and Department of Science and Technology as Ramanujan fellowship grant no. SB/S2/RJN-001/2016. We gratefully acknowledge the Indian Institute of Technology Indore for providing facilities and support.


  1. 1.
    Vapnik, V.N.: Statistical Learning Theory. Wiley, New York (1998)zbMATHGoogle Scholar
  2. 2.
    Vapnik, V.N.: The Nature of Statistical Learning Theory, 2nd edn. Springer, New York (2000)CrossRefGoogle Scholar
  3. 3.
    Osuna, E., Freund, R., Girosi, F.: Training support vector machines: an application to face detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 130–136 (1997)Google Scholar
  4. 4.
    Phillips, P.J.: Support vector machines applied to face recognition, In: Proceedings of the 1998 Conference on Advances in Neural Information Processing Systems, vol. 11, pp. 803–809 (1998)Google Scholar
  5. 5.
    Michel, P., El Kaliouby R.: Real time facial expression recognition in video using support vector machines. In: Proceedings of the 5th International Conference on Multimodal Interfaces, pp. 258–264 (2003). ISBN 1-58113-621-8Google Scholar
  6. 6.
    Borovikov, E.: An evaluation of support vector machines as a pattern recognition tool. University of Maryland at College Park (2005).
  7. 7.
    Schmidt, M. Gish, H.: Speaker identification via support vector classifiers. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. ICASSP-96, Atlanta, GA, vol. 1, pp. 105–108 (1996)Google Scholar
  8. 8.
    Khan, L., Awad, M., Thuraisingham, B.: A new intrusion detection system using support vector machines and hierarchical clustering. VLDB J. 16, 507–521 (2007)CrossRefGoogle Scholar
  9. 9.
    Jayadeva, K.R., Chandra, S.: Twin support vector machines for pattern classification. In: IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 29, pp. 905–910 (2007)Google Scholar
  10. 10.
    Kumar, M.A., Gopal, M.: Least squares twin support vector machines for pattern classification. Expert Syst. Appl. 36(4), 7535–7543 (2009)CrossRefGoogle Scholar
  11. 11.
    Shao, Y.H., Zhang, C.H., Wang, X.B., Deng, N.Y.: Improvements on twin support vector machines. IEEE Trans. Neural Netw. 22(6), 962–968 (2011)CrossRefGoogle Scholar
  12. 12.
    Tanveer, M., Khan, M.A., Ho, S.S.: Robust energy-based least squares twin support vector machines. Appl. Intell. 45(1), 174–186 (2016)CrossRefGoogle Scholar
  13. 13.
    Tanveer, M.: Robust and sparse linear programming twin support vector machines. Cogn. Comput. 7(1), 137–149 (2015)CrossRefGoogle Scholar
  14. 14.
    Tanveer, M.: Application of smoothing techniques for linear programming twin support vector machines. Knowl. Inf. Syst. 45(1), 191–214 (2015)CrossRefGoogle Scholar
  15. 15.
    Tanveer, M., Shubham, K., Aldhaifallah, M., Ho, S.S.: An efficient regularized K-nearest neighbor based weighted twin support vector regression. Knowl. Based Syst. 94, 70–87 (2016)CrossRefGoogle Scholar
  16. 16.
    Chen, W.J., Shao, Y.H., Li, C.N., Deng, N.Y.: MLTSVM: a novel twin support vector machine to multi-label learning. Pattern Recogn. 52, 61–74 (2016)CrossRefGoogle Scholar
  17. 17.
    Khemchandani, R., Goyal, K., Chandra, S.: TWSVR: regression via twin support vector machine. Neural Netw. 74, 14–21 (2016)CrossRefGoogle Scholar
  18. 18.
    Tanveer, M., Mangal, M., Ahmad, I., Shao, Y.H.: One norm linear programming support vector regression. Neurocomputing 173, 1508–1518 (2016)CrossRefGoogle Scholar
  19. 19.
    Khemchandani, R., Chandra, S.: Knowledge based proximal support vector machines. Eur. J. Oper. Res. 195(3), 914–923 (2009)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Weston, J., Collobert, R., Sinz, F., Bottou, L., Vapnik, V.: Inference with the universum. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 1009–1016. ACM, 25 Jun 2006Google Scholar
  21. 21.
    Bai, X., Cherkassky V.: Gender classification of human faces using inference through contradictions. In: Proceedings of the IEEE International Joint Conference (IEEE World Congress on Computational Intelligence) on Neural Networks. IJCNN 2008, pp. 746–750, 1 Jun 2008Google Scholar
  22. 22.
    Chen, S., Zhang, C.: Selecting informative universum sample for semi-supervised learning. In: IJCAI, vol. 6, pp. 1016–1021, 11 Jul 2009Google Scholar
  23. 23.
    Cherkassky, V., Dhar, S., Dai, W.: Practical conditions for effectiveness of the universum learning. IEEE Trans. Neural Netw. 22(8), 1241–1255 (2011)Google Scholar
  24. 24.
    Qi, Z., Tian, Y., Shi Y.: Twin support vector machine with universum data. Neural Netw. 36, 112–119 (2012)Google Scholar
  25. 25.
    Xu, Y., Chen M., Li, G.: Least squares twin support vector machine with universum data for classification. Int. J. Syst. Sci. 47(15), 3637–3645 (2016)Google Scholar
  26. 26.
    Lin, C.F., Wang, S.D.: Fuzzy support vector machines. IEEE Trans. Neural Netw. 13(2), 464–471 (2002)CrossRefGoogle Scholar
  27. 27.
    Batuwita, R., Palade, V.: FSVM-CIL: fuzzy support vector machines for class imbalance learning. IEEE Trans. Fuzzy Syst. 18(3), 558–571 (2010)Google Scholar
  28. 28.
    Balasundaram, S., Tanveer, M.: On proximal bilateral-weighted fuzzy support vector machine classifiers. Int. J. Adv. Intel. Paradig. 4(3–4), 199–210 (2012)CrossRefGoogle Scholar
  29. 29.
    Tian, D.Z., Peng G.B., Ha, M.H.: Fuzzy support vector machine based on non-equilibrium data. International Conference on Machine Learning and Cybernetics, Xi’an, China, pp. 15–17 (2012)Google Scholar
  30. 30.
    Wang, Y., Wang, S., Lai, K.K.: A new fuzzy support vector machine to evaluate credit risk. IEEE Trans. Fuzzy Syst. 13(6), 820–831 (2005)CrossRefGoogle Scholar
  31. 31.
    Chaudhuri, A., De, K.: Fuzzy support vector machine for bankruptcy prediction. Appl. Soft Comput. 11(2), 2472–2486 (2010)CrossRefGoogle Scholar
  32. 32.
    Fan, Q., Wang, Z., Li, D., Gao, D., Zha, H.: Entropy-based fuzzy support vector machine for imbalanced datasets. Knowl. Based Syst. 115, 87–99 (2017)CrossRefGoogle Scholar
  33. 33.
    Alcalá-Fdez, J., Fernandez, A., Luengo, J., Derrac, J., García, S., Sánchez, L., Herrera, F.: KEEL Data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. J. Multiple-valued Log. Soft Comput. 17, 255–287 (2011)Google Scholar
  34. 34.
    Murphy, P.M., Aha, D.W.: UCI repository of machine learning databases, University of California, Irvine (1992).
  35. 35.
    Tsang, I., Kocsor, A., Kwok, J.: Efficient kernel feature extraction for massive data sets. In: International Conference on Knowledge Discovery and Data Mining (2006)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.Discipline of MathematicsIndian Institute of Technology IndoreSimrol, IndoreIndia

Personalised recommendations