Advertisement

Twin maximum entropy discriminations for classification

  • Xijiong XieEmail author
  • Huahui Chen
  • Jiangbo Qian
Article
  • 21 Downloads

Abstract

Maximum entropy discrimination (MED) is an excellent classification method based on the maximum entropy and maximum margin principles, and can produce hard-margin support vector machines (SVMs) under certain condition. In this paper, we propose a novel maximum entropy discrimination classifier called twin maximum entropy discriminations (TMED) which construct two discrimination functions for two classes such that each discrimination function is closer to one of the two classes and is at least γt distance from the other. Therefore, it is more flexible and has better generalization ability than typical MED. Furthermore, it solves a pair of convex optimization problems and has the same advantages as those of non-parallel SVM (NPSVM) which is only the special case of our TMED when the priors and parameters are chosen appropriately. It also owns the inherent sparseness as MED. Experimental results confirm the effectiveness of our proposed method.

Keywords

Maximum entropy discrimination Non-parallel support vector machines Maximum margin principle Convex optimization problems 

Notes

Acknowledgments

This work is supported by Ningbo University talent project 421703670 as well as programs sponsored by K.C. Wong Magna Fund in Ningbo University. It is also supported by the Research Foundation of Education Department of Zhejiang Province under Project Y201635608, and the Zhejiang Provincial Natural Science Foundation of China under Project LQ18F020001, the Natural Science Foundation of Ningbo city of Zhejiang Province of China under Project 2018A610155, the Open Project Program of the State Key Lab of CAD&CG in Zhejiang University, under Project A1815, the National Natural Science Foundation of China under Projects 61572266 and 61472194.

References

  1. 1.
    Jaakkola T, Meila M, Jebara T (1999) Maximum entropy discrimination. In: Proceedings of the 13th Annual Conference on Neural Information Processing Systems, pp 470–476Google Scholar
  2. 2.
    Shawe-Taylor J, Sun S (2011) A review of optimization methodologies in support vector machines. Neurocomputing 74:3609–3618CrossRefGoogle Scholar
  3. 3.
    Jebara T, Jaakkola T (2000) Feature selection and dualities in maximum entropy discrimination. In: Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, pp 291–300Google Scholar
  4. 4.
    Jebara T (2004) Multi-task feature and kernel selection for SVMs. In: Proceedings of the 21st International Conference on Machine Learning, pp 55–62Google Scholar
  5. 5.
    Jebara T (2011) Multitask sparsity via maximum entropy discrimination. J Mach Learn Res 12:75–110MathSciNetzbMATHGoogle Scholar
  6. 6.
    Long P, Wu X (2004) Mistake bounds for maximum entropy discrimination. In: Proceedings of the 18th Annual Conference on Neural Information Processing Systems, pp 833–840Google Scholar
  7. 7.
    Zhu J, Xing E (2009) Maximum entropy discrimination Markov networks. J Mach Learn Res 10:2531–2569MathSciNetzbMATHGoogle Scholar
  8. 8.
    Zhu J, Xing E, Zhang B (2008) Laplace maximum margin Markov networks. In: Proceedings of the 25th International Conference on Machine Learning, pp 1256–1263Google Scholar
  9. 9.
    Zhu J, Xing E, Zhang B (2008) Partially observed maximum entropy discrimination Markov networks. In: Proceedings of the 22th Annual Conference on Neural Information Processing Systems, pp 1977–1984Google Scholar
  10. 10.
    Zhao J, Xie X, Xu X, Sun S (2017) Multi-view learning overview: Recent progress and new challenges. Inf Fusion 38:43– 54CrossRefGoogle Scholar
  11. 11.
    Sun S, Chao G (2013) Multi-view maximum entropy discrimination. In: Proceedings of the 23rd International Joint Conference on Artificial Intelligence, pp 1706–1712Google Scholar
  12. 12.
    Chao G, Sun S (2016) Alternative multi-view maximum entropy discrimination. IEEE Trans Neural Netw Learn Syst 27:1445– 1456MathSciNetCrossRefGoogle Scholar
  13. 13.
    Chao G, Sun S (2016) Consensus and complementarity based maximun entropy discrimination for multi-view classification. Inf Sci 367:296–310CrossRefGoogle Scholar
  14. 14.
    Mao L, Sun S (2016) Soft margin consistency based scalable multi-view maximum entropy discrimination. In: Proceedings of the 25th International Joint Conference on Artificial Intelligence, pp 1839–1845Google Scholar
  15. 15.
    Domingos (2000) Bayesian averaging of classifiers and the overfitting problem. In: Proceedings of the 17th International Conference on Machine Learning, pp 223–230Google Scholar
  16. 16.
    Chao G, Sun S (2016) Multi-kernel maximum entropy discrimination for multi-view learning. Intell Data Anal 20:481–493CrossRefGoogle Scholar
  17. 17.
    Tian Y, Qi Z, Ju X, Shi Y, Liu X (2014) Nonparallel support vector machines for pattern classification. IEEE Trans Cybern 44:1–12CrossRefGoogle Scholar
  18. 18.
    Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 74:905–910zbMATHGoogle Scholar
  19. 19.
    Chen S, Wu X, Xu J (2018) Locality preserving projection least squares twin support vector machine for?pattern classifcation. Pattern Analysis and Applications.  https://doi.org/10.1007/s10044-018-0728-x

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.The School of Information Science and EngineeringNingbo UniversityZhejiangChina

Personalised recommendations