Advertisement

A Unified Framework of Binary Classifiers Ensemble for Multi-class Classification

  • Takashi Takenouchi
  • Shin Ishii
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7664)

Abstract

We present a novel methods for multi-class classification by ensemble of binary classifiers for multi-class classification. The proposed method is characterized by a minimization problem of weighted divergences, and includes a lot of conventional methods as special cases. We discuss relationship between the proposed method and conventional methods and statistical properties of the proposed method. A small experiment shows that the proposed method can effectively incorporate information of multiple binary classifiers into multi-class classifier.

Keywords

Ensemble learning Mixture of divergences 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Allwein, E.L., Schapire, R.E., Singer, Y.: Reducing multiclass to binary: A unifying approach for margin classifiers. Journal of Machine Learning Research 1, 113–141 (2001)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Amari, S., Nagaoka, H.: Methods of Information Geometry. Translations of Mathematical Monographs, vol. 191. Oxford University Press (2000)Google Scholar
  3. 3.
    Crammer, K., Singer, Y.: On the learnability and design of output codes for multiclass problems. Machine Learning 47(2), 201–233 (2002)zbMATHCrossRefGoogle Scholar
  4. 4.
    Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research 2, 263–286 (1995)zbMATHGoogle Scholar
  5. 5.
    Genest, C., Zidek, J.V.: Combining probability distributions: A critique and an annotated bibliography. Statistical Science 1(1), 114–135 (1986)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Hastie, T., Tibshirani, R.: Classification by pairwise coupling. Annals of Statistics 26, 451–471 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Murata, N., Takenouchi, T., Kanamori, T., Eguchi, S.: Information geometry of U-boost and bregman divergence. Neural Computation 16(7), 1437–1481 (2004)zbMATHCrossRefGoogle Scholar
  8. 8.
    Takenouchi, T., Ishii, S.: A multi-class classification method based on decoding of binary classifiers. Neural Computation 21(7), 2049–2081 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Takenouchi, T., Ishii, S.: Ternary bradley-terry model-based decoding for multi-class classification and its extensions. Machine Learning, 1–24 (2011)Google Scholar
  10. 10.
    Weston, J., Watkins, C.: Support vector machines for multi-class pattern recognition. In: Proceedings of the Seventh European Symposium on Artificial Neural Networks, vol. 4(6) (1999)Google Scholar
  11. 11.
    Zadrozny, B.: Reducing multiclass to binary by coupling probability estimates. In: Dietterich, T.G., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems, vol. 14. MIT Press, Cambridge (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Takashi Takenouchi
    • 1
  • Shin Ishii
    • 2
  1. 1.Future University HakodateHakodateJapan
  2. 2.Graduate School of InformaticsKyoto UniversityUjiJapan

Personalised recommendations