A Unified Framework of Binary Classifiers Ensemble for Multi-class Classification

  • Takashi Takenouchi
  • Shin Ishii
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7664)


We present a novel methods for multi-class classification by ensemble of binary classifiers for multi-class classification. The proposed method is characterized by a minimization problem of weighted divergences, and includes a lot of conventional methods as special cases. We discuss relationship between the proposed method and conventional methods and statistical properties of the proposed method. A small experiment shows that the proposed method can effectively incorporate information of multiple binary classifiers into multi-class classifier.


Ensemble learning Mixture of divergences 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Allwein, E.L., Schapire, R.E., Singer, Y.: Reducing multiclass to binary: A unifying approach for margin classifiers. Journal of Machine Learning Research 1, 113–141 (2001)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Amari, S., Nagaoka, H.: Methods of Information Geometry. Translations of Mathematical Monographs, vol. 191. Oxford University Press (2000)Google Scholar
  3. 3.
    Crammer, K., Singer, Y.: On the learnability and design of output codes for multiclass problems. Machine Learning 47(2), 201–233 (2002)zbMATHCrossRefGoogle Scholar
  4. 4.
    Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research 2, 263–286 (1995)zbMATHGoogle Scholar
  5. 5.
    Genest, C., Zidek, J.V.: Combining probability distributions: A critique and an annotated bibliography. Statistical Science 1(1), 114–135 (1986)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Hastie, T., Tibshirani, R.: Classification by pairwise coupling. Annals of Statistics 26, 451–471 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Murata, N., Takenouchi, T., Kanamori, T., Eguchi, S.: Information geometry of U-boost and bregman divergence. Neural Computation 16(7), 1437–1481 (2004)zbMATHCrossRefGoogle Scholar
  8. 8.
    Takenouchi, T., Ishii, S.: A multi-class classification method based on decoding of binary classifiers. Neural Computation 21(7), 2049–2081 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Takenouchi, T., Ishii, S.: Ternary bradley-terry model-based decoding for multi-class classification and its extensions. Machine Learning, 1–24 (2011)Google Scholar
  10. 10.
    Weston, J., Watkins, C.: Support vector machines for multi-class pattern recognition. In: Proceedings of the Seventh European Symposium on Artificial Neural Networks, vol. 4(6) (1999)Google Scholar
  11. 11.
    Zadrozny, B.: Reducing multiclass to binary by coupling probability estimates. In: Dietterich, T.G., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems, vol. 14. MIT Press, Cambridge (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Takashi Takenouchi
    • 1
  • Shin Ishii
    • 2
  1. 1.Future University HakodateHakodateJapan
  2. 2.Graduate School of InformaticsKyoto UniversityUjiJapan

Personalised recommendations