Probability Estimation in Error Correcting Output Coding Framework Using Game Theory

  • Mikhail Petrovskiy
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3809)


This paper is devoted to the problem of obtaining class probability estimates for multi-class classification problem in Error-correcting output coding (ECOC) framework. We consider the problem of class prediction via ECOC ensemble of binary classifiers as a decision-making problem and propose to solve it using game theory approach. We show that class prediction problem in ECOC framework can be formulated as a matrix game of special form. Investigation of the optimal solution in pure and mixed strategies is resulted in development of novel method for obtaining class probability estimates. Experimental performance evaluation on well-known benchmark datasets has demonstrated that proposed game theoretic method outperforms traditional methods for class probabilities estimation in ECOC framework.


Mixed Strategy Pure Strategy Class Probability Binary Classifier Probabilistic Output 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Dietterich, T.G., Bakiri, G.: Solving multi-class learning problems via error-correcting output codes. Journal of Artificial Intelligence Research 2, 263–286 (1995)zbMATHGoogle Scholar
  2. 2.
    Ghaderi, R., Windeatt, T.: Least squares and estimation measures via error correcting output code. In: 2nd Int. Workshop Multiple Classifier Systems. Lecture notes in computer science, pp. 148–157. Springer, Heidelberg (2001)Google Scholar
  3. 3.
    Allwein, E.L., Schapire, R.E., Singer, Y.: Reducing multiclass to binary: A unifying approach for margin classifiers. Journal of Machine Learning Research (1), 113–141 (2000)Google Scholar
  4. 4.
    Platt, J.: Probabilistic Outputs for Support Vector Machines and Comparison to Regularized Likelihood Methods. In: Adv. in Large Margin Classifiers, pp. 61–74. MIT Press, Cambridge (1999)Google Scholar
  5. 5.
    Fudenberg, D., Tirole, J.: Game Theory. MIT Press, Cambridge (1991)Google Scholar
  6. 6.
    Dekel, O., Singer, Y.: Multiclass Learning by Probabilistic Embeddings. In: NIPS 2002, pp. 945–952 (2002)Google Scholar
  7. 7.
    Zadrozny, B., Elkan, C.: Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In: ICML proceedings, pp. 609–616 (2000)Google Scholar
  8. 8.
    Lin, K.-M., Lin, C.-J.: A study on reduced support vector machines, Neural Networks. IEEE Transactions on 14(6), 1449–1459 (2003)Google Scholar
  9. 9.
    Schapire, R.E.: Using output codes to boost multiclass learning problems. In: ICML proceedings, pp. 313–321 (1997)Google Scholar
  10. 10.
    David, H.A.: The method of paired comparisons, 2nd edn. Oxford University Press, New York (1988)zbMATHGoogle Scholar
  11. 11.
    Huang, T.-K., Weng, R., Lin, C.-J.: A Generalized Bradley-Terry Model: From Group Competition to Individual Skill. In: NIPS 2004 (2004)Google Scholar
  12. 12.
    Kong, E.B., Diettrich, T.G.: Probability estimation via error-correcting output coding. In: Int. Conf. of Artificial Intelligence and soft computing, Banff,Canada, p. 6 (1997)Google Scholar
  13. 13.
    Windeatt, T.M., Ghaderi, R.: Coding and decoding strategies for multi-class learning problems. Information Fusion 4(1), 11–21 (2003)CrossRefGoogle Scholar
  14. 14.
    Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines 2001, Software available at,

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Mikhail Petrovskiy
    • 1
  1. 1.Computer Science Department of LomonosovMoscow State UniversityMoscowRussia

Personalised recommendations