Abstract
The benefits obtained from the decomposition of a classification task involving several classes, into a set of smaller classification problems involving two classes only, usually called dichotomies, have been exposed in various occasions. Among the multiple ways of applying the referred decomposition, Pairwise Coupling is one of the best known. Its principle is to separate a pair of classes in each binary subproblem, ignoring the remaining ones, resulting in a decomposition scheme containing as much subproblems as the number of possible pairs of classes in the original task. Pairwise Coupling decomposition has so far been used in different applications. In this paper, various ways of recombining the outputs of all the classifiers solving the existing subproblems are explored, and an important handicap of its intrinsic nature is exposed, which consists in the use, for the classification, of impertinent information. A solution for this problem is suggested and it is shown how it can significantly improve the classification accuracy. In addition, a powerful decomposition scheme derived from the proposed correcting procedure is presented.
The authors are thankful to Prof. Alain Hertz for the valuable discussions that greatly contributed to the present work. The support of the Swiss National Science Foundation under grant FN 21-46974.96 is also gratefully acknowledged.
Chapter PDF
References
E. Boros, P. L. Hammer, Toshihide Ibaraki, A. Kogan, E. Mayoraz, and I. Muchnik. An implementation of logical analysis of data. RRR 22-96, RUTCOR-Rutgers University's Center For Operations Research, http://rut cor. rutgers.edu:80/≈rrr/, Submitted, July 1996.
L. Breiman, J. Olshen, and C. Stone.Classification and Regression Trees. Wadsworth International Group, 1984.
Pierre J. Castellano, Stefan Slomka, and Sridha Sridharan. Telephone based speaker recognition using multiplt binary classifier and Gaussian mixture models. In ICASSP, volume 2, pages 1075–1078. IEEE Computer Society Press, 1997.
Yves Crama, Peter L. Hammer, and Toshihide Ibaraki. Cause-effect relationships and partially defined boolean functions. Annals of Operations Research, 16:299–326, 1988.
Thomas G. Dietterich and Ghulum Bakiri. Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research, 2:263–286, 1995.
Thomas G. Dietterich. Statistical tests for comparing supervised classification learning algorithms. OR 97331, Department of Computer Science, Oregon State University,, 1996.
T. G. Dietterich and G. Bakiri. Error-correcting output codes: A general method for improving multiclass inductive learning programs. In Proceedings of AAAI-91, pages 572–577. AAAI Press / MIT Press, 1991.
R. O. Duda and P. E. Hart. Pattern Classification and Scene Analysis. John Wiley & Sons, New York, 1973.
Trevor Hastie and Robert Tibshirani.Classification by pairwise coupling. Technical report, Stanford University and University of Toronto, 1996. ftp://utstat.Toronto.edu/pub/tibs/coupling.ps,to appear in the Proceedings of NIPS*97.
E. B. Kong and T. G. Dietterich. Error-correcting output coding corrects bias and variance. In The XII International Conference on Machine Learning, pages 313–321, San Francisco, CA, 1995. Morgan Kaufmann.
Eddy Mayoraz and Miguel Moreira. On the decomposition of polychotomies into dichotomies. In Douglas H. Fisher, editor, The Fourteenth International Conference on Machine Learning, pages 219–226, 1997.
C.J. Merz and P.M. Murphy. UCI repository of machine learning databases. Irvine, CA: University of California, Department of Information and Computer Sciences, http://AAA.ics.uci.edu/≈mlearn/MLRepository.html,1996.
David Price, Stefan Knerr, Leon Personnaz, and Gerard Dreyfus. Pairwise neural network classifiers with probabilistic outputs. In G. Tesauro, D. Touretzky, and T. Leen, editors, Advances in Neural Information Processing Systems 7 (NIPS*94), volume 7, pages 1109–1116. The MIT Press, 1995.
J. R. Quinlan. Induction of decision trees. Machine Learning, 1:81–106, 1986.
J. R. Quinlan. C4.5 Programs for Machine Learning. Morgan Kaufmann, 1993.
F. Rosenblatt. The perceptron: a probabilistic model for information storage and organization in the brain. Psychological Review, 63:386–408, 1958.
Laszlo Rudasi and Stephen A. Zahorian. Text-independent talker indentification with neural networks. In ICASSP, volume 1, pages 389–392, 1991.
Robert E. Shapire. Using output codes to boost multiclass learning problems. In Douglas H. Fisher, editor, The Fourteenth International Conference on Machine Learning, pages 313–321, 1997.
P. Werbos. Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. PhD thesis, Harvard University, 1974.
Stephen A. Zahorian, Peter Silsbee, and Xihong Wang. Phone classification with segmental features and a binary-pair partitioned neural network classifier. In ICASSP, volume 2, pages 1011–1014. IEEE Computer Society Press, 1997.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1998 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Moreira, M., Mayoraz, E. (1998). Improved pairwise coupling classification with correcting classifiers. In: Nédellec, C., Rouveirol, C. (eds) Machine Learning: ECML-98. ECML 1998. Lecture Notes in Computer Science, vol 1398. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0026686
Download citation
DOI: https://doi.org/10.1007/BFb0026686
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-64417-0
Online ISBN: 978-3-540-69781-7
eBook Packages: Springer Book Archive