Abstract
In the framework of decomposition methods for multiclass classification problems, error correcting output codes (ECOC) can be fruitfully used as codewords for coding classes in order to enhance the generalization capability of learning machines. The effectiveness of error correcting output codes depends mainly on the independence of codeword bits and on the accuracy by which each dichotomy is learned. Separated and non-linear dichotomizers can improve the independence among computed codeword bits, thus fully exploiting the error recovering capabilities of ECOC. In the experimentation presented in this paper we compare ECOC decomposition methods implemented through monolithic multi-layer perceptrons and sets of linear and non-linear independent dichotomizers. The most effectiveness of ECOC decomposition scheme is obtained by Parallel Non-linear Dichotomizers (PND), a learning machine based on decomposition of polychotomies into dichotomies, using non linear independent dichotomizers.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
E. Alpaydin and E. Mayoraz. Combining linear dichotomizers to construct nonlinear polychotomizers. Technical report, IDIAP-RR 98-05-Dalle Molle Institute for Perceptual Artificial Intelligence, Martigny (Switzerland) 1998. ftp://ftp.idiap.ch/pub/reports/1998/rr98-05.ps.gz.
R. Anand, G. Mehrotra, C.K. Mohan and S. Ranka. Efficient classification for multiclass problems using modular neural networks. IEEE Transactions on Neural Networks, 6:117–124, 1995.
R.C. Bose and D.K. Ray-Chauduri. On a class of error correcting binary group codes. Information and Control, (3):68–79, 1960.
V. N. Cherkassky and F. Mulier. Learning from data: Concepts, Theory and Methods Wiley & Sons, New York, 1998.
T. Dietterich and G. Bakiri. Error-correcting output codes: A general method for improving multiclass inductive learning programs. In Proceedings of AAAI-91, pages 572–577. AAAI Press / MIT Press, 1991.
T. Dietterich and G. Bakiri. Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research, (2):263–286, 1995.
T.G. Dietterich. Approximate statistical test for comparing supervised classification learning algorithms. Neural Computation, 7(10):1895–1924, 1998.
B.S. Everitt. The analysis of contingency tables Chapman and Hall, London, 1977.
S. Hashem. Optimal linear combinations of neural networks. Neural Computation, 10:599–614, 1997.
J.R. Quinlan. C4.5 Programs for Machine Learning Morgan Kauffman, 1993.
E. Kong and T. Dietterich. Error-correcting output coding correct bias and variance. In The XII International Conference on Machine Learning, pages 313–321, San Francisco, CA, 1995. Morgan Kauffman.
S. Lin and D.J.Jr. Costello. Error Control Coding: Fundamentals and Applications Prentice-Hall, Englewood Cliffs, 1983.
F. Masulli and G. Valentini. Comparing decomposition methods for classification. In KES’2000, Fourth International Conference on Knowledge-Based Intelligent Engineering Systems & Allied Technologies, Brighton, England. (in press).
F. Masulli and G. Valentini. Parallel Non linear Dichotomizers. In IJCNN2000, The IEEE-INNS-ENNS International Joint Conference on Neural Networks, Como, Italy. (in press).
E. Mayoraz and M. Moreira. On the decomposition of polychotomies into dichotomies. In The XIV International Conference on Machine Learning, pages 219–226, Nashville, TN, July 1997.
C. J. Merz and P.M. Murphy. UCI repository of machine learning databases, 1998. http://www.ics.uci.edu/mlearn/MLRepository.html.
M.P. Perrone and L.N. Cooper. When networks disagree: ensemble methods for hybrid neural networks. In Mammone R.J., editor, Artificial Neural Networks for Speech and Vision, pages 126–142. Chapman & Hall, London, 1993.
W.W. Peterson and E.J. Jr. Weldon. Error correcting codes MIT Press, Cambridge, MA, 1972.
D.E. Rumelhart, G.E. Hinton and R.J. Williams. Learning internal reperesentations by error propagation. In Rumelhart D.E., McClelland J.L., editor, Parallel Distributed Processing: Explorations in the Micro structure of Conition, volume 1, chapter 8. MIT Press, Cambridge, MA, 1986.
T. J. Sejnowski and C. R. Rosenberg. Parallel networks that learn to pronounce english text. Journal of Artificial Intelligence Research, (1):145–168, 1987.
G. Valentini. Metodi scompositivi per la classificazione. Master’s thesis, Dipartimento di Informatica e Scienze Informazione-Universitá di Genova, Genova, Italy, 1999.
G. Valentini and F. Masulli. NEURObjects, a set of library classes for neural networks development. In Proceedings of the third International ICSC Symposia on Intelligent Industrial Automation (IIA’99) and Soft Computing (SOCO’99), pages 184–190, Millet, Canada, 1999. ICSC Academic Press.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Masulli, F., Valentini, G. (2000). Effectiveness of Error Correcting Output Codes in Multiclass Learning Problems. In: Multiple Classifier Systems. MCS 2000. Lecture Notes in Computer Science, vol 1857. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45014-9_10
Download citation
DOI: https://doi.org/10.1007/3-540-45014-9_10
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-67704-8
Online ISBN: 978-3-540-45014-6
eBook Packages: Springer Book Archive