Abstract
In this paper we consider multiclass learning tasks based on Support Vector Machines (SVMs). In this regard, currently used methods are One-Against-All or One-Against-One, but there is much need for improvements in the field of multiclass learning. We developed a novel combination algorithm called Comb-ECOC, which is based on posterior class probabilities. It assigns, according to the Bayesian rule, the respective instance to the class with the highest posterior probability. A problem with the usage of a multiclass method is the proper choice of parameters. Many users only take the default parameters of the respective learning algorithms (e.g. the regularization parameter C and the kernel parameter γ). We tested different parameter optimization methods on different learning algorithms and confirmed the better performance of One-Against-One versus One-Against-All, which can be explained by the maximum margin approach of SVMs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bose, R.C., Ray-Chaudhuri, D.K.: On A Class of Error Correcting Binary Group Codes. Information and Control 3 (1960)
Breiman, L.: Bagging Predictors. Machine Learning 24, 123–140 (1996)
Christianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press, Cambridge (2000)
Crammer, K., Singer, Y.: On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines. Journal of Machine Learning Reseach 2, 265–292 (2001)
Dietterich, T., Bakiri, G.: Solving Multiclass Learning Problems via Error-Correcting Output Codes. Journal of Artificial Intelligence Research 2, 263–286 (1995)
Dimitriadou, E., Hornik, K., Leisch, F., Meyer, D., Weingessel, A.: The e1071 package. Manual (2006)
Efron, B., Tibshirani, R.: An Introduction to the Bootstrap. Chapman & Hall/CRC (1993)
Friedrich, C.: Kombinationen evolutionär optimierter Klassifikatoren. PhD thesis, Universität Witten/Herdecke (2005)
García-Pedrajas, N., Ortiz-Boyer, D.: Improving Multiclass Pattern Recognition by the Combination of Two Strategies IEEE Transactions on Pattern Analysis and Machine Intelligence 28 (2006)
Hastie, T., Rosset, S., Tibshirani, R., Zhu, J.: The Entire Regularization Path for the Support Vector Machine. Technical Report, Statistics Department, Stanford University (2004)
Hsu, C.-W., Chang, C.-C., Lin, C.-J.: A Practical Guide to Support Vector Classification. Department of Computer Science and Information Engineering, National Taiwan University (2006)
Hsu, C.-W., Lin, C.-J.: A comparison of methods for multi-class Support Vector Machines. IEEE Transactions on Neural Networks 13, 415–425 (2002)
Huang, T.-J., Weng, R.C., Lin, C.-J.: Generalized Bradley-Terry Models and Multi-class Probability Estimates. Journal of Machine Learning Research 7, 85–115 (2006)
Hülsmann, M.: Vergleich verschiedener kernbasierter Methoden zur Realisierung eines effizienten Multiclass-Algorithmus des Maschinellen Lernens. Master’s thesis, Universität zu Köln (2006)
Joachims, T.: Making large-Scale SVM learning practical. In: Advances in Kernel Methods – Support Vector Learning, pp. 41–56. MIT Press, Cambridge (1999)
Mencía, E.L.: Paarweises Lernen von Multilabel-Klassifikatoren mit dem Perzeptron-Algorithmus. Master’s thesis, Technische Universität Darmstadt (2006)
Meyer, D.: Support Vector Machines, the Interface to libsvm in package e1071. Vignette (2006)
Newman, D.J., Hettich, S., Blake, C.L., Merz, C.J.: UCI Repository of machine learning databases (1998), http://www.ics.uci.edu/~mlearn/MLRespository.html
Platt, J.C.: Probabilistic Ouputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods. In: Proceedings of Advances in Large-Margin Classifiers, pp. 61–74. MIT Press, Cambridge (1999)
Roever, C., Raabe, N., Luebke, K., Ligges, U., Szepanek, G., Zentgraf, M.: The klaR package. Manual (2006)
Ihaka, R., Gentleman, R.R.: A Language for Data Analysis and Graphics. Journal of Computational and Graphical Statistics 5, 299–314 (1996)
Schölkopf, B., Smola, A.: Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond. MIT Press, Cambridge (2002)
Szedmak, S., Shawe-Taylor, J.: Multiclass Learning at One-Class Complexity. Information-Signals, Images, Systems (ISIS Group), Electronics and Computer Science. Technical Report (2005)
Tsochantaridis, I., Hofmann, T., Joachims, T., Altun, Y.: Support Vector Machine Learning for Interdependent and Structured Output Spaces. In: Proceedings of the 21th International Conference on Machine Learning. Banff, Canada (2004)
Vapnik, V.: The Nature of Statistical Learning Theory. Springer, Heidelberg (1995)
Wolpert, D.H.: No Free Lunch Theorems for Optimization. In: Proceedings of IEEE Transactions on Evolutionary Computation 1, pp. 67–82 (1997)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hülsmann, M., Friedrich, C.M. (2007). Comparison of a Novel Combined ECOC Strategy with Different Multiclass Algorithms Together with Parameter Optimization Methods. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2007. Lecture Notes in Computer Science(), vol 4571. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73499-4_3
Download citation
DOI: https://doi.org/10.1007/978-3-540-73499-4_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73498-7
Online ISBN: 978-3-540-73499-4
eBook Packages: Computer ScienceComputer Science (R0)