Abstract
Many classification problems of high technological value are multiclass. In the last years, several improved solutions based on the combination of simple classifiers were introduced. An interesting kind of methods creates a hierarchy of sub-problems by clustering prototypes of each one of the classes, but the solution produced by the clustering stage is heavily influenced by the label’s information. In this work we introduce a new strategy to solve multiclass problems that makes more use of spatial information than other methods. Based on our previous work on imbalanced problems, we construct a hierarchy of subproblems, but opposite to previous developments, based only on spatial information and not using class labels at any time. We consider different clustering methods (either agglomerative or divisive) for this task. We use an SVM for each sub-problem (if needed, because in several cases the clustering method directly gives a subset with samples of a single class). Using publicly available datasets we compare the new method with several previous approaches, finding promising results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ahumada, H., Grinblat, G., Uzal, L., Granitto, P., Ceccatto, A.: Repmac: A new hybrid approach to highly imbalanced classification problems. In: 8th Int. Conference on Hybrid Intelligent Systems. pp. 386–391 (2008)
Asuncion, A., Newman, D.: UCI machine learning repository (2007), http://www.ics.uci.edu/~mlearn/MLRepository.html
Bahl, L., Jelinek, F., Mercer, R.: A maximum likelihood approach to continuous speech recognition. IEEE T. Pattern Anal. (2), 179–190 (2009)
Benabdeslem, K., Bennani, Y.: Dendogram-based SVM for Multi-Class Classification. J. Comput. Inform. Tech. 14(4), 283–289 (2006)
Crammer, K., Singer, Y.: On the algorithmic implementation of multiclass kernel-based vector machines. J Mach. Learn. Res. 2, 265–292 (2002)
Cristianini, N., Shawe–Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press, Cambridge (2000)
Fei, B., Liu, J.: Binary tree of SVM: a new fast multiclass training and classification algorithm. IEEE T. Neural Networ. 17(3), 696–704 (2006)
Freund, Y., Schapire, R.: A desicion-theoretic generalization of on-line learning and an application to boosting. In: Proc. of COLT, pp. 23–37 (1995)
Granitto, P., Verdes, P., Ceccatto, H.: Large-scale investigation of weed seed identification by machine vision. Comput. Electron. Agr. 47(1), 15–24 (2005)
Hastie, T., Tibshirani, R.: Classification by pairwise coupling. Ann. Stat. 26(2), 451–471 (1998)
Hsu, C.W., Lin, C.J.: A comparison of methods for multiclass support vector machines. IEEE T. Neural Networ. 13(2), 415–425 (2002)
King, B.: Step-wise clustering procedures. J. Am. Stat. Assoc. 69, 86–101 (1967)
Liu, S., Yi, H., Chia, L.T., Rajan, D.: Adaptive hierarchical multi-class SVM classifier for texture-based image classification. In: IEEE Int. Conf. on Multimedia and Expo, pp. 1–4 (2005)
Lorena, A.C., Carvalho, A.C., Gama, J.M.: A review on the combination of binary classifiers in multiclass problems. Artif. Intell. Rev. 30, 19–37 (2008)
McQueen, J.: Some methods for classification and analysis of multivariate observations. In: Proc. of the Fifth Berkeley Symposium on Mathematics, Statistics and Probability, pp. 281–297 (1967)
Platt, J., Cristianini, N., Shawe-Taylor, J.: Large margin dags for multiclass classification. In: Adv. in Neural Information Processing Systems, vol. 12, pp. 547–553 (2000)
Ramaswamy, S., Tamayo, P., Rifkin, R., et al.: Multiclass cancer diagnosis using tumor gene expression signatures. P. Natl. Acad. Sci. USA 98(26), 15149 (2001)
Rifkin, R., Klautau, A.: In defense of one-vs-all classification. J. Mach. Learn. Res. 5, 101–141 (2004)
Sneath, P.H.A., Sokal, R.R.: Numerical Taxonomy. W.H. Freeman and Company, San Francisco (1973)
Songsiri, P., Kijsirikul, B., Phetkaew, T.: Information-based dichotomization: A method for multiclass Support Vector Machines. In: IEEE Int. Joint Conference on Neural Networks, pp. 3284–3291 (2008)
Weston, J., Watkins, C.: Support vector machines for multi-class pattern recognition. In: 7th European Symposium On Art. Neural Networks, pp. 4–6 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ahumada, H.C., Grinblat, G.L., Granitto, P.M. (2011). Unsupervized Data-Driven Partitioning of Multiclass Problems. In: Honkela, T., Duch, W., Girolami, M., Kaski, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2011. ICANN 2011. Lecture Notes in Computer Science, vol 6791. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21735-7_15
Download citation
DOI: https://doi.org/10.1007/978-3-642-21735-7_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21734-0
Online ISBN: 978-3-642-21735-7
eBook Packages: Computer ScienceComputer Science (R0)