Abstract
Classification task involves inducing a predictive model using a set of labeled samples. The accuracy of the model usually increases as more labeled samples are available. When one has only few samples, the obtained model tends to offer poor results. Even when labeled samples are difficult to get, a lot of unlabeled samples are generally available on which unsupervised learning can be done. In this chapter, a way to combine supervised and unsupervised learning in order to use both labeled and unlabeled samples is explored. The efficiency of the method is evaluated on various UCI datasets and on the classification of a very high resolution remote sensing image when the number of labeled samples is very low.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bennett, K.P., Demiriz, A., Maclin, R.: Exploiting unlabeled data in ensemble methods. In: Proc. 8th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, Edmonton, AB, pp. 289–296. ACM, New York (2002)
Blum, A., Mitchell, T.: Combining labeled and unlabeled data with co-training. In: Proc 11th Annual Conf. Comp. Learn. Theory, Madison, WI, pp. 92–100. ACM, New York (1998)
Cai, W., Chen, S., Zhang, D.: A simultaneous learning framework for clustering and classification. Pattern Recogn. (in press)
Chawla, N.V., Karakoulas, G.J.: Learning from labeled and unlabeled data: an empirical study across techniques and domains. J. Artif. Intell. Res. 23, 331–366 (2005)
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. the Royal Stat. Society Series B 39(1), 1–38 (1977)
Deodhar, M., Ghosh, J.: A framework for simultaneous co-clustering and learning from complex data. In: Berkhin, P., Caruana, R., Wu, X. (eds.) Proc. 13th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, San Jose, CA, pp. 250–259. ACM, New York (2007)
Goldman, S., Zhou, Y.: Enhancing supervised learning with unlabeled data. In: Langley, P. (ed.) Proc. 17th Int. Conf. Mach. Learn., Stanford, CA, pp. 327–334. Morgan Kaufmann, San Francisco (2000)
Hoffbeck, J.P., Landgrebe, D.A.: Covariance matrix estimation and classification with limited training data. IEEE Trans. Pattern Analysis Mach. Intell. 18(7), 763–767 (1996)
Hughes, G.: On the mean accuracy of statistical pattern recognizers. IEEE Trans. Inf. Theory 14(1), 5–63 (1968)
Jia, X., Richards, J.A.: Cluster-space representation for hyperspectral data classification. IEEE Trans. Geoscience and Remote Sensing 40(3), 593–598 (2002)
Joachims, T.: Transductive inference for text classification using support vector machines. In: Bratko, I., Dzerovski, S. (eds.) Proc. 16th Int. Conf. Mach. Learn., Bled, Slovenia, pp. 200–209. Morgan Kaufmann, San Francisco (1999)
MacQueen, J.: Some methods for classification and analysis of multivariate observations. In: Le Cam, L.M., Neyman, J. (eds.) Proc. 5th Berkeley Symp. Math. Stat. Probability, Berkeley, CA, pp. 281–297 (1967)
Morgan, J.T., Ham, J., Crawford, M.M., Henneguelle, A., Ghosh, J.: Adaptive feature spaces for land cover classification with limited ground truth data. Int. J Pattern Recogn. Artif. Intell. 18(5), 777–799 (2004)
Newman, D.J., Hettich, S., Blake, C.L., Merz, C.J.: UCI Repository of machine learning databases, University of California, Irvine, Dept. of Information and Computer Sciences, http://www.ics.uci.edu/~mlearn/MLRepository.html
Nigam, K., McCallum, A.K., Thrun, S., Mitchell, T.: Text classification from labeled and unlabeled documents using EM. Mach. Learn. 39(2–3), 103–134 (2000)
Raskutti, B., Ferrá, H.L., Kowalczyk, A.: Combining clustering and co-training to enhance text classification using unlabelled data. In: Proc. 8th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, Edmonton, AB, pp. 620–625. ACM, New York (2002)
Seeger, M.: Learning with labeled and unlabeled data. University of Edinburgh, Scotland (2002), http://www.kyb.mpg.de/bs/people/seeger/papers/review.pdf
Shahshahani, B.M., Landgrebe, D.: The effect of unlabeled samples in reducing the small sample size problem and mitigating the Hughes phenomenon. IEEE Trans. Geoscience and Remote Sensing 32(5), 1087–1095 (1994)
Vincent, L., Soille, P.: Watersheds in digital spaces: an efficient algorithm based on immersion simulations. IEEE Trans. Pattern Analysis Mach. Intell. 13(6), 583–598 (1991)
Zhou, Z.-H., Zhan, D.-C., Yang, Q.: Semi-supervised learning with very few labeled training examples. In: Proc. 22nd AAAI Conf. Artif. Intell., Vancouver, BC, pp. 675–680. AAAI Press, Menlo Park (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Wemmert, C., Forestier, G., Derivaux, S. (2009). Improving Supervised Learning with Multiple Clusterings. In: Okun, O., Valentini, G. (eds) Applications of Supervised and Unsupervised Ensemble Methods. Studies in Computational Intelligence, vol 245. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03999-7_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-03999-7_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03998-0
Online ISBN: 978-3-642-03999-7
eBook Packages: EngineeringEngineering (R0)