Abstract
We present a novel method for improving disambiguation accuracy by building an optimal ensemble (OE) of systems where we predict the best available system for target word using a priori case factors (e.g. amount of training per sense). We report promising results of a series of best-system prediction tests (best prediction accuracy is 0.92) and show that complex/simple systems disambiguate tough/easy words better. The method provides the following benefits: (1) higher disambiguation accuracy for virtually any base systems (current best OE yields close to 2% accuracy gain over Senseval-3 state of the art) and (2) economical way of building more effective ensembles of all types (e.g. optimal, weighted voting and cross-validation based). The method is also highly scalable in that it utilizes readily available factors available for any ambiguous word in any language for estimating word difficulty and defines classifier complexity using known properties only.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aha, D.W.: Generalizing from case studies: A case study. In: Proceedings of the Ninth International Conference on Machine Learning, Morgan Kaufmann, San Francisco (1992)
Aha, D., Kibler, D.: Instance-based learning algorithms. Machine Learning 6, 37–66 (1991)
Bay, S.D., Pazzani, M.J.: Characterizing model errors and differences. In: 17th International Conference on Machine Learning (2000)
Edmonds, P., Kilgarriff, A.: Introduction to the Special Issue on evaluating word sense disambiguation programs. Journal of Natural Language Engineering 8(4) (2002)
Forman, G., Cohen, I.: Learning from Little: Comparison of Classifiers Given Little Training. In: Boulicaut, J.-F., Esposito, F., Giannotti, F., Pedreschi, D. (eds.) ECML 2004. LNCS (LNAI), vol. 3201, Springer, Heidelberg (2004)
Hoste, V., Hendrickx, I., Daelemans, W., van den Bosch, A.: Parameter optimization for machine-learning of word sense disambiguation. Journal of Natural Language Engineering 8(4), 311–327 (2002)
John, G., Langley, P.: Estimating Continuous Distributions in Bayesian Classifiers. In: Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann, San Mateo (1995)
Legrand, S., Pulido, J.G.R.: A Hybrid Approach to Word Sense Disambiguation: Neural Clustering with Class Labeling. In: Knowledge Discovery and Ontologies workshop at 15th European Conference on Machine Learning (ECML) (2004)
Luo, F., Khan, L., Bastani, F., Yen, I.-L., Zhou, J.A.: dynamically growing self-organizing tree (DGSOT) for hierarchical clustering gene expression profiles. Bioinformatics 20(16), 2605–2617 (2004)
Manning, C., Tolga Ilhan, H., Kamvar, S., Klein, D., Toutanova, K.: Combining Heterogeneous Classifiers for Word-Sense Disambiguation. In: Proceedings of SENSEVAL-2, Second International Workshop on Evaluating WSD Systems, pp. 87–90 (2001)
Mierswa, I., Wurst, M., Klinkenberg, R., Scholz, M., Euler, T.: YALE: Rapid Prototyping for Complex Data Mining Tasks. In: Proceedings of 12th ACM SIGKDD, ACM Press, New York (2006)
Mihalcea, R.: Word sense disambiguation with pattern learning and automatic feature selection. Journal of Natural Language Engineering 8(4), 343–359 (2002)
Mihalcea, R., Kilgarriff, A., Chklovski, T.: The SENSEVAL-3 English lexical sample task. In: Proceedings of SENSEVAL-3 Workshop at ACL (2004)
Pedersen, T.: Assessing System Agreement and Instance Difficulty in the Lexical Sample Tasks of SENSEVAL-2. In: Proceedings of the SIGLEX/SENSEVAL Workshop on Word Sense Disambiguation (2002)
Pedersen, T.: Machine learning with lexical features: The Duluth approach to Senseval. In: Proceedings of the Senseval-2 Workshop (2001)
Quinlan, R.: C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, San Mateo (1993)
Saarikoski, H., Legrand, S.: Building an Optimal WSD Ensemble Using Per-Word Selection of Best System. In: Martínez-Trinidad, J.F., Carrasco Ochoa, J.A., Kittler, J. (eds.) CIARP 2006. LNCS, vol. 4225, Springer, Heidelberg (2006)
Seo, H-C., Rim, H-C., Kim, S-H.: KUNLP system in Senseval-3. In: Proceedings of SENSEVAL-2 Workshop, pp. 222–225 (2001)
Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, Heidelberg (1995)
Witten, I., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)
Yarowsky, D., Cucerzan, S., Florian, R., Schafer, C., Wicentowski, R.: The Johns Hopkins SENSEVAL2 System Descriptions. In: Proceedings of SENSEVAL-2 workshop (2002)
Yarowsky, D., Florian, R.: Evaluating sense disambiguation across diverse parameter spaces. Journal of Natural Language Engineering 8(4), 293–311 (2002)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Saarikoski, H.M.T., Legrand, S., Gelbukh, A. (2007). Case-Sensitivity of Classifiers for WSD: Complex Systems Disambiguate Tough Words Better. In: Gelbukh, A. (eds) Computational Linguistics and Intelligent Text Processing. CICLing 2007. Lecture Notes in Computer Science, vol 4394. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-70939-8_23
Download citation
DOI: https://doi.org/10.1007/978-3-540-70939-8_23
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-70938-1
Online ISBN: 978-3-540-70939-8
eBook Packages: Computer ScienceComputer Science (R0)