Skip to main content
SpringerLink
Log in

Evaluation of WSD Systems

  • Chapter
Word Sense Disambiguation

Part of the book series: Text, Speech and Language Technology ((TLTB,volume 33))

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Atkins, Sue. 1993. Tools for computer-aided corpus lexicography: The Hector project. Acta Linguistica Hungarica, 41: 5-72.

    Google Scholar 

  • Baker, Collin F., Charles J. Fillmore & Beau Cronin. 2003. The structure of the FrameNet database. International Journal of Lexicography, 16(3): 281-296.

    Article  Google Scholar 

  • Bruce, Rebecca & Janyce Wiebe. 1994. Word-sense disambiguation using decomposable models. Proceedings of the 32nd Annual Meeting of the Association for Computational Linguistics, Las Cruces, New Mexico, 139-145.

    Chapter  Google Scholar 

  • Bruce, Rebecca & Janyce Wiebe. 1999. Recognizing subjectivity: A case study in manual tagging. Natural Language Engineering, 5(2): 187-205.

    Article  Google Scholar 

  • Calzolari, Nicoletta & Ornella Corazzari. 2000. Senseval/Romanseval: The framework for Italian. Computers and the Humanities, 34(1-2): 61-78.

    Article  Google Scholar 

  • Chklovski, Timothy & Rada Mihalcea. 2002. Building a sense tagged corpus with Open Mind Word Expert. Proceedings of the workshop on word sense disambiguation: Recent successes and future directions, Philadelphia, PA, USA, 116-122.

    Chapter  Google Scholar 

  • Cohen, J. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20: 37-46.

    Article  Google Scholar 

  • Dang, Hoa Trang, Ching-yi Chia, Fu-Dong Chiou & Martha Palmer. 2002. Simple features for Chinese word sense disambiguation. Proceedings of the 19th International Conference on Computational Linguistics (COLING-2002), Taipei, Taiwan.

    Google Scholar 

  • Edmonds, Philip & Scott Cotton. 2001. Senseval-2: Overview. Proceedings of Senseval-2: Second International Workshop on Evaluating Word Sense Disambiguation Systems, Toulouse, France, 1-5.

    Google Scholar 

  • Edmonds, Philip & Adam Kilgarriff. 2002. Introduction to the special issue on evaluating word sense disambiguation systems. Natural Language Engineering, 8(4): 279-291.

    Article  Google Scholar 

  • Fellbaum, Christiane, ed. 1998. WordNet: An Electronic Lexical Database. Cambridge, MA: MIT Press.

    Google Scholar 

  • Fellbaum, Christiane, Martha Palmer, Hoa Trang Dang, Lauren Delfs & Susanne Wolf. 2001. Manual and automatic semantic annotation with WordNet. Proceedings of the Workshop on WordNet and Other Lexical Resources, Pittsburgh, PA.

    Google Scholar 

  • Fellbaum, Christiane, Lauren Delfs, Susanne Wolf & Martha Palmer. 2005. Word meaning in dictionaries, corpora, and the speaker’s mind.  Meaningful Texts: The Extraction of Semantic Information from Monolingual and Multilingual Corpora ed. by Geoff Barnbrook, Pernilla Danielsson & Michaela Mahlberg. London: Continuum.

    Google Scholar 

  • Francis, W. Nelson & Henry Kuþera. 1982. Frequency Analysis of English Usage: Lexicon and Grammar. Boston, MA: Houghton Mifflin.

    Google Scholar 

  • Gale, William, Kenneth Ward Church & David Yarowsky. 1992. Estimating upper and lower bounds on the performance of word-sense disambiguation programs. Proceedings of the 30th Annual Meeting of the Association for Computational Linguistics, 249-256.

    Google Scholar 

  • Hanks, Patrick. 2000. Do word meanings exist? Computers and the Humanities, 34(1-2): 205-215.

    Article  Google Scholar 

  • Ide, Nancy & Jean Véronis. 1998. Introduction to the special issue on word sense disambiguation. Computational Linguistics, 24(1): 1-40.

    Google Scholar 

  • Kilgarriff, Adam. 1997. “I don’t believe in word senses”. Computers and the Humanities, 31(2): 91-113.

    Article  Google Scholar 

  • Kilgarriff, Adam. 1998. Senseval: An exercise in evaluating word sense disambiguation programs. Proceedings of the European Conference on Lexicography (EURALEX), 176-174, Liège, Belgium. Also in Proceedings of the 1st Conference on Language Resources and Evaluation (LREC), Granada, Spain, 581-588.

    Google Scholar 

  • Kilgarriff, Adam & Martha Palmer. 2000. Introduction to the special issue on Senseval. Computers and the Humanities, 34(1-2): 1-13.

    Article  Google Scholar 

  • Kilgarriff, Adam & Joseph Rosenzweig. 2000. Framework and results for English SENSEVAL. Computers and the Humanities, 34(1-2): 15-48.

    Article  Google Scholar 

  • Kilgarriff, Adam. 2001. English lexical sample task description. Proceedings of Senseval-2: Second International Workshop on Evaluating Word Sense Disambiguation Systems, Toulouse, France, 17-20.

    Google Scholar 

  • Krishnamurthy, Ramesh & Diane Nicholls. 2000. Peeling an onion: The lexicographer’s experience of manual sense-tagging. Computers and the Humanities, 34 (1-2): 85-97.

    Article  Google Scholar 

  • Leacock, Claudia, Geoffrey Towell & Ellen Voorhees. 1993. Corpus-based statistical sense resolution. Proceedings of the ARPA Human Language Technology Workshop, 260-265.

    Google Scholar 

  • Lesk, Michael. 1986. Automatic sense disambiguation: How to tell a pine cone from an ice cream cone. Proceedings of SIGDOC-86: 5th International Conference on Systems Documentation, Toronto, Canada, 24-26.

    Chapter  Google Scholar 

  • Levin, Beth. 1993. English Verb Classes and Alternations: A Preliminary Investigation. Chicago: University of Chicago Press.

    Google Scholar 

  • Lin, Dekang. 1998. Automatic retrieval and clustering of similar words. Proceedings of the 17th International Conference on Computational Linguistics (COLING-ACL-98). Montreal, Canada, 768-774.

    Chapter  Google Scholar 

  • Manning, Christopher D. & Hinrich Schütze. 1999. Foundations of Statistical Natural Language Processing. Cambridge, MA: MIT Press.

    Google Scholar 

  • Marcus, Mitchell P., Beatrice Santorini & Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2): 313-330.

    Google Scholar 

  • Melamed, I. Dan & Philip Resnik. 2000. Tagger evaluation given hierarchical tag sets. Computers and the Humanities, 34(1-2): 79-84.

    Article  Google Scholar 

  • Mihalcea, Rada & Dan Moldovan. 2001. Automatic generation of a coarse grained wordnet. Proceedings of NAACL-2001 Workshop on WordNet and Other Lexical Resources, Pittsburgh, PA, 35-41.

    Google Scholar 

  • Mihalcea, Rada & Philip Edmonds, eds. 2004. Proceedings of Senseval-3: Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text, Barcelona, Spain. (http://www.senseval.org/ )

  • Mihalcea, Rada, Timothy Chlovski & Adam Kilgarriff. 2004. The Senseval-3 English lexical sample task. Proceedings of Senseval-3: Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text, Barcelona, Spain, 25-28.

    Google Scholar 

  • Miller, George A., ed. 1990. Special Issue, WordNet: An on-line lexical database. International Journal of Lexicography, 3(4).

    Google Scholar 

  • Miller, George A. & Christiane Fellbaum. 1991. Semantic networks of English. Cognition, 41(1-3): 197-229.

    Article  Google Scholar 

  • Miller, George A., Claudia Leacock, Randee Tengi, and R. T. Bunker. 1993. A semantic concordance. Proceedings of the ARPA Workshop on Human Language Technology, San Francisco, 303-308.

    Google Scholar 

  • Mooney, Raymond J. 1996. Comparative experiments on disambiguating word senses: An illustration of the role of bias in machine learning. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 82-91.

    Google Scholar 

  • Ng, Hwee Tou & Hian Beng Lee. 1996. Integrating multiple knowledge sources to disambiguate word sense: An exemplar-based approach. Proceedings of the 34th Annual Meeting of the Association for Computational Linguistics, Santa Cruz, California, 40-47.

    Chapter  Google Scholar 

  • Ng, Hwee Tou. 1997a. Getting serious about word sense disambiguation. Proceedings of the ACL SIGLEX Workshop on Tagging Text with Lexical Semantics: Why, What, and How? Washington D.C., USA, 1-7.

    Google Scholar 

  • Ng, Hwee Tou. 1997b. Exemplar-based word sense disambiguation Some recent improvements. Proceedings of the Second Conference on Empirical Methods in Natural Language Processing (EMNLP), Providence, Rhode Island, USA, 208-213.

    Google Scholar 

  • Ng, Hwee Tou, Chung Yong Lim & Shou King Foo. 1999. A case study on interannotator agreement for word sense disambiguation. Proceedings of the ACL SIGLEX Workshop on Standardizing Lexical Resources, College Park, Maryland, USA, 9-13.

    Google Scholar 

  • Ng, Hwee Tou, Bin Wang & Yee Seng Chan. 2003. Exploiting parallel texts for word sense disambiguation: An empirical study. Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, Sapporo, Japan, 455-462.

    Google Scholar 

  • Palmer, Martha. 1990. Customizing verb definitions for specific semantic domains. Machine Translation, 5: 5-30.

    Article  Google Scholar 

  • Palmer, Martha & Marc Light. 1999. Introduction to the special issue on semantic tagging. Natural Language Engineering, 5(2): i-iv.

    Google Scholar 

  • Palmer, Martha, Chunghye Han, Fei Xia, Dania Egedi & Joseph Rosenzweig. 2000a. Constraining lexical selection across languages using TAGs. Tree Adjoining Grammars: Formal, Computational and Linguistic Aspects, ed. by Anne Abeille & Owen Rambow., Palo Alto, CA: CSLI.

    Google Scholar 

  • Palmer, Martha, Hoa Trang Dang, Joseph Rosenzweig. 2000b. Sense tagging the Penn Treebank. Proceedings of the Second International Conference Language Resources and Evaluation Conference (LREC 2000), Athens, Greece.

    Google Scholar 

  • Palmer, Martha, Christiane Fellbaum, Scott Cotton, Lauren Delfs & Hoa Trang Dang. 2001. English tasks: All-words and verb lexical sample. Proceedings of Senseval-2: Second International Workshop on Evaluating Word Sense Disambiguation Systems, Toulouse, France, 21-24.

    Google Scholar 

  • Palmer, Martha, Christiane Fellbaum & Hoa Trang Dang. 2006. Making finegrained and coarse-grained sense distinctions, both manually and automatically. Natural Language Engineering, 12(3).

    Google Scholar 

  • Pustejovsky, James. 1991. The generative lexicon. Computational Linguistics, 17(4): 409-411.

    Google Scholar 

  • Resnik, Philip & David Yarowsky. 1999. Distinguishing systems and distinguishing senses: New evaluation methods for word sense disambiguation. Natural Language Engineering, 5(2): 113-133.

    Article  Google Scholar 

  • Sanderson, Mark. 1994. Word sense disambiguation and information retrieval. Proceedings of the 17th Annual International ACM SIGIR Conference on Re-search and Development in Information Retrieval, Dublin, Ireland, 142-151.

    Google Scholar 

  • Schütze, Hinrich. 1998. Automatic word sense discrimination. Computational Linguistics, 24(1): 97-123.

    Google Scholar 

  • Segond, Frédérique. 2000. Framework and results for French. Computers and the Humanities, 34(1-2): 49-60.

    Article  Google Scholar 

  • Snyder, Ben & Martha Palmer. 2004. The English all-words task. Proceedings of Senseval-3: Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text, Barcelona, Spain, 41-43.

    Google Scholar 

  • Tufiú, Dan, Dan Cristea & Sofia Stamou. 2004. BalkaNet: Aims, methods, results, and perspectives. A general overview. Romanian Journal of Information Science and Technology, 7(1-2): 9-43.

    Google Scholar 

  • Vossen, Piek, ed. 1998. EuroWordNet: A Multilingual Database with Lexical Semantic Networks. Dordrecht: Kluwer Academic Publishers.

    Google Scholar 

  • Weischedel, Ralph & Martha Palmer. 2004. An OntoBank pilot study: Annotating word sense and co-reference. DARPA TIDES PI Meeting, Philadelphia, PA, July 13-15.

    Google Scholar 

  • Yarowsky, David. 1993. One sense per collocation. Proceedings of ARPA Human Language Technology Workshop, Princeton, NJ, 266-271.

    Google Scholar 

  • Yarowsky, David, Radu Florian, Siviu Cucerzan & Charles Schafer. 2001. The Johns Hopkins Senseval-2 system description. Proceedings of Senseval-2: Second International Workshop on Evaluating Word Sense Disambiguation Systems, Toulouse, France, 163-166.

    Google Scholar 

  • Yarowsky, David & Radu Florian. 2002. Evaluating sense disambiguation performance across diverse parameter spaces. Natural Language Engineering, 8 (4): 293-310.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Departments of Linguistics and Computer Science, University of Colorado, Hellems 295, 80309, Boulder, CO, USA

    Associate Professor Martha Palmer

  2. Department of Computer Science, National University of Singapore, 3 Science Drive 2, 117543, Singapore

    Associate Professor Hwee Tou Ng

  3. National Institute of Standards and Technology, 100 Bureau Drive, 8940, 20899-8940, Gaithersburg, MD, USA

    Computer Scientist Hoa Trang Dang

Authors
  1. Associate Professor Martha Palmer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Associate Professor Hwee Tou Ng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Computer Scientist Hoa Trang Dang
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science, University of the Basque Country, Manuel de Lardizabal 1, E-20018, Donostia, Basque Country, Spain

    Eneko Agirre

  2. Sharp Laboratories of Europe Limited, Oxford Science Park, OX4 4GB, Oxford, UK

    Philip Edmonds

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer

About this chapter

Cite this chapter

Palmer, M., Ng, H.T., Dang, H.T. (2007). Evaluation of WSD Systems. In: Agirre, E., Edmonds, P. (eds) Word Sense Disambiguation. Text, Speech and Language Technology, vol 33. Springer, Dordrecht. https://doi.org/10.1007/978-1-4020-4809-8_4

Download citation

Publish with us

Policies and ethics

Search

Navigation