Skip to main content
SpringerLink
Log in

Learning Word Sense Embeddings from Word Sense Definitions

  • Conference paper
  • First Online:
Natural Language Understanding and Intelligent Applications (ICCPOL 2016, NLPCC 2016)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10102))

Abstract

Word embeddings play a significant role in many modern NLP systems. Since learning one representation per word is problematic for polysemous words and homonymous words, researchers propose to use one embedding per word sense. Their approaches mainly train word sense embeddings on a corpus. In this paper, we propose to use word sense definitions to learn one embedding per word sense. Experimental results on word similarity tasks and a word sense disambiguation task show that word sense embeddings produced by our approach are of high quality.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://wordnet.princeton.edu/.

  2. 2.

    https://code.google.com/archive/p/word2vec/.

  3. 3.

    https://wordnet.princeton.edu/man/sensemap.5WN.html.

  4. 4.

    http://dumps.wikimedia.org/enwiki/.

References

  1. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate (2014). CoRR arXiv:1409.0473

  2. Chen, X., Liu, Z., Sun, M.: A unified model for word sense representation and disambiguation. In: EMNLP (2014)

    Google Scholar 

  3. Chung, J., Gülehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling (2014). CoRR arXiv:1412.3555

  4. Finkelstein, L., Gabrilovich, E., Matias, Y., Rivlin, E., Solan, Z., Wolfman, G., Ruppin, E.: Placing search in context: the concept revisited. ACM Trans. Inf. Syst. 20, 116–131 (2001)

    Article  Google Scholar 

  5. Guo, J., Che, W., Wang, H., Liu, T.: Learning sense-specific word embeddings by exploiting bilingual resources. In: COLING (2014)

    Google Scholar 

  6. Hermann, K.M., Kociský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., Blunsom, P.: Teaching machines to read and comprehend (2015). CoRR arXiv:1506.03340

  7. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–1780 (1997)

    Article  Google Scholar 

  8. Huang, E.H., Socher, R., Manning, C.D., Ng, A.Y.: Improving word representations via global context and multiple word prototypes. In: Annual Meeting of the Association for Computational Linguistics (ACL) (2012)

    Google Scholar 

  9. Iacobacci, I., Pilehvar, M.T., Navigli, R.: SensEmbed: learning sense embeddings for word and relational similarity. In: ACL (2015)

    Google Scholar 

  10. Kalchbrenner, N., Blunsom, P.: Recurrent continuous translation models. In: EMNLP (2013)

    Google Scholar 

  11. Li, J., Jurafsky, D.: Do multi-sense embeddings improve natural language understanding? In: EMNLP (2015)

    Google Scholar 

  12. Liu, Y., Liu, Z., Chua, T.S., Sun, M.: Topical word embeddings. In: AAAI (2015)

    Google Scholar 

  13. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space (2013). CoRR arXiv:1301.3781

  14. Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality (2013). CoRR arXiv:1310.4546

  15. Miller, G.A.: Wordnet: a lexical database for English. Commun. ACM 38, 39–41 (1992)

    Article  Google Scholar 

  16. Moro, A., Raganato, A., Navigli, R.: Entity linking meets word sense disambiguation: a unified approach. TACL 2, 231–244 (2014)

    Google Scholar 

  17. Navigli, R., Litkowski, K.C., Hargraves, O.: SemEval-2007 task 07: Coarse-grained English all-words task (2007)

    Google Scholar 

  18. Navigli, R., Ponzetto, S.P.: Babelnet: The automatic construction, evaluation and application of a wide-coverage multilingual semantic network. Artif. Intell. 193, 217–250 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  19. Neelakantan, A., Shankar, J., Passos, A., McCallum, A.: Efficient non-parametric estimation of multiple embeddings per word in vector space. In: EMNLP (2014)

    Google Scholar 

  20. Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: EMNLP (2014)

    Google Scholar 

  21. Reisinger, J., Mooney, R.J.: Multi-prototype vector-space models of word meaning. In: NAACL (2010)

    Google Scholar 

  22. Rocktäschel, T., Grefenstette, E., Hermann, K.M., Kociský, T., Blunsom, P.: Reasoning about entailment with neural attention (2015). CoRR arXiv:1509.06664

  23. Tian, F., Dai, H., Bian, J., Gao, B., Zhang, R., Chen, E., Liu, T.Y.: A probabilistic model for learning multi-prototype word embeddings. In: COLING (2014)

    Google Scholar 

  24. Wu, Z., Giles, C.L.: Sense-aware semantic analysis: a multi-prototype word representation model using Wikipedia (2015)

    Google Scholar 

  25. Zeiler, M.D.: ADADELTA: an adaptive learning rate method (2012). CoRR arXiv:1212.5701

Download references

Acknowledgments

This work is supported by National Key Basic Research Program of China under Grant No. 2014CB340504 and National Natural Science Foundation of China under Grant No. 61273318. The Corresponding author of this paper is Baobao Chang.

Author information

Authors and Affiliations

  1. Key Laboratory of Computational Linguistics, Ministry of Education School of Electronics Engineering and Computer Science, Peking University, No. 5 Yiheyuan Road, Haidian District, Beijing, 100871, China

    Qi Li, Tianshi Li & Baobao Chang

  2. Collaborative Innovation Center for Language Ability, Xuzhou, 221009, China

    Qi Li, Tianshi Li & Baobao Chang

Authors
  1. Qi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tianshi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Baobao Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Baobao Chang .

Editor information

Editors and Affiliations

  1. Microsoft Research Asia, Beijing, China

    Chin-Yew Lin

  2. Brandeis University, Waltham, Massachusetts, USA

    Nianwen Xue

  3. Peking University, Beijing, China

    Dongyan Zhao

  4. Fudan University, Shanghai, China

    Xuanjing Huang

  5. Peking University, Beijing, China

    Yansong Feng

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Li, Q., Li, T., Chang, B. (2016). Learning Word Sense Embeddings from Word Sense Definitions. In: Lin, CY., Xue, N., Zhao, D., Huang, X., Feng, Y. (eds) Natural Language Understanding and Intelligent Applications. ICCPOL NLPCC 2016 2016. Lecture Notes in Computer Science(), vol 10102. Springer, Cham. https://doi.org/10.1007/978-3-319-50496-4_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-50496-4_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-50495-7

  • Online ISBN: 978-3-319-50496-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation