Advertisement

Learning Lexical-Semantic Relations Using Intuitive Cognitive Links

  • Georgios Balikas
  • Gaël DiasEmail author
  • Rumen Moraliyski
  • Houssam Akhmouch
  • Massih-Reza Amini
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11437)

Abstract

Identifying the specific semantic relations between words is crucial for IR and NLP systems. Our goal in this paper is twofold. First, we want to understand whether learning a classifier for one semantic relation (e.g. hypernymy) can gain from concurrently learning another classifier for a cognitively-linked semantic relation (e.g. co-hyponymy). Second, we evaluate how these systems perform where only few labeled examples exist. To answer the first question, we rely on a multi-task neural network architecture, while for the second we use self-learning to evaluate whether semi-supervision improves performance. Our results on two popular datasets as well as a novel dataset proposed in this paper show that concurrent learning of semantic relations consistently benefits performance. On the other hand, we find that semi-supervised learning can be useful depending on the semantic relation. The code and the datasets are available at https://bit.ly/2Qitasd.

References

  1. 1.
    Amini, M., Laviolette, F., Usunier, N.: A transductive bound for the voted classifier with an application to semi-supervised learning. In: 22nd Annual Conference on Neural Information Processing Systems (NIPS), pp. 65–72 (2008)Google Scholar
  2. 2.
    Amini, M.R., Usunier, N.: Learning with Partially Labeled and Interdependent Data. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-15726-9CrossRefzbMATHGoogle Scholar
  3. 3.
    Attia, M., Maharjan, S., Samih, Y., Kallmeyer, L., Solorio, T.: Cogalex-V shared task: GHHH - detecting semantic relations via word embeddings. In: Workshop on Cognitive Aspects of the Lexicon, pp. 86–91 (2016)Google Scholar
  4. 4.
    Baroni, M., Bernardi, R., Do, N.Q., Shan, C.-C: Entailment above the word level in distributional semantics. In: 13th Conference of the European Chapter of the Association for Computational Linguistics (EACL), pp. 23–32 (2012)Google Scholar
  5. 5.
    Baroni, M., Lenci, A.: How we blessed distributional semantic evaluation. In: Workshop on Geometrical Models of Natural Language Semantics (GEMS) Associated to Conference on Empirical Methods on Natural Language Processing (EMNLP), pp. 1–10 (2011)Google Scholar
  6. 6.
    Benotto, G.: Distributional Models for Semantic Relations: A Study on Hyponymy and Antonymy. Ph.D. thesis, University of Pisa (2015)Google Scholar
  7. 7.
    Bingel, J., Søgaard, A.: Identifying beneficial task relations for multi-task learning in deep neural networks. arXiv preprint arXiv:1702.08303 (2017)
  8. 8.
    Braud, C., Plank, B., Søgaard, A.: Multi-view and multi-task training of RST discourse parsers. In: 26th International Conference on Computational Linguistics (COLING), pp. 1903–1913 (2016)Google Scholar
  9. 9.
    Caruana, R.: Multitask learning. In: Thrun, S., Pratt, L. (eds.) Learning to Learn, pp. 95–133. Springer, Boston (1998).  https://doi.org/10.1007/978-1-4615-5529-2_5CrossRefGoogle Scholar
  10. 10.
    Chapelle, O., Scholkopf, B., Zien, A.: Semi-supervised learning. IEEE Trans. Neural Networks 20(3), 542 (2009)CrossRefGoogle Scholar
  11. 11.
    Chollet, F.: Keras. https://keras.io (2015)
  12. 12.
    Dias, G., Moraliyski, R., Cordeiro, J., Doucet, A., Ahonen-Myka, H.: Automatic discovery of word semantic relations using paraphrase alignment and distributional lexical semantics analysis. Nat. Lang. Eng. 16(4), 439–467 (2010)CrossRefGoogle Scholar
  13. 13.
    Dong, L., Mallinson, J., Reddy, S., Lapata, M.: Learning to paraphrase for question answering. In: Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 886–897 (2017)Google Scholar
  14. 14.
    Gambhir, M., Gupta, V.: Recent automatic text summarization techniques: a survey. Artif. Intell. Rev. 47(1), 1–66 (2017)CrossRefGoogle Scholar
  15. 15.
    Glavas, G., Ponzetto, S.P.: Dual tensor model for detecting asymmetric lexico-semantic relations. In: Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1758–1768 (2017)Google Scholar
  16. 16.
    Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: 13th International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)Google Scholar
  17. 17.
    Harris, Z.S.: Distributional structure. Word 10(2–3), 146–162 (1954)CrossRefGoogle Scholar
  18. 18.
    Hearst, M.: Automatic acquisition of hyponyms from large text corpora. In: 14th Conference on Computational Linguistics (COLING), pp. 539–545 (1992)Google Scholar
  19. 19.
    Kathuria, N., Mittal, K., Chhabra, A.: A comprehensive survey on query expansion techniques, their issues and challenges. Int. J. Comput. Appl. 168(12), (2017)CrossRefGoogle Scholar
  20. 20.
    Kiela, D., Hill, F., Clark, S.: Specializing word embeddings for similarity or relatedness. In: Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 2044–2048 (2015)Google Scholar
  21. 21.
    Kotlerman, L., Dagan, I., Szpektor, I., Zhitomirsky-Geffet, M.: Directional distributional similarity for lexical inference. Nat. Lang. Eng. 16(4), 359–389 (2010)CrossRefGoogle Scholar
  22. 22.
    Levy, O., Remus, S., Biemann, C., Dagan, I.: Do supervised distributional methods really learn lexical inference relations? In: Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 970–976 (2015)Google Scholar
  23. 23.
    Liu, P., Qiu, X., Huang, X.: Adversarial multi-task learning for text classification. In: 55th Annual Meeting of the Association for Computational Linguistics (ACL) (2017)Google Scholar
  24. 24.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. CoRR abs/1301.3781 (2013)Google Scholar
  25. 25.
    Miller, G.A., Beckwith, R., Fellbaum, C., Gross, D., Miller, K.J.: Introduction to wordnet: an on-line lexical database. Int. J. Lexicogr. 3(4), 235–244 (1990)CrossRefGoogle Scholar
  26. 26.
    Nguyen, K.A., Köper, M., Schulte im Walde, S., Vu, N.T.: Hierarchical embeddings for hypernymy detection and directionality. In: Conference on Empirical Methods in Natural Language Processing, pp. 233–243 (2017)Google Scholar
  27. 27.
    Nguyen, K.A., Schulte im Walde, S., Vu, N.T.: Distinguishing antonyms and synonyms in a pattern-based neural network. In: 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL), pp. 76–85 (2017)Google Scholar
  28. 28.
    Pedregosa, F., et al.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12(Oct), 2825–2830 (2011)MathSciNetzbMATHGoogle Scholar
  29. 29.
    Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: Conference on Empirical Methods on Natural Language Processing (EMNLP), pp. 1532–1543 (2014)Google Scholar
  30. 30.
    Roller, S., Erk, K., Boleda, G.: Inclusive yet selective: Supervised distributional hypernymy detection. In: 25th International Conference on Computational Linguistics (COLING), pp. 1025–1036 (2014)Google Scholar
  31. 31.
    Ruder, S., Plank, B.: Strong baselines for neural semi-supervised learning under domain shift. In: 56th Annual Meeting of the Association for Computational Linguistics (ACL) (2018)Google Scholar
  32. 32.
    Santus, E., Lenci, A., Chiu, T., Lu, Q., Huang, C.: Nine features in a random forest to learn taxonomical semantic relations. In: 10th International Conference on Language Resources and Evaluation, pp. 4557–4564 (2016)Google Scholar
  33. 33.
    Santus, E., Shwartz, V., Schlechtweg, D.: Hypernyms under siege: linguistically-motivated artillery for hypernymy detection. In: 15th Conference of the European Chapter of the Association for Computational Linguistics, pp. 65–75 (2017)Google Scholar
  34. 34.
    Santus, E., Yung, F., Lenci, A., Huang, C.R.: Evalution 1.0: an evolving semantic dataset for training and evaluation of distributional semantic models. In: 4th Workshop on Linked Data in Linguistics (LDL) Associated to Association for Computational Linguistics and Asian Federation of Natural Language Processing (ACL-IJCNLP), pp. 64–69 (2015)Google Scholar
  35. 35.
    Shwartz, V., Dagan, I.: Cogalex-V shared task: lexnet - integrated path-based and distributional method for the identification of semantic relations. CoRR abs/1610.08694 (2016)Google Scholar
  36. 36.
    Shwartz, V., Goldberg, Y., Dagan, I.: Improving hypernymy detection with an integrated path-based and distributional method. In: 54th Annual Meeting of the Association for Computational Linguistics, pp. 2389–2398 (2016)Google Scholar
  37. 37.
    Snow, R., Jurafsky, D., Ng, A.Y.: Learning syntactic patterns for automatic hypernym discovery. In: 17th International Conference on Neural Information Processing Systems (NIPS), pp. 1297–1304 (2004)Google Scholar
  38. 38.
    Vylomova, E., Rimell, L., Cohn, T., Baldwin, T.: Take and took, gaggle and goose, book and read: evaluating the utility of vector differences for lexical relation learning. In: 54th Annual Meeting of the Association for Computational Linguistics, pp. 1671–1682 (2016)Google Scholar
  39. 39.
    Weeds, J., Clarke, D., Reffin, J., Weir, D.J., Keller, B.: Learning to distinguish hypernyms and co-hyponyms. In: 5th International Conference on Computational Linguistics (COLING), pp. 2249–2259 (2014)Google Scholar
  40. 40.
    Yu, Z., Wang, H., Lin, X., Wang, M.: Learning term embeddings for hypernymy identification. In: 24th International Joint Conference on Artificial Intelligence, pp. 1390–1397 (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Georgios Balikas
    • 1
  • Gaël Dias
    • 2
    Email author
  • Rumen Moraliyski
    • 3
  • Houssam Akhmouch
    • 2
    • 4
  • Massih-Reza Amini
    • 5
  1. 1.Kelkoo GroupGrenobleFrance
  2. 2.Normandy University, CNRS GREYCCaenFrance
  3. 3.Kodar Ltd.PlovdivBulgaria
  4. 4.Credit Agricole Brie PicardieAmiensFrance
  5. 5.University of Grenoble Alps, CNRS LIGGrenobleFrance

Personalised recommendations