Entity Synonym Discovery via Multiple Attentions

  • Jiale Yu
  • Weiming LuEmail author
  • Wei Xu
  • Zeyun Tang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12032)


Entity synonym discovery is an important task, and it can benefit many downstream applications, such as web search, question answering and knowledge graph construction. Two types of approaches are widely exploited to discover synonyms from a raw text corpus, including the distributional based approaches and pattern based approaches. However, they suffered from either low precision or low recall. In this paper, we propose a novel framework SynMine to extract synonyms from massive raw text corpora. The framework can integrate corpus-level statistics and local contexts in a unified way via a multi-attention mechanism. Extensive experiments on a real-world dataset show the effectiveness of our approach.


Synonym discovery Distant supervision Multi-attention 



This work is supported by the Zhejiang Provincial Natural Science Foundation of China (No. LY17F020015), the Fundamental Research Funds for the Central Universities (No. 2019FZA5013), the Chinese Knowledge Center of Engineering Science and Technology (CKCEST) and MOE Engineering Research Center of Digital Library.


  1. 1.
    Antoniak, M., Bell, E., Xia, F.: Leveraging paraphrase labels to extract synonyms from twitter. In: FLAIRS Conference (2015)Google Scholar
  2. 2.
    Bordes, A., Usunier, N., García-Durán, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: NIPS (2013)Google Scholar
  3. 3.
    Boteanu, A., Kiezun, A., Artzi, S.: Synonym expansion for large shopping taxonomies. In: AKBC (2019)Google Scholar
  4. 4.
    Cafarella, M.J., Halevy, A.Y., Wang, D.Z., Wu, E., Zhang, Y.: Webtables: exploring the power of tables on the web. Proc. VLDB Endow. 1(1), 538–549 (2008)CrossRefGoogle Scholar
  5. 5.
    Chakrabarti, K., Chaudhuri, S., Cheng, T., Xin, D.: A framework for robust discovery of entity synonyms. In: KDD (2012)Google Scholar
  6. 6.
    Chaudhuri, S., Ganti, V., Xin, D.: Exploiting web search to generate synonyms for entities. In: WWW (2009)Google Scholar
  7. 7.
    Cheng, T., Lauw, H.W., Paparizos, S.: Entity synonyms for structured web search. IEEE Trans. Knowl. Data Eng. 24, 1862–1875 (2012)CrossRefGoogle Scholar
  8. 8.
    Clements, M., de Vries, A.P., Reinders, M.J.T.: Detecting synonyms in social tagging systems to improve content retrieval. In: SIGIR (2008)Google Scholar
  9. 9.
    Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. CoRR abs/1810.04805 (2018)Google Scholar
  10. 10.
    Fellbaum, C.: Wordnet: An Electronic Lexical Database (2000)Google Scholar
  11. 11.
    He, Y., Chakrabarti, K., Cheng, T., Tylenda, T.: Automatic discovery of attribute synonyms using query logs and table corpora. In: WWW (2016)Google Scholar
  12. 12.
    Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.R.: Improving neural networks by preventing co-adaptation of feature detectors. CoRR abs/1207.0580 (2012)Google Scholar
  13. 13.
    Ji, G., Liu, K., He, S., Zhao, J.: Distant supervision for relation extraction with sentence-level attention and entity descriptions. In: AAAI (2017)Google Scholar
  14. 14.
    Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: ICLR (2015)Google Scholar
  15. 15.
    Lehmann, J., et al.: Dbpedia - a large-scale, multilingual knowledge base extracted from wikipedia. Semant. Web 6, 167–195 (2015)CrossRefGoogle Scholar
  16. 16.
    Li, Q., et al.: Truepie: discovering reliable patterns in pattern-based information extraction. In: KDD, pp. 1675–1684. ACM (2018)Google Scholar
  17. 17.
    Lin, D., Zhao, S., Qin, L., Zhou, M.: Identifying synonyms among distributionally similar words. In: IJCAI, vol. 3, pp. 1492–1493. Citeseer (2003)Google Scholar
  18. 18.
    Lin, Y., Shen, S., Liu, Z., Luan, H., Sun, M.: Neural relation extraction with selective attention over instances. In: ACL, vol. 1, pp. 2124–2133 (2016)Google Scholar
  19. 19.
    Mintz, M., Bills, S., Snow, R., Jurafsky, D.: Distant supervision for relation extraction without labeled data. In: ACL/IJCNLP, pp. 1003–1011. Association for Computational Linguistics (2009)Google Scholar
  20. 20.
    Nguyen, K.A., Schulte im Walde, S., Vu, N.T.: Distinguishing antonyms and synonyms in a pattern-based neural network. In: EACL, pp. 76–85 (2017)Google Scholar
  21. 21.
    Pantel, P., Crestan, E., Borkovsky, A., Popescu, A., Vyas, V.: Web-scale distributional similarity and entity set expansion. In: EMNLP, pp. 938–947. Association for Computational Linguistics (2009)Google Scholar
  22. 22.
    Qin, P., Xu, W., Wang, W.Y.: Robust distant supervision relation extraction via deep reinforcement learning. In: ACL (2018)Google Scholar
  23. 23.
    Qu, M., Ren, X., Han, J.: Automatic synonym discovery with knowledge bases. In: KDD, pp. 997–1005. ACM (2017)Google Scholar
  24. 24.
    Qu, M., Ren, X., Zhang, Y., Han, J.: Weakly-supervised relation extraction by pattern-enhanced embedding learning. In: WWW, pp. 1257–1266. International World Wide Web Conferences Steering Committee (2018)Google Scholar
  25. 25.
    Ren, X., Cheng, T.: Synonym discovery for structured entities on heterogeneous graphs. In: WWW (2015)Google Scholar
  26. 26.
    Riedel, S., Yao, L., McCallum, A.: Modeling relations and their mentions without labeled text. In: Balcázar, J.L., Bonchi, F., Gionis, A., Sebag, M. (eds.) ECML PKDD 2010, Part III. LNCS (LNAI), vol. 6323, pp. 148–163. Springer, Heidelberg (2010). Scholar
  27. 27.
    Roller, S., Erk, K., Boleda, G.: Inclusive yet selective: supervised distributional hypernymy detection. In: COLING, pp. 1025–1036 (2014)Google Scholar
  28. 28.
    Shen, J., Wu, Z., Lei, D., Shang, J., Ren, X., Han, J.: Setexpan: corpus-based set expansion via context feature selection and rank ensemble. In: ECML/PKDD (2017)Google Scholar
  29. 29.
    Snow, R., Jurafsky, D., Ng, A.Y.: Learning syntactic patterns for automatic hypernym discovery. In: NIPS, pp. 1297–1304 (2005)Google Scholar
  30. 30.
    Speer, R., Chin, J., Havasi, C.: Conceptnet 5.5: an open multilingual graph of general knowledge. In: AAAI (2017)Google Scholar
  31. 31.
    Wang, J., Lin, C., Li, M., Zaniolo, C.: An efficient sliding window approach for approximate entity extraction with synonyms. In: EDBT (2019)Google Scholar
  32. 32.
    Wang, Z., Hamza, W., Florian, R.: Bilateral multi-perspective matching for natural language sentences. In: IJCAI, pp. 4144–4150 (2017)Google Scholar
  33. 33.
    Wei, X., Peng, F., Tseng, H., Lu, Y., Dumoulin, B.: Context sensitive synonym discovery for web search queries. In: CIKM, pp. 1585–1588. ACM (2009)Google Scholar
  34. 34.
    Zeng, D., Liu, K., Chen, Y., Zhao, J.: Distant supervision for relation extraction via piecewise convolutional neural networks. In: EMNLP (2015)Google Scholar
  35. 35.
    Zhou, G., Liu, Y., Liu, F., Zeng, D., Zhao, J.: Improving question retrieval in community question answering using world knowledge. In: IJCAI (2013)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.College of Computer ScienceZhejiang UniversityHangzhouChina

Personalised recommendations