Is a Common Phrase an Entity Mention or Not? Dual Representations for Domain-Specific Named Entity Recognition

  • Jiangtao Zhang
  • Juanzi Li
  • Xiao-Li Li
  • Yixin Cao
  • Lei Hou
  • Shuai Wang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10827)


Named Entity Recognition (NER) for specific domains is critical for building and managing domain-specific knowledge bases, but conventional NER methods cannot be applied to specific domains effectively. We found that one of reasons is the problem of common-phrase-like entity mention prevalent in many domains. That is, many common phrases frequently occurring in general corpora may or may not be treated as named entities in specific domains. Therefore, determining whether a common phrase is an entity mention or not is a challenge. To address this issue, we present a novel BLSTM based NER model tailored for specific domains by learning dual representations for each word. It learns not only general domain knowledge derived from an external large scale general corpus via a word embedding model, but also the specific domain knowledge by training a stacked deep neural network (SDNN) integrating the results of a low-cost pre-entity-linking process. Extensive experiments on a real-world dataset of movie comments demonstrate the superiority of our model over existing state-of-the-art methods.



The work is supported by major national research and development projects (2017YFB1002101), NSFC key project (U1736204, 61661146007), Fund of Online Education Research Center, Ministry of Education (No. 2016ZD102), and THU-NUS NExT Co-Lab.


  1. 1.
    Ando, R.K., Zhang, T.: A framework for learning predictive structures from multiple tasks and unlabeled data. J. Mach. Learn. Res. 6, 1817–1853 (2005)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, vol. 26, pp. 2787–2795 (2013)Google Scholar
  3. 3.
    Cao, Y., Huang, L., Ji, H., Chen, X., Li, J.: Bridge text and knowledge by learning multi-prototype entity mention embedding. In: ACL (2017)Google Scholar
  4. 4.
    Cao, Y., Li, J., Guo, X., Bai, S., Ji, H., Tang, J.: Name list only? Target entity disambiguation in short texts. In: EMNLP (2015)Google Scholar
  5. 5.
    Chiu, J.P.C., Nichols, E.: Named entity recognition with bidirectional LSTM-CNNs. TACL 4, 357–370 (2016)Google Scholar
  6. 6.
    Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)zbMATHGoogle Scholar
  7. 7.
    Durrett, G., Klein, D.: A joint model for entity analysis: coreference, typing, and linking. In: TACL (2014)Google Scholar
  8. 8.
    Florian, R., Ittycheriah, A., Jing, H., Zhang, T.: Named entity recognition through classifier combination. In: Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, CONLL 2003, vol. 4, pp. 168–171 (2003)Google Scholar
  9. 9.
    Fukuda, K., Tsunoda, T., Tamura, A., Takagi, T.: Information extraction: identifying protein names from biological papers. In: PSB, pp. 707–718 (1998)Google Scholar
  10. 10.
    Gaizauskas, R., Demetriou, G., Humphreys, K.: Term recognition and classification in biological science journal articles. In: Proceedings of the Computational Terminology for Medical and Biological Applications Workshop of the 2nd International Conference on NLP, pp. 37–44 (2000)Google Scholar
  11. 11.
    Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw. 18(5–6), 602–610 (2005)CrossRefGoogle Scholar
  12. 12.
    Gu, B.: Recognizing nested named entities in GENIA corpus. In: Proceedings of the BioNLP Workshop on Linking Natural Language Processing and Biology at HLT-NAACL 2006, pp. 112–113 (2006)Google Scholar
  13. 13.
    Henriksson, A., Dalianis, H., Kowalski, S.: Generating features for named entity recognition by learning prototypes in semantic space: the case of de-identifying health records. In: 2014 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 450–457 (2014)Google Scholar
  14. 14.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  15. 15.
    Hovy, E., Marcus, M., Palmer, M., Ramshaw, L., Weischedel, R.: Ontonotes: the 90% solution. In: NAACL-Short 2006, pp. 57–60 (2006)Google Scholar
  16. 16.
    Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. CoRR abs/1508.01991 (2015)Google Scholar
  17. 17.
    Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, pp. 2181–2187. AAAI (2015)Google Scholar
  18. 18.
    Luo, G., Huang, X., Nie, Z., Lin, C.-Y.: Joint named entity recognition and disambiguation. In: EMNLP, pp. 879–888 (2015)Google Scholar
  19. 19.
    Ma, X., Hovy, E.H.: End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. CoRR abs/1603.01354 (2016)Google Scholar
  20. 20.
    Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: EMNLP, vol. 14, pp. 1532–1543 (2014)Google Scholar
  21. 21.
    Ratinov, L., Roth, D.: Design challenges and misconceptions in named entity recognition. In: CoNLL (2009)Google Scholar
  22. 22.
    Tjong Kim Sang, E.F., De Meulder, F.: Introduction to the CoNLL-2003 shared task: language-independent named entity recognition. In: HLT-NAACL 2003, vol. 4, pp. 142–147 (2003)Google Scholar
  23. 23.
    Shen, W., Wang, J., Han, J.: Entity linking with a knowledge base: issues, techniques, and solutions. Trans. Knowl. Data Eng. 27, 443–460 (2015)CrossRefGoogle Scholar
  24. 24.
    Tomori, S., Ninomiya, T., Mori, S.: Domain specific named entity recognition referring to the real world by deep neural networks. In: ACL, vol. 2, Short Papers (2016)Google Scholar
  25. 25.
    Viterbi, A.: Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Trans. Inf. Theor. 13, 260–269 (2006)CrossRefGoogle Scholar
  26. 26.
    Wang, J., Zhao, W.X., Wei, H., Yan, H., Li, X.: Mining new business opportunities: identifying trend related products by leveraging commercial intents from microblogs. In: EMNLP, pp. 1337–1347 (2013)Google Scholar
  27. 27.
    Wang, P., Qian, Y., Soong, F.K., He, L., Zhao, H.: A unified tagging solution: bidirectional LSTM recurrent neural network with word embedding. CoRR abs/1511.00215 (2015)Google Scholar
  28. 28.
    Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: AAAI, pp. 1112–1119 (2014)Google Scholar
  29. 29.
    Yang, Z., Lin, H., Li, Y.: Exploiting the contextual cues for bio-entity name recognition in biomedical literature. J. Biomed. Inform. 41, 580–587 (2008)CrossRefGoogle Scholar
  30. 30.
    Zhang, J., Li, J., Li, X.-L., Shi, Y., Li, J., Wang, Z.: Domain-specific entity linking via fake named entity detection. In: Navathe, S.B., Wu, W., Shekhar, S., Du, X., Wang, X.S., Xiong, H. (eds.) DASFAA 2016. LNCS, vol. 9642, pp. 101–116. Springer, Cham (2016). Scholar
  31. 31.
    Zhang, J., Cao, Y., Hou, L., Li, J., Zheng, H.-T.: XLink: an unsupervised bilingual entity linking system. In: Sun, M., Wang, X., Chang, B., Xiong, D. (eds.) CCL/NLP-NABD -2017. LNCS (LNAI), vol. 10565, pp. 172–183. Springer, Cham (2017). Scholar
  32. 32.
    Zhao, S.: Named entity recognition in biomedical texts using an hmm model. In: Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and Its Applications, JNLPBA 2004, pp. 84–87 (2004)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Jiangtao Zhang
    • 1
  • Juanzi Li
    • 1
  • Xiao-Li Li
    • 2
  • Yixin Cao
    • 1
  • Lei Hou
    • 1
  • Shuai Wang
    • 1
  1. 1.Department of Computer Science and TechnologyTsinghua UniversityBeijingChina
  2. 2.Institute for Infocomm Research, A*STARSingaporeSingapore

Personalised recommendations