Bidirectional Gated Recurrent Unit Networks for Relation Classification with Multiple Attentions and Semantic Information

  • Bixiao Meng
  • Baomin XUEmail author
  • Erjing Zhou
  • Shuangyuan YU
  • Hongfeng Yin
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11554)


Relation classification is an important part in natural language processing (NLP) field. The main task of relation classification is extracting the relations between target entities. In recent years, there are many methods for relation classification and some of them have achieved quite good results, but these methods have not given enough attention to the target words, and the semantic information of words is also lack of utilization. In order to make good use of the contextual information in the sentences as much as possible, we adopt the bidirectional gated recurrent unit networks (BGRU). On this basis, in order to focus on the computing process of target entities and target sentences, we add the multiple attention mechanism. Meanwhile, other semantic information such as the named entity and part of speech information of the word are also added as input data so as to make full use of the words’ information in the corpus. We have conducted some experiments on the widely used datasets, and we got up to 3% improvement in the F1 value compared to previous optimal method.


Relation extraction Attention mechanism Bidirectional GRU 



This work was supported by the Key Projects of Science and Technology Research of Hebei Province Higher Education [ZD2017304].


  1. 1.
    Cui, H., Sun, R., Li, K., Kan, M.Y., Chua, T.S.: Question answering passage retrieval using dependency relations. In: 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 400–407. ACM Press, Salvador (2005)Google Scholar
  2. 2.
    Mottin, D., Lissandrini, M., Velegrakis, Y., Palpanas, T.: Exemplar queries: give me an example of what you need. Proc. VLDB Endow. 7(5), 365–376 (2014)Google Scholar
  3. 3.
    Zelenko, D., Aone, C., Richardella, A.: Kernel methods for relation extraction. Mach. Learn. Res. 3(Feb), 1083–1106 (2003)Google Scholar
  4. 4.
    Kambhatla, N.: Combining lexical, syntactic, and semantic features with maximum entropy models for extracting relations. In: ACL 2004 on Interactive Poster and Demonstration Sessions, p. 22. Association for Computational Linguistics. Barcelona (2004)Google Scholar
  5. 5.
    Qu, M., Ren, X., Zhang, Y., Han, J.: Weakly-supervised relation extraction by pattern-enhanced embedding learning. In: 2018 World Wide Web Conference on World Wide Web, pp. 1257–1266. Lyon (2018)Google Scholar
  6. 6.
    Zeng, D., Liu, K., Lai, S., Zhou, G., Zhao, J.: Relation classification via convolutional deep neural network. In: the 25th International Conference on Computational Linguistics: Technical Papers, pp. 2335–2344, Dublin (2014)Google Scholar
  7. 7.
    Santos, C.N.D., Xiang, B., Zhou, B.: Classifying relations by ranking with convolutional neural networks. In: International Joint Conference on Natural Language Processing, pp. 626–634 (2015)Google Scholar
  8. 8.
    Zeng, D., Liu, K., Chen, Y., Zhao, J.: Distant supervision for relation extraction via piecewise convolutional neural networks. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1753–1762, Lisbon (2015)Google Scholar
  9. 9.
    Lin, Y., Shen, S., Liu, Z., Luan, H., Sun, M.: Neural relation extraction with selective attention over instances. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pp. 2124–2133. Berlin (2016)Google Scholar
  10. 10.
    Zhang, D., Wang, D.: Relation classification via recurrent neural network. J. Comput. Lang. arXiv:1508.01006v2 (2015)
  11. 11.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)Google Scholar
  12. 12.
    Xu, Y., Mou, L., Li, G., Chen, Y., Peng, H., Jin, Z.: Classifying relations via long short term memory networks along shortest dependency paths. In: 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1785–1794. Lisbon (2015)Google Scholar
  13. 13.
    Peng, N., Poon, H., Quirk, C., Toutanova, K., Yih, W.T.: Cross-sentence N-ary relation extraction with graph LSTMs. Transact. Assoc. Comput. Linguist. 5(1), 101–115 (2017)Google Scholar
  14. 14.
    Zhou, P., et al.: Attention-based bidirectional long short-term memory networks for relation classification. In: the 54th Annual Meeting of the Association for Computational Linguistics, pp. 207–212. Berlin (2016)Google Scholar
  15. 15.
    Cho, K., et al.: Learning phrase representations using RNN encoder–decoder for statistical machine translation. In: Empirical Methods in Natural Language Processing, pp. 1724–1734 (2014)Google Scholar
  16. 16.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119 (2013)Google Scholar
  17. 17.
    Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.R.: Improving neural networks by preventing co-adaptation of feature detectors. Comput. Sci., arXiv preprint arXiv:1207.0580 (2012)
  18. 18.
    Zhang, S., Zheng, D., Hu, X., Yang, M.: Bidirectional long short-term memory networks for relation classification. In: the 29th Pacific Asia Conference on Language, Information and Computation, Shanghai, pp. 73–78 (2015)Google Scholar
  19. 19.
    Ji, G., Liu, K., He, S., Zhao, J.: Distant supervision for relation extraction with sentence-level attention and entity descriptions. In: the 31st AAAI Conference on Artificial Intelligence, San Francisco, pp. 3060–3066 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Bixiao Meng
    • 1
  • Baomin XU
    • 1
    Email author
  • Erjing Zhou
    • 1
  • Shuangyuan YU
    • 1
  • Hongfeng Yin
    • 2
  1. 1.School of Computer and Information TechnologyBeijing Jiaotong UniversityBeijingChina
  2. 2.Department of Computer ScienceBeijing Jiaotong University Haibin CollegeHuanghuaChina

Personalised recommendations