Attention-Based Convolutional Neural Networks for Chinese Relation Extraction

  • Wenya Wu
  • Yufeng ChenEmail author
  • Jinan Xu
  • Yujie Zhang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11221)


Relation extraction is an important part of many information extraction systems that mines structured facts from texts. Recently, deep learning has achieved good results in relation extraction. Attention mechanism is also gradually applied to networks, which improves the performance of the task. However, the current attention mechanism is mainly applied to the basic features on the lexical level rather than the higher overall features. In order to obtain more information of high-level features for relation predicting, we proposed attention-based piecewise convolutional neural networks (PCNN_ATT), which add an attention layer after the piecewise max pooling layer in order to get significant information of sentence global features. Furthermore, we put forward a data extension method by utilizing an external dictionary HIT IR-Lab Tongyici Cilin (Extended). Experiments results on ACE-2005 and COAE-2016 Chinese datasets both demonstrate that our approach outperforms most of the existing methods.


Relation extraction Convolutional neural networks Attention mechanism 



The authors are supported by National Nature Science Foundation of China (Contract 61370130 and 61473294), and the Fundamental Research Funds for the Central Universities (2015JBM033), and International Science and Technology Cooperation Program of China under grant No. 2014DFA11350.


  1. Zeng, D., Liu, K., Lai, S., Zhou, G., Zhao, J.: Relation classification via convolutional deep neural network. In: Proceedings of COLING, pp. 2335–2344 (2014)Google Scholar
  2. Zeng, D., Liu, K., Chen, Y., Zhao, J.: Distant supervision for relation extraction via piecewise convolutional neural networks. In: Proceedings of EMNLP, pp. 17–21. Association for Computational Linguistics, Stroudsburg (2015)Google Scholar
  3. Lin, Y., Shen, S., Liu, Z., Luan, H., Sun, M.: Neural relation extraction with selective attention over instances. In: Proceedings of ACL, pp. 2124–2133. Association for Computational Linguistics, Berlin (2016)Google Scholar
  4. Jiang, X., Wang, Q., Li, P., Wang, B.: Relation extraction with multi-instance multi-label convolutional neural networks. In: Proceedings of COLING, pp. 1471–1480 (2016)Google Scholar
  5. dos Santos, C.N., Xiang, B., Zhou, B.: Classifying relations by ranking with convolutional neural networks. In: Proceedings of ACL (2015)Google Scholar
  6. Liu, Y., Wei, F., Li, S., Ji, H., Zhou, M., Wang, H.: A dependency-based neural network for relation classification. Comput. Sci. (2015)Google Scholar
  7. Bunescu, R.C., Mooney, R.J.: A shortest path dependency kernel for relation extraction. In: Proceedings of HLT/EMNLP, pp. 724–731. Association for Computational Linguistics, Vancouver (2005)Google Scholar
  8. Wang, L., Cao, Z., Melo, G.D., Liu, Z.: Relation classification via multi-level attention CNNs. In: Proceedings of ACL, pp. 1298–1307. Association for Computational Linguistics, Berlin (2016)Google Scholar
  9. Li, S., Xu, J., Zhang, Y., Chen, Y.: A method of unknown words processing for neural machine translation using HowNet. In: Wong, D.F., Xiong, D. (eds.) CWMT 2017. CCIS, vol. 787, pp. 20–29. Springer, Singapore (2017). Scholar
  10. Sun, J., Gu, X., Li, Y., Xu, W.: Chinese entity relation extraction algorithms based on COAE2016 datasets. J. Shandong Univ. (Nat. Sci.) 52(9), 7–12 (2017)Google Scholar
  11. Liu, D., Peng, C., Qian, L., Zhou, G.: The effect of Tongyici Cilin in Chinese entity relation extraction. J. Chin. Inf. Process. 28(2), 91–99 (2014)Google Scholar
  12. Cai, R., Zhang, X., Wang, H.: Bidirectional recurrent convolutional neural network for relation classification. In: Proceedings of ACL, pp. 756–765. Association for Computational Linguistics, Berlin (2016)Google Scholar
  13. Xu, K., Feng, Y., Huang, S., Zhao, D.: Semantic relation classification via convolutional neural networks with simple negative sampling. Comput. Sci. 71(7), 941–949 (2015)Google Scholar
  14. Zhou, P., et al.: Attention-based bidirectional long short-term memory networks for relation classification. In: Proceedings of ACL, pp. 207–212. Association for Computational Linguistics, Berlin (2016)Google Scholar
  15. Hashimoto, K., Miwa, M., Tsuruoka, Y., Chikayama, T.: Simple customization of recursive neural networks for semantic relation classification. In: Proceedings of EMNLP, pp. 1372–1376. Association for Computational Linguistics, Seattle (2013)Google Scholar
  16. Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. JMLR 12, 2493–2537 (2011)zbMATHGoogle Scholar
  17. Socher, R., Huval, B., Manning, C.D., Ng, A.Y.: Semantic compositionality through recursive matrix-vector spaces. In: Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 1201–1211 (2012)Google Scholar
  18. Rink, B., Harabagiu, S.: UTD: classifying semantic relations by combining lexical and semantic resources. In: Proceedings of the 5th International Workshop on Semantic Evaluation, pp. 256–259. Association for Computational Linguistics (2010)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.School of Computer and Information TechnologyBeijing Jiaotong UniversityBeijingChina

Personalised recommendations