Advertisement

Dynamic Prototype Selection by Fusing Attention Mechanism for Few-Shot Relation Classification

  • Linfang Wu
  • Hua-Ping ZhangEmail author
  • Yaofei Yang
  • Xin Liu
  • Kai GaoEmail author
Conference paper
  • 314 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12033)

Abstract

In a relation classification task, few-shot learning is an effective method when the number of training instances decreases. The prototypical network is a few-shot classification model that generates a point to represent each class, and this point is called a prototype. The mean is used to select prototypes for each class from a support set in a prototypical network. This method is fixed and static, and will lose some information at the sentence level. Therefore, we treat the mean selection as a special attention mechanism, then we expand the mean selection to dynamic prototype selection by fusing a self-attention mechanism. We also propose a query-attention mechanism to more accurately select prototypes. Experimental results on the FewRel dataset show that our model achieves significant and consistent improvements to baselines on few-shot relation classification.

Keywords

Relation classification Few-shot learning Attention mechanism 

Notes

Acknowledgement

This paper is sponsored by National Science Foundation of China (61772075) and National Science Foundation of Hebei Province (F2017208012).

References

  1. 1.
    Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv:1409.0473 [cs, stat], September 2014
  2. 2.
    Caruana, R.: Learning many related tasks at the same time with backpropagation. In: Tesauro, G., Touretzky, D.S., Leen, T.K. (eds.) Advances in Neural Information Processing Systems 7, pp. 657–664. MIT Press, Cambridge (1995)Google Scholar
  3. 3.
    Garcia, V., Bruna, J.: Few-shot learning with graph neural networks. arXiv:1711.04043 [cs, stat], November 2017
  4. 4.
    Gormley, M.R., Yu, M., Dredze, M.: Improved relation extraction with feature-rich compositional embedding models. arXiv:1505.02419 [cs], May 2015
  5. 5.
    Han, X., et al.: FewRel: a large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation. arXiv:1810.10147 [cs, stat], October 2018
  6. 6.
    Huang, Y.Y., Wang, W.Y.: Deep residual learning for weakly-supervised relation extraction. arXiv:1707.08866 [cs], July 2017
  7. 7.
    Ji, G., Liu, K., He, S., Zhao, J.: Distant supervision for relation extraction with sentence-level attention and entity descriptions. In: Thirty-First AAAI Conference on Artificial Intelligence, February 2017. https://www.aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14491
  8. 8.
    Mintz, M., Bills, S., Snow, R., Jurafsky, D.: Distant supervision for relation extraction without labeled data. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2 (ACL-IJCNLP 2009), vol. 2, p. 1003. Association for Computational Linguistics, Suntec, Singapore (2009). http://portal.acm.org/citation.cfm?doid=1690219.1690287
  9. 9.
    Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems 30, pp. 4077–4087. Curran Associates, Inc. (2017). http://papers.nips.cc/paper/6996-prototypical-networks-for-few-shot-learning.pdf
  10. 10.
    Vaswani, A., et al.: Attention is all you need. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems 30, pp. 5998–6008. Curran Associates, Inc. (2017). http://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf
  11. 11.
    Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, k., Wierstra, D.: Matching networks for one shot learning. In: Lee, D.D., Sugiyama, M., Luxburg, U.V., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems 29, pp. 3630–3638. Curran Associates, Inc. (2016). http://papers.nips.cc/paper/6385-matching-networks-for-one-shot-learning.pdf
  12. 12.
    Xie, R., Liu, Z., Jia, J., Luan, H., Sun, M.: Representation learning of knowledge graphs with entity descriptions. In: Thirtieth AAAI Conference on Artificial Intelligence, March 2016. https://www.aaai.org/ocs/index.php/AAAI/AAAI16/paper/view/12216

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Lab of NLPIR Big Data Search and MiningBeijing Institute of TechnologyBeijingChina
  2. 2.School of Information Science and EngineeringHebei University of Science and TechnologyShijiazhuangChina
  3. 3.Beijing Information Science and Technology UniversityBeijingChina
  4. 4.Beijing Institute of Information TechnologyBeijingChina

Personalised recommendations