Advertisement

Exploiting Explicit Matching Knowledge with Long Short-Term Memory

  • Xinqi Bao
  • Yunfang WuEmail author
Conference paper
  • 1.4k Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10565)

Abstract

Recently neural network models are widely applied in text-matching tasks like community-based question answering (cQA). The strong generalization power of neural networks enables these methods to find texts with similar topics but miss detailed matching information. However, as proven by traditional methods, the explicit lexical matching knowledge is important for effective answer retrieval. In this paper, we propose an ExMaLSTM model to incorporate the explicit matching knowledge into the long short-term memory (LSTM) neural network. We extract explicit lexical matching features with prior knowledge and then add them to the local representations of questions. We summarize the overall matching status by using a bi-directional LSTM. The final relevance score is calculated using a gate network, which can dynamically assign appropriate weights to the explicit matching score and the implicit relevance score. We conduct extensive experiments for answer retrieval in a cQA dataset. The results show that our proposed ExMaLSTM model outperforms both the traditional methods and various state-of-the-art neural network models significantly.

Keywords

Lexical matching knowledge LSTM Question answering 

Notes

Acknowledgement

This work is supported by the National High Technology Research and Development Program of China (2015AA015403), the National Natural Science Foundation of China (61371129).

References

  1. Kishky, A., Yanglei, S., Chi, Voss Clare, W., Jiawei, H.: Scalable topical phrase mining from text corpora. In: Proceedings of the VLDB Endowment, pp. 305–316 (2014)Google Scholar
  2. Graves, A., Mohamed, A.R., Hinton, G.: Speech recognition with deep recurrent neural networks. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6645–6649 (2013)Google Scholar
  3. Hu, B., Lu, Z., Li, H., Chen, Q.: Convolutional neural network architectures for matching natural language sentences. In: Advances in Neural Information Processing Systems (NIPS), pp. 2042–2050 (2014)Google Scholar
  4. Wang, B., Liu, B., Wang, X., Sun, C., Zhang, D.: Deep learning approaches to semantic relevance modeling for chinese question-answer pairs. ACM Trans. Asian Lang. Inf. Process. (TALIP) 10 (2011)Google Scholar
  5. Dyer, C., Ballesteros, M., Ling, W., Matthews, A., Smith, N.A.: Transition based dependency parsing with stack long short-term memory. In: Proceedings of ACL, pp. 334–343 (2015)Google Scholar
  6. Wang, D., Nyberg, E.: A long short-term memory model for answer sentence selection in question answering. In: Proceedings of ACL, pp. 707–712 (2015)Google Scholar
  7. Hu, H., Liu, B., Wang, B., Liu, M., Wang, X.: Multimodal DBN for predicting high-quality answers in CQA portals. In: Proceedings of ACL, pp. 843–847 (2013)Google Scholar
  8. Palangi, H., Deng, L., Shen, Y., Gao, J., He, X., Chen, J., Song, X., Ward, R.: Deep sentence embedding using the long short term memory network: analysis and application to information retrieval. IEEE/ACM Trans. Audio Speech Lang. Process., 694–707 (2015)Google Scholar
  9. He, X., Chen, J., Song, X., Ward, R.: Deep sentence embedding using the long short term memory network: analysis and application to information retrieval. IEEE/ACM Trans. Audio Speech Lang. Process., 694–707 (2015)Google Scholar
  10. Zhang, H.-P., Yu, H.-K., Xiong, D.-Y., Liu, Q.: HHMM-based chinese lexical analyzer ICTCLAS. In: SIGHAN 2003 Proceedings of the Second SIGHAN Workshop on Chinese Language Processing, vol. 17, pp. 184–187 (2003)Google Scholar
  11. Pang, L., Lan, Y., Guo, J., Xu, J., Wan, S., Cheng, X.: Text matching as image recognition. In: Proceedings of AAAI (2016)Google Scholar
  12. Iyyer, M., Boyd-Graber, J., Claudino, L., Socher, R., Daumé III, H.: A neural network for factoid question answering over paragraphs. In: Proceedings of EMNLP, pp. 633–644 (2014)Google Scholar
  13. Hochreiter, S., Schmidhuber, J.: Long short term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  14. Wan, S., Lan, Y., Guo, J., Xu, J., Cheng, X.: A deep architecture for semantic matching with multiple positional sentence representations. In: Proceedings of AAAI (2016)Google Scholar
  15. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: Workshop at ICLR (2013)Google Scholar
  16. Yin, W., Schütze, H.: MultiGranCNN: an architecture for general matching of text chunks on multiple levels of granularity. In: Proceedings of ACL, pp. 63–73 (2015)Google Scholar
  17. Yih, W.-T., Chang, M.-W., Meek, C., Pastusiak, A.: Question answering using enhanced lexical semantic models. In: Proceedings of ACL, pp. 1744–1753 (2013)Google Scholar
  18. Zhou, X., Hu, B., Chen, Q., Tang, B., Wang, X.: Answer sequence learning with neural networks for answer selection in community question answering. In: Proceedings of ACL, pp. 713–718 (2015)Google Scholar
  19. Qiu, X., Huang, X.: Convolutional neural tensor network architecture for community based question answering. In: Proceedings of IJCAI, pp. C1305–C1311 (2015)Google Scholar
  20. Lu, Z., Li, H.: A deep architecture for matching short texts. In: Advances in Neural Information Processing Systems (NIPS), pp. 1367–1375 (2013)Google Scholar
  21. Bordes, A., Chopra, S., Weston, J.: Question answering with subgraph embeddings. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 615–620. Doha, Qatar (2014) Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Key Laboratory of Computational Linguistics (Peking University), School of Electronic Engineering and Computer SciencePeking UniversityBeijingChina

Personalised recommendations