Abstract
Recently neural network models are widely applied in text-matching tasks like community-based question answering (cQA). The strong generalization power of neural networks enables these methods to find texts with similar topics but miss detailed matching information. However, as proven by traditional methods, the explicit lexical matching knowledge is important for effective answer retrieval. In this paper, we propose an ExMaLSTM model to incorporate the explicit matching knowledge into the long short-term memory (LSTM) neural network. We extract explicit lexical matching features with prior knowledge and then add them to the local representations of questions. We summarize the overall matching status by using a bi-directional LSTM. The final relevance score is calculated using a gate network, which can dynamically assign appropriate weights to the explicit matching score and the implicit relevance score. We conduct extensive experiments for answer retrieval in a cQA dataset. The results show that our proposed ExMaLSTM model outperforms both the traditional methods and various state-of-the-art neural network models significantly.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kishky, A., Yanglei, S., Chi, Voss Clare, W., Jiawei, H.: Scalable topical phrase mining from text corpora. In: Proceedings of the VLDB Endowment, pp. 305–316 (2014)
Graves, A., Mohamed, A.R., Hinton, G.: Speech recognition with deep recurrent neural networks. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6645–6649 (2013)
Hu, B., Lu, Z., Li, H., Chen, Q.: Convolutional neural network architectures for matching natural language sentences. In: Advances in Neural Information Processing Systems (NIPS), pp. 2042–2050 (2014)
Wang, B., Liu, B., Wang, X., Sun, C., Zhang, D.: Deep learning approaches to semantic relevance modeling for chinese question-answer pairs. ACM Trans. Asian Lang. Inf. Process. (TALIP) 10 (2011)
Dyer, C., Ballesteros, M., Ling, W., Matthews, A., Smith, N.A.: Transition based dependency parsing with stack long short-term memory. In: Proceedings of ACL, pp. 334–343 (2015)
Wang, D., Nyberg, E.: A long short-term memory model for answer sentence selection in question answering. In: Proceedings of ACL, pp. 707–712 (2015)
Hu, H., Liu, B., Wang, B., Liu, M., Wang, X.: Multimodal DBN for predicting high-quality answers in CQA portals. In: Proceedings of ACL, pp. 843–847 (2013)
Palangi, H., Deng, L., Shen, Y., Gao, J., He, X., Chen, J., Song, X., Ward, R.: Deep sentence embedding using the long short term memory network: analysis and application to information retrieval. IEEE/ACM Trans. Audio Speech Lang. Process., 694–707 (2015)
He, X., Chen, J., Song, X., Ward, R.: Deep sentence embedding using the long short term memory network: analysis and application to information retrieval. IEEE/ACM Trans. Audio Speech Lang. Process., 694–707 (2015)
Zhang, H.-P., Yu, H.-K., Xiong, D.-Y., Liu, Q.: HHMM-based chinese lexical analyzer ICTCLAS. In: SIGHAN 2003 Proceedings of the Second SIGHAN Workshop on Chinese Language Processing, vol. 17, pp. 184–187 (2003)
Pang, L., Lan, Y., Guo, J., Xu, J., Wan, S., Cheng, X.: Text matching as image recognition. In: Proceedings of AAAI (2016)
Iyyer, M., Boyd-Graber, J., Claudino, L., Socher, R., Daumé III, H.: A neural network for factoid question answering over paragraphs. In: Proceedings of EMNLP, pp. 633–644 (2014)
Hochreiter, S., Schmidhuber, J.: Long short term memory. Neural Comput. 9(8), 1735–1780 (1997)
Wan, S., Lan, Y., Guo, J., Xu, J., Cheng, X.: A deep architecture for semantic matching with multiple positional sentence representations. In: Proceedings of AAAI (2016)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: Workshop at ICLR (2013)
Yin, W., Schütze, H.: MultiGranCNN: an architecture for general matching of text chunks on multiple levels of granularity. In: Proceedings of ACL, pp. 63–73 (2015)
Yih, W.-T., Chang, M.-W., Meek, C., Pastusiak, A.: Question answering using enhanced lexical semantic models. In: Proceedings of ACL, pp. 1744–1753 (2013)
Zhou, X., Hu, B., Chen, Q., Tang, B., Wang, X.: Answer sequence learning with neural networks for answer selection in community question answering. In: Proceedings of ACL, pp. 713–718 (2015)
Qiu, X., Huang, X.: Convolutional neural tensor network architecture for community based question answering. In: Proceedings of IJCAI, pp. C1305–C1311 (2015)
Lu, Z., Li, H.: A deep architecture for matching short texts. In: Advances in Neural Information Processing Systems (NIPS), pp. 1367–1375 (2013)
Bordes, A., Chopra, S., Weston, J.: Question answering with subgraph embeddings. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 615–620. Doha, Qatar (2014)
Acknowledgement
This work is supported by the National High Technology Research and Development Program of China (2015AA015403), the National Natural Science Foundation of China (61371129).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Bao, X., Wu, Y. (2017). Exploiting Explicit Matching Knowledge with Long Short-Term Memory. In: Sun, M., Wang, X., Chang, B., Xiong, D. (eds) Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data. NLP-NABD CCL 2017 2017. Lecture Notes in Computer Science(), vol 10565. Springer, Cham. https://doi.org/10.1007/978-3-319-69005-6_26
Download citation
DOI: https://doi.org/10.1007/978-3-319-69005-6_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-69004-9
Online ISBN: 978-3-319-69005-6
eBook Packages: Computer ScienceComputer Science (R0)