Abstract
Recognizing textual entailment is a fundamental task in a variety of text mining or natural language processing applications. This paper proposes a simple neural model for RTE problem. It first matches each word in the hypothesis with its most-similar word in the premise, producing an augmented representation of the hypothesis conditioned on the premise as a sequence of word pairs. The LSTM model is then used to model this augmented sequence, and the final output from the LSTM is fed into a softmax layer to make the prediction. Besides the base model, in order to enhance its performance, we also proposed three techniques: the integration of multiple word-embedding library, bi-way integration, and ensemble based on model averaging. Experimental results on the SNLI dataset have shown that the three techniques are effective in boosting the predicative accuracy and that our method outperforms several state-of-the-state ones.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aharon, R.B., Szpektor, I., Dagan, I.: Generating entailment rules from FrameNet. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (Short Papers), pp. 241–246. Association for Computational Linguistics (2010)
Baker, C.F., Fillmore, C.J., Lowe, J.B.: The Berkeley FrameNet project. In: Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, vol. 1, pp. 86–90. Association for Computational Linguistics (1998)
Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5(2), 157–166 (1994)
Bowman, S.R., Angeli, G., Potts, C., Manning, C.D.: A large annotated corpus for learning natural language inference. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 632–642 (2015)
Bowman, S.R., Gauthier, J., Rastogi, A., Gupta, R., Manning, C.D., Potts, C.: A fast unified model for parsing and sentence understanding. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pp. 1466–1477 (2016)
Cheng, J., Dong, L., Lapata, M.: Long short-term memory-networks for machine reading. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 551–561 (2016)
Dagan, I., Dolan, B., Magnini, B., Roth, D.: Recognizing textual entailment: rational, evaluation and approaches. Nat. Lang. Eng. 15(4), i–xvii (2009)
Dagan, I., Glickman, O.: Probabilistic textual entailment: generic applied modeling of language variability. In: Learning Methods for Text Understanding and Mining, pp. 26–29 (2004)
Fellbaum, C. (ed.): Wordnet: An Electronic Lexical Database. MIT Press, Cambridge (1998)
Giampiccolo, D., Magnini, B., Dagan, I., Dolan, B.: The third pascal recognizing textual entailment challenge. In: Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pp. 1–9. Association for Computational Linguistics (2007)
Glickman, O., Dagan, I.: Acquiring lexical paraphrases from a single corpus. In: Recent Advances in Natural Language Processing III, pp. 81–90. John Benjamins Publishing, Amsterdam (2004)
Harris, Z.S.: Distributional structure. Word 10(2–3), 146–162 (1954)
Harabagiu, S., Hickl, A.: Methods for using textual entailment in open-domain question answering. In: Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pp. 905–912 (2006)
Hochreiter, S.: Untersuchungen zu dynamischen neuronalen netzen. Technische Universität München, Diploma (1991)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Landauer, T.K., Dumais, S.T.: A solution to Platos problem: the latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychol. Rev. 104(2), 211–240 (1997)
Lin, D., Pantel, P.: DIRT - Discovery of inference rules from text. In: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 323–328 (2001)
Lloret, E., Ferrández, O., Munoz, R., Palomar, M.: A text summarization approach under the influence of textual entailment. In: Proceedings of the 5th International Workshop on Natural Language Processing and Cognitive Science, pp. 22–31 (2008)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems (NIPS), pp. 3111–3119 (2013)
Mou, L., Men, R., Li, G., Xu, Y., Zhang, L., Yan, R., Jin, Z.: Natural language inference by tree-based convolution and heuristic matching. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Short Papers), pp. 130–136 (2016)
Pekar, V.: Acquisition of verb entailment from text. In: Proceedings of the Main Conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, pp. 49–56. Association for Computational Linguistics (2006)
Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods on Natural Language Processing, pp. 1532–1543 (2014)
Rocktäschel, T., Grefenstette, E., Hermann, K.M., Kočiský, T., Blunsom, P.: Reasoning about entailment with neural attention. ArXiv Preprint arXiv:1509.06664 (2016)
Sekine, S.: Automatic paraphrase discovery based on context and keywords between NE pairs. In: Proceedings of International Workshop on Paraphrase (IWP 2005), pp. 4–6 (2005)
Srivastava, N.: Improving neural networks with dropout. Ph.D. Thesis, University of Toronto (2013)
Szpektor, I., Dagan, I.: Augmenting wordnet-based inference with argument mapping. In: Proceedings of the 2009 Workshop on Applied Textual Inference, pp. 27–35. Association for Computational Linguistics (2009)
Szpektor, I., Tanev, H., Dagan, I., Coppola, B., Kouylekov, M.: Unsupervised acquisition of entailment relations from the web. Nat. Lang. Eng. 21(01), 3–47 (2015)
Wang, S., Jiang, J.: Learning natural language inference with LSTM. In: Proceedings of the 15th Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp. 700–704 (2016)
Zaremba, W., Sutskever, I., Vinyals, O.: Recurrent neural network regularization. ArXiv Preprint arXiv:1409.2329 (2014)
Acknowledgments
This work is supported by National High-Tech R&D Program of China (863 Program) (No. 2015AA015404), and Science and Technology Commission of Shanghai Municipality (No. 14511106802). We are grateful to the anonymous reviewers for their valuable comments.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Xie, Z., Hu, J. (2017). Max-Cosine Matching Based Neural Models for Recognizing Textual Entailment. In: Candan, S., Chen, L., Pedersen, T., Chang, L., Hua, W. (eds) Database Systems for Advanced Applications. DASFAA 2017. Lecture Notes in Computer Science(), vol 10177. Springer, Cham. https://doi.org/10.1007/978-3-319-55753-3_19
Download citation
DOI: https://doi.org/10.1007/978-3-319-55753-3_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-55752-6
Online ISBN: 978-3-319-55753-3
eBook Packages: Computer ScienceComputer Science (R0)