Constrained Deep Answer Sentence Selection

  • Ahmad AghaebrahimianEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10415)


In this paper, we propose Constrained Deep Neural Network (CDNN) a simple deep neural model for answer sentence selection. CDNN makes its predictions based on neural reasoning compound with some symbolic constraints. It integrates pattern matching technique into sentence vector learning. When trained using enough samples, CDNN outperforms regular models. We show how using other sources of training data as a mean of transfer learning can enhance the performance of the network. In a well-studied dataset for answer sentence selection, our network improves the state of the art in answer sentence selection significantly.


Deep neural network Sentence selection Transfer learning 



This research was partially funded by the Ministry of Education, Youth and Sports of the Czech Republic under SVV project number 260 453, core research funding, and GAUK 207-10/250098 of Charles University in Prague.


  1. 1.
    Aghaebrahimian, A., Jurčíček, F.: Constraint-based open-domain question answering using knowledge graph search. In: Sojka, P., Horák, A., Kopeček, I., Pala, K. (eds.) TSD 2016. LNCS (LNAI), vol. 9924, pp. 28–36. Springer, Cham (2016). doi: 10.1007/978-3-319-45510-5_4 Google Scholar
  2. 2.
    Aghaebrahimian, A., Jurčíček, F.: Open-domain factoid question answering via knowledge graph search. In: Proceedings of the Workshop on Human-Computer Question Answering, The North American Chapter of the Association for Computational Linguistics (NAACL) (2016)Google Scholar
  3. 3.
    Bordes, A., Usunier, N., Chopra, S., Weston, J.: Large-scale simple question answering with memory networks. arXiv preprint arXiv:1506.02075 (2015)
  4. 4.
    Clarke, J., Goldwasser, D., Chang, M.W., Roth, D.: Driving semantic parsing from the worlds response. In: Proceedings of the Conference on Computational Natural Language Learning (2010)Google Scholar
  5. 5.
    Feng, M., Xiang, B., Glass, M.R., Wang, L., Zhou, B.: Applying deep learning to answer selection: a study and an open task. In: Proceedings of IEEE ASRU Workshop (2015)Google Scholar
  6. 6.
    Fern, S., Stevenson, M.: A semantic similarity approach to paraphrase detection. In: Proceedings of the 11th Annual Research Colloquium of the UK Special-Interest Group for Computational Linguistics (2008)Google Scholar
  7. 7.
    He, H., Gimpel, K., Lin, J.: Multi-perspective sentence similarity modeling with convolutional neural networks. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP) (2015)Google Scholar
  8. 8.
    He, H., Lin, J.: Pairwise word interaction modeling with deep neural networks for semantic similarity measurement. In: The North American Chapter of the Association for Computational Linguistics (NAACL) (2016)Google Scholar
  9. 9.
    Hermann, K.M., Kocisky, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., Blunsom, P.: Teaching machines to read and comprehend. In: Advances in Neural Information Processing Systems (2015)Google Scholar
  10. 10.
    Kadlec, R., Vodolan, M., Libovicky, J., Macek, J., Kleindienst, J.: Knowledge-based dialog state tracking. In: 2014 IEEE Spoken Language Technology Workshop (SLT) (2014)Google Scholar
  11. 11.
    Khashabi, D., Khot, T., Sabharwal, A., Clark, P., Etzioni, O., Roth, D.: Question answering via integer programming over semi-structured knowledge. In: Proceedings of International Joint Conference on Artificial Intelligence (IJCAI) (2016)Google Scholar
  12. 12.
    Kingma, D., Ba, J.: Adam: a method for stochastic optimization. arXiv:1412.6980 (2014)
  13. 13.
    Kwiatkowski, T., Zettlemoyer, L., Goldwater, S., Steedman., M.: Inducing probabilistic CCG grammars from logical form with higher-order unification. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (2010)Google Scholar
  14. 14.
    Madnani, N., Tetreault, J., Chodorow, M.: Re-examining machine translation metrics for paraphrase identification. In: Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (2012)Google Scholar
  15. 15.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: Proceedings of Workshop at ICLR (2013)Google Scholar
  16. 16.
    Min, S., Seo, M., Hajishirzi, H.: Question answering through transfer learning from large fine-grained supervision data. arXiv:1702.02171 (2017)
  17. 17.
    Pennington, J., Socher, R., Manning, C.D.: Glove: Global vectors for word representation. In: Proceedings of the Empirical Methods in Natural Language Processing (2014)Google Scholar
  18. 18.
    Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 (2016)
  19. 19.
    Rao, J., He, H., Lin, J.: Noise-contrastive estimation for answer selection with deep neural networks. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, CIKM (2016)Google Scholar
  20. 20.
    Santos, C.D., Tan, M., Xiang, B., Zhou, B.: Attentive pooling networks. arXiv:1602.03609 (2016)
  21. 21.
    Tellex, S., Katz, B., Lin, J., Fernandes, A., Marton, G.: Quantitative evaluation of passage retrieval algorithms for question answering. In: SIGIR (2003)Google Scholar
  22. 22.
    Xu, W., Ritter, A., Callison-Burch, C., Dolan, W.B., Ji, Y.: Extracting lexically divergent paraphrases from twitter. Trans. Assoc. Comput. Linguist. 2, 435–448 (2014)Google Scholar
  23. 23.
    Yang, Y., Yih, W.T., Meek, C.: WikiQA: a challenge dataset for open-domain question answering. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP) (2015)Google Scholar
  24. 24.
    Yao, X., Durme, B.V.: Information extraction over structured data: question answering with freebase. In: Proceedings of Association for Computational Linguistics (2014)Google Scholar
  25. 25.
    Yao, X., Van Durme, B., Callison-Burch, C., Clark, P.: Answer extraction as sequence tagging with tree edit distance. In: HLT-NAACL (2013)Google Scholar
  26. 26.
    Yih, W.T., Chang, M.W., Meek, C., Pastusiak, A.: Question answering using enhanced lexical semantic models. In: Proceedings of Association for Computational Linguistics (ACL) (2013)Google Scholar
  27. 27.
    Yin, W., Schtze, H., Xiang, B., Zhou, B.: ABCNN: attention-based convolutional neural network for modeling sentence pairs. arXiv:1512.05193 (2015)
  28. 28.
    Yu, L., Moritz Hermann, K., Blunsom, P., Pulman, S.: Deep learning for answer sentence selection. In: NIPS Deep Learning Workshop (2014)Google Scholar
  29. 29.
    Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). doi: 10.1007/978-3-319-10590-1_53 Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Faculty of Mathematics and Physics, Institute of Formal and Applied LinguisticsCharles University in PraguePraha 1Czech Republic

Personalised recommendations