A Hybrid Embedding Approach to Noisy Answer Passage Retrieval

  • Daniel CohenEmail author
  • W. Bruce Croft
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10772)


Answer passage retrieval is an increasingly important information retrieval task as queries become more precise and mobile and audio interfaces more prevalent. In this task, the goal is to retrieve a contiguous series of sentences (a passage) that concisely addresses the information need expressed in the query. Recent work with deep learning has shown the efficacy of distributed text representations for retrieving sentences or tokens for question answering. However, determining the relevancy of answer passages remains a significant challenge, specifically when there exists a lexical and semantic gap between the text representation used for training and the collection’s vocabulary. In this paper, we demonstrate the flexibility of a character based approach on the task of answer passage retrieval, agnostic to the source of embeddings and with improved performance in P@1 and MRR metrics over a word based approach as the collections degrade in quality.


Answer passage Representation learning Hybrid embedding 



This work was supported in part by the Center for Intelligent Information Retrieval, in part by NSF IIS-1160894 and in part by NSF grant #IIS-1419693. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect those of the sponsor.


  1. 1.
    Callan, J.P.: Passage-level evidence in document retrieval. In: ACM SIGIR 1994, pp. 302–310. Springer, New York (1994).
  2. 2.
    Cohen, D., Ai, Q., Croft, W.B.: Adaptability of neural networks on varying granularity IR tasks. In: SIGIR Neu-IR Workshop, Pisa, Italy (2016)Google Scholar
  3. 3.
    Cohen, D., Croft, W.B.: End to end long short term memory networks for non-factoid question answering. In: ICTIR, Newark, DE, USA (2016)Google Scholar
  4. 4.
    Diaz, F., Mitra, B., Craswell, N.: Query expansion with locally-trained word embeddings. In: ACL, pp. 367–377. ACL (2016)Google Scholar
  5. 5.
    dos Santos, C.N., Tan, M., Xiang, B., Zhou, B.: Attentive pooling networks. CoRR, abs/1602.03609 (2016)Google Scholar
  6. 6.
    Graves, A., Jaitly, N., Mohamed, A.-R.: Hybrid speech recognition with deep bidirectional LSTM. In: ASRU, 2013, pp. 273–278 (2013)Google Scholar
  7. 7.
    Guo, J., Fan, Y., Ai, Q., Croft, W.B.: A deep relevance matching model for ad-hoc retrieval. In: CIKM 2016, pp. 55–64. ACM, New York (2016)Google Scholar
  8. 8.
    Harding, S.M., Croft, W.B., Weir, C.: Probabilistic retrieval of OCR degraded text using N-grams. In: Peters, C., Thanos, C. (eds.) ECDL 1997. LNCS, vol. 1324, pp. 345–359. Springer, Heidelberg (1997). CrossRefGoogle Scholar
  9. 9.
    He, X., Golub, D.: Character-level question answering with attention. In: EMNLP, pp. 1598–1607. ACL (2016)Google Scholar
  10. 10.
    Huang, P.-S., He, X., Gao, J., Deng, L., Acero, A., Heck, L.: Learning deep structured semantic models for web search using clickthrough data. In: CIKM 2013, pp. 2333–2338. ACM (2013)Google Scholar
  11. 11.
    Keikha, M., Park, J., Croft, W.B.: Evaluating answer passages using summarization measures. In: SIGIR 2014, pp. 963–966 (2014)Google Scholar
  12. 12.
    Kim, Y., Jernite, Y., Sontag, D., Rush, A.M.: Character-aware neural language models. In: AAAI, pp. 2741–2749 (2016)Google Scholar
  13. 13.
    Meng, L., Li, Y., Liu, M., Shu, P.: Skipping word: a character-sequential representation based framework for question answering. In: CIKM 2016, pp. 1869–1872 (2016)Google Scholar
  14. 14.
    Miao, Y., Yu, L., Blunsom, P.: Neural variational inference for text processing. In: ICML, pp. 1727–1736 (2016).
  15. 15.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: ICLR Workshop, Scottsdale, AZ, USA (2013)Google Scholar
  16. 16.
    Seo, M.J., Kembhavi, A., Farhadi, A., Hajishirzi, H.: Bidirectional attention flow for machine comprehension. In: ICLR, Toulon, France (2017)Google Scholar
  17. 17.
    Severyn, A., Moschitti, A.: Learning to rank short text pairs with convolutional deep neural networks. In: SIGIR 2015, pp. 373–382. ACM, New York (2015)Google Scholar
  18. 18.
    Surdeanu, M., Ciaramita, M., Zaragoza, H.: Learning to rank answers on large online QA collections. In ACL:HLT, pp. 719–727 (2008)Google Scholar
  19. 19.
    Tan, M., Xiang, B., Zhou, B.: LSTM-based deep learning models for non-factoid answer selection. CoRR, abs/1511.04108 (2015)Google Scholar
  20. 20.
    Tielman, T., Hinton, G.: Lecture 6.5-rmsprop: Divide the Gradient by a Running Average of its Recent Magnitude (2012)Google Scholar
  21. 21.
    Wang, D., Nyberg, E.: A long short-term memory model for answer sentence selection in question answering. In: ACL-IJCNLP, ACL 2015 (Volume 2: Short Papers), pp. 707–712, July 26–31, 2015, Beijing, China (2015)Google Scholar
  22. 22.
    Wei, K., Iyer, R., Bilmes, J.: Submodularity in data subset selection and active learning. In: ICML, Lille, France (2015)Google Scholar
  23. 23.
    Yang, L., Ai, Q., Spina, D., Chen, R.-C., Pang, L., Croft, W.B., Guo, J., Scholer, F.: Beyond factoid QA: effective methods for non-factoid answer sentence retrieval. In: Ferro, N., Crestani, F., Moens, M.-F., Mothe, J., Silvestri, F., Di Nunzio, G.M., Hauff, C., Silvello, G. (eds.) ECIR 2016. LNCS, vol. 9626, pp. 115–128. Springer, Cham (2016). CrossRefGoogle Scholar
  24. 24.
    Yin, W., Schütze, H., Xiang, B., Zhou, B.: ABCNN: attention-based convolutional neural network for modeling sentence pairs. In: TACL, vol. 4, pp. 259–272 (2016)Google Scholar
  25. 25.
    Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. In: NIPS, NIPS 2015, pp. 649–657, Montreal, Canada (2015)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.College of Information and Computer SciencesUniversity of Massachusetts AmherstAmherstUSA

Personalised recommendations