Abstract
In many questions in Community Question Answering sites users look for the advice or opinion of other users who might offer diverse perspectives on a topic at hand. The novel task we address is providing supportive evidence for human answers to such questions, which will potentially help the asker in choosing answers that fit her needs. We present a support retrieval model that ranks sentences from Wikipedia by their presumed support for a human answer. The model outperforms a state-of-the-art textual entailment system designed to infer factual claims from texts. An important aspect of the model is the integration of relevance oriented and support oriented features.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
Smoothing is performed using the term statistics in the document corpus \(D\).
- 3.
- 4.
- 5.
IMDB snapshot from 08/01/2014.
- 6.
The order of concatenation has no effect since unigram language models are used.
- 7.
- 8.
Available at http://iew3.technion.ac.il/~kurland/supportRanking.
- 9.
- 10.
- 11.
The implementations of LinearSVM and PolySVM are from http://www.cs.cornell.edu/people/tj/svm_light/svm_rank.html. The LambdaMART implementation is from http://sourceforge.net/p/lemur/wiki/RankLib/. All methods are used with default free-parameter values of the corresponding implementations.
- 12.
- 13.
- 14.
We trained P1EDA using the SNLI data set [15], which contains 549,366 examples.
- 15.
Integrating P1EDA in PolySVM did not yield support-ranking improvements.
- 16.
For relevance ranking, LambdaMART was trained for binary relevance.
- 17.
Actual numbers are omitted due to space considerations and as they convey no additional insight.
- 18.
Ablation tests reveal that removing this feature results in the second most substantial decrease of support-ranking performance among all features.
References
Rieke, R., Sillars, M.: Argumentation and Critical Decision Making. Longman Series in Rhetoric and Society, Longman (1997)
Murdock, V.: Exploring Sentence Retrieval. VDM Verlag, Saarbrücken (2008)
Metzler, D., Croft, W.B.: A markov random field model for term dependencies. In: Proceedings of SIGIR, pp. 472–479 (2005)
Zhai, C., Lafferty, J.D.: A study of smoothing methods for language models applied to ad hoc information retrieval. In: Proceedings of SIGIR, pp. 334–342 (2001)
Liu, T.Y.: Learning to rank for information retrieval. Found. Trends Inf. Retrieval 3(3), 225–331 (2009)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Proceedings of NIPS, pp. 3111–3119 (2013)
Vulic, I., Moens, M.: Monolingual and cross-lingual information retrieval models based on (bilingual) word embeddings. In: Proceedings of SIGIR, pp. 363–372 (2015)
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C., Ng, A., Potts, C.: Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of EMNLP, pp. 363–372 (2013)
Pang, B., Lee, L.: Opinion mining and sentiment analysis. Found. Trends Inf. Retrieval 2(1–2), 1–135 (2007)
Bendersky, M., Croft, W.B., Diao, Y.: Quality-biased ranking of web documents. In: Proceedings of WSDM, pp. 95–104 (2011)
Joachims, T.: Training linear SVMs in linear time. In: Proceedings of KDD, pp. 217–226 (2006)
Burges, C.J.: From RankNet to LambdaRank to LambdaMART: an overview. Technical report, Microsoft Research (2010)
Zhou, Y., Croft, B.: Query performance prediction in web search environments. In: Proceedings of SIGIR, pp. 543–550 (2007)
Noh, T.G., Pado, S., Shwartz, V., Dagan, I., Nastase, V., Eichler, K., Kotlerman, L., Adler, M.: Multi-level alignments as an extensible representation basis for textual entailment algorithms. In: Proceedings of SEM (2015)
Bowman, S.R., Angeli, G., Potts, C., Manning, C.D.: A large annotated corpus for learning natural language inference. In: Proceedings of EMNLP, pp. 632–642 (2015)
Boltužić, F., Šnajder, J.: Back up your stance: recognizing arguments in online discussions. In: Proceedings of the First Workshop on Argumentation Mining, Association for Computational Linguistics, pp. 49–58 (2014)
Dagan, I., Roth, D., Sammons, M., Zanzotto, F.M.: Recognizing textual entailment: models and applications. Synth. Lect. Hum. Lang. Technol. 6(4), 1–220 (2013)
Blanco, R., Zaragoza, H.: Finding support sentences for entities. In: Proceedings of SIGIR, pp. 339–346 (2010)
Kim, H.D., Castellanos, M., Hsu, M., Zhai, C., Dayal, U., Ghosh, R.: Ranking explanatory sentences for opinion summarization. In: Proceedings of SIGIR, pp. 1069–1072 (2013)
Cabrio, E., Villata, S.: Combining textual entailment and argumentation theory for supporting online debates interactions. In: Proceedings of ACL, pp. 208–212 (2012)
Green, N., Ashley, K., Litman, D., Reed, C., Walker, V. (eds.) Proceedings of the First Workshop on Argumentation Mining, Baltimore, Maryland. Association for Computational Linguistics (2014). http://www.aclweb.org/anthology/W/W14/W14-21
Sato, M., Yanai, K., Yanase, T., Miyoshi, T., Iwayama, M., Sun, Q., Niwa, Y.: End-to-end argument generation system in debating. In: Proceedings of ACL-IJCNLP 2015 System Demonstrations (2015)
Cardie, C. (ed.): Proceedings of the 2nd Workshop on Argumentation Mining, Denver, CO. Association for Computational Linguistics (2015). http://www.aclweb.org/anthology/W15-05
Rinott, R., Dankin, L., Alzate Perez, C., Khapra, M.M., Aharoni, E., Slonim, N.: Show me your evidence - an automatic method for context dependent evidence detection. In: Proceedings of EMNLP, pp. 440–450 (2015)
Brill, E., Lin, J.J., Banko, M., Dumais, S.T., Ng, A.Y., et al.: Data-intensive question answering. In: Proceedings of TREC, vol. 56, p. 90 (2001)
Acknowledgments
We thank the reviewers for their helpful comments, and Omer Levy and Vered Shwartz for their help with the textual entailment tool used for experiments. This work was supported in part by a Yahoo! faculty research and engagement award.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Braunstain, L., Kurland, O., Carmel, D., Szpektor, I., Shtok, A. (2016). Supporting Human Answers for Advice-Seeking Questions in CQA Sites. In: Ferro, N., et al. Advances in Information Retrieval. ECIR 2016. Lecture Notes in Computer Science(), vol 9626. Springer, Cham. https://doi.org/10.1007/978-3-319-30671-1_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-30671-1_10
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-30670-4
Online ISBN: 978-3-319-30671-1
eBook Packages: Computer ScienceComputer Science (R0)