Abstract
In this paper we present a learning algorithm to estimate a risk-sensitive and document-relation embedded ranking function so that the ranking score can reflect both the query-document relevance degree and the risk of estimating relevance when the document relation is considered. With proper assumptions, an analytic form of the ranking function is attainable with a ranking score being a linear combination among the expectation of relevance score, the variance of relevance estimation and the covariance with the other documents. We provide a systematic framework to study the roles of the relevance, the variance and the covariance in ranking documents and their relations with the different performance metrics. The experiments show that incorporating the variance in ranking score improves both the relevance and diversity.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Allan, J., Van Rijsbergen, C.J.: Probabilistic models of information retrieval based on measuring the divergence from randomness. ACM Trans. On Information Systems 20(4), 357–389 (2002)
Baeza-Yates, R., Ribeiro-Neto, B.: Modern information retrieval. Addison-Wesley Publisher, Reading (1999)
Carbonell, J., Goldstein, J.: The use of MMR, diversity-based reranking for reording documents and producing summaries. In: Proc. of SIGIR 1998 (1998)
Chen, H., Karger, D.R.: Less is more: probabilistic models for retrieving fewer relevant documents. In: Proc. of SIGIR 2006 (2006)
Jelinek, F., Mercer, R.: Interpolated estimation of markov source parameters from sparse data. Pattern Recognition in Practice, 381–402 (1980)
Lafferty, J.D., Zhai, C.: Document language models, query models and risk minimization for information retrieval. In: Proc. of SIGIR 2001 (2001)
Lavrenko, V., Croft, W.B.: Relevance-based language models. In: Proc. of SIGIR 2001 (2001)
Ponte, J.M., Croft, W.B.: A language modeling approach to information retrieval. In: Proc. of SIGIR 1998 (1998)
Robertson, S.E., Sparck Jones, K.: Relevance weighting of search terms. Journal of the American Society for Information Science 27(3), 129–146 (1976)
Robertson, S.E., Walker, S.: Some simple effective approximations to the 2-poisson models for probabilistic weighted retrieval. In: Proc. of SIGIR 1994 (1994)
Robertson, S.E.: The probability ranking principle in IR. Readings in information Retrieval, 281–286 (1997)
Robertson, S.E., Walker, S., Hancock-Beaulieu, M., Gatford, M., Payne, A.: Okapi at TREC-4. In: Proc. of Text Retrieval Conference, TREC (1995)
Wang, J., Zhu, J.H.: Portfolio theory of information retrieval. In: Proc. of SIGIR 2009 (2009)
Zaragoza, H., Hiemstra, D., Tipping, M., Robertson, S.E.: Bayesian extension to the language model for ad hoc information retrieval. In: Proc. of SIGIR 2003 (2003)
Zellner, A.: Bayesian estimation and prediction using asymmetric loss functions. Journal of the American Statistical Association 81(394), 446–451 (1986)
Zhai, C., Lafferty, J.D.: A risk minimization framework for information retrieval. Information Processing and Management 42(1), 31–55 (2006)
Zhai, C., Lafferty, J.D.: A study of smoothing methods for language models applied to information retrieval. ACM Trans. on Information Systems 22(2), 179–214 (2004)
Zhu, J.H., Wang, J., Cox, I., Taylor, M.: Risk business: modeling and exploiting uncertainty in information retrieval. In: Proc. of SIGIR 2009 (2009)
Zhai, C., Cohen, W., Lafferty, J.: Beyond independent relevance: methods and evaluation metrics for subtopic retrieval. In: Proc. of SIGIR 2003 (2003)
Gollapudi, S., Sharma, A.: An axiomatic approach for result diversification. In: Proc. of WWW 2009 (2009)
Radlinski, F., Kleinberg, R., Joachims, T.: Learning diverse rankings with multi-armed bandits. In: Proc. of ICML 2008 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gao, S., Li, H. (2011). Effective Large Scale Text Retrieval via Learning Risk-Minimization and Dependency-Embedded Model. In: Lee, KT., Tsai, WH., Liao, HY.M., Chen, T., Hsieh, JW., Tseng, CC. (eds) Advances in Multimedia Modeling. MMM 2011. Lecture Notes in Computer Science, vol 6524. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17829-0_10
Download citation
DOI: https://doi.org/10.1007/978-3-642-17829-0_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-17828-3
Online ISBN: 978-3-642-17829-0
eBook Packages: Computer ScienceComputer Science (R0)