Abstract
This chapter extends the basic MRF model by automatically learning query-dependent concept weights. The extension is a generic framework for learning the importance of query term concepts in a way that directly optimizes an underlying retrieval metric. By implementing concept weighting directly into the underlying retrieval model it avoids the issue of metric divergence. The chapter concludes with a rigorous experimental evaluation that demonstrates this weighting strategy is capable of yielding strong gains in retrieval effectiveness.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Available from the Linguistic Data Consortium catalog.
- 2.
Available as a part of Microsoft 2006 RFP dataset.
- 3.
Available at: http://download.wikimedia.org/enwiki/.
References
Bai, J., Chang, Y., Cui, H., Zheng, Z., Sun, G., & Li, X. (2008). Investigation of partial query proximity in web search. In Proc. 17th international conference on World Wide Web (pp. 1183–1184).
Bendersky, M., & Croft, W. B. (2008). Discovering key concepts in verbose queries. In Proc. 31st ann. intl. ACM SIGIR conf. on research and development in information retrieval.
Bendersky, M., Croft, W. B., & Smith, D. A. (2009). Two-stage query segmentation for information retrieval. In Proc. 32nd ann. intl. ACM SIGIR conf. on research and development in information retrieval.
Bendersky, M., Metzler, D., & Croft, W. B. (2010). Learning concept importance using a weighted dependence model. In Proceedings of the third ACM international conference on Web search and data mining (WSDM 2010) (pp. 31–40), New York.
Bergsma, S., & Wang, Q. I. (2007). Learning noun phrase query segmentation. In Proc. of EMNLP-CoNLL.
Cao, G., Nie, J.-Y., Gao, J., & Robertson, S. (2008). Selecting good expansion terms for pseudo-relevance feedback. In Proc. 31st ann. intl. ACM SIGIR conf. on research and development in information retrieval.
Cummins, R., & O’Riordan, C. (2009). Learning in a pairwise term-term proximity framework for information retrieval. In Proc. 32nd ann. intl. ACM SIGIR conf. on research and development in information retrieval.
Gey, F. (1994). Inferring probability of relevance using the method of logistic regression. In Proc. 17th ann. intl. ACM SIGIR conf. on research and development in information retrieval.
Guo, J., Xu, G., Li, H., & Cheng, X. (2008). A unified and discriminative model for query refinement. In Proc. of 31st ann. intl. ACM SIGIR conf. on research and development in information retrieval. New York: ACM.
Järvelin, K., & Kekäläinen, J. (2002). Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems, 20(4), 422–446.
Kumaran, G., & Carvalho, V. R. (2009). Reducing long queries using query quality predictors. In Proc. 32nd ann. intl. ACM SIGIR conf. on research and development in information retrieval.
Lavrenko, V., & Croft, W. B. (2001). Relevance-based language models. In Proc. 24th ann. intl. ACM SIGIR conf. on research and development in information retrieval (pp. 120–127).
Lease, M. (2009). An improved Markov random field model for supporting verbose queries. In Proc. 32nd ann. intl. ACM SIGIR conf. on research and development in information retrieval.
Lease, M., Croft, W. B., & Allan, J. (2009). Regression rank: Learning to meet the opportunity of descriptive queries. In Proc. 31st European conf. on information retrieval.
Liu, T.-Y. (2009). Learning to rank for information retrieval. Foundations and Trends in Information Retrieval, 3(3).
Metzler, D., & Croft, W. B. (2005). A Markov random field model for term dependencies. In Proc. 28th ann. intl. ACM SIGIR conf. on research and development in information retrieval (pp. 472–479).
Metzler, D., & Croft, W. B. (2007). Latent concept expansion using Markov random fields. In Proc. 30th ann. intl. ACM SIGIR conf. on research and development in information retrieval.
Mishne, G., & de Rijke, M. (2005). Boosting web retrieval through query operations. In Proc. 27th European conf. on information retrieval (pp. 502–516).
Morgan, W., Greiff, W., & Henderson, J. (2004). Direct maximization of average precision by hill-climbing with a comparison to a maximum entropy approach (Technical report). MITRE.
Pickens, J., & Croft, W. B. (1999). An exploratory analysis of phrases in text retrieval. In Proc. of RIAO 2000.
Ponte, J., & Croft, W. B. (1998). A language modeling approach to information retrieval. In Proc. 21st ann. intl. ACM SIGIR conf. on research and development in information retrieval (pp. 275–281).
Salton, G., & Buckley, C. (1988). Term-weighting approaches in automatic text retrieval. Information Processing & Management, 24(5), 513–523.
Tan, B., & Peng, F. (2008). Unsupervised query segmentation using generative language models and wikipedia. In Proc. 17th intl. World Wide Web conf. New York: ACM.
Tao, T. & Zhai, C. X. (2007). An exploration of proximity measures in information retrieval. In Proc. 30th ann. intl. ACM SIGIR conf. on research and development in information retrieval (pp. 295–302). New York: ACM.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Metzler, D. (2011). Query-Dependent Feature Weighting. In: A Feature-Centric View of Information Retrieval. The Information Retrieval Series, vol 27. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22898-8_5
Download citation
DOI: https://doi.org/10.1007/978-3-642-22898-8_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-22897-1
Online ISBN: 978-3-642-22898-8
eBook Packages: Computer ScienceComputer Science (R0)