Abstract
The difficulty of a user query can affect the performance of Information Retrieval (IR) systems. This work presents a formal model for quantifying and reasoning about query difficulty as follows: Query difficulty is considered to be a subjective belief, which is formulated on the basis of various types of evidence. This allows us to define a belief model and a set of operators for combining evidence of query difficulty. The belief model uses subjective logic, a type of probabilistic logic for modeling uncertainties. An application of this model with semantic and pragmatic evidence about 150 TREC queries illustrates the potential flexibility of this framework in expressing and combining evidence. To our knowledge, this is the first application of subjective logic to IR.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
van Rijsbergen, C.J.: A non-classical logic for information retrieval. Comput. J. 29(6), 481–485 (1986)
van Rijsbergen, C.J.: The Geometry of Information Retrieval. CUP, Cambridge (2004)
van Rijsbergen, C.J., Crestani, F., Lalmas, M.: Information Retrieval: Uncertainty and Logics. Springer, Heidelberg (1998)
Carmel, D., Yom-Tov, E., Darlow, A., Pelleg, D.: What makes a query difficult? In: SIGIR, pp. 390–397 (2006)
Chiaramella, Y., Chevallet, J.-P.: About retrieval models and logic. Comput. J. 35(3), 233–242 (1992)
Cronen-Townsend, S., Zhou, Y., Croft, W.B.: Predicting query performance. In: SIGIR, pp. 299–306 (2002)
Dempster, A.P.: A generalization of Bayesian inference. Journal of the Royal Statistical Society B(30), 205–247 (1968)
Fishburn, P.C.: The axioms of subjective probability. Statistical Science 3(1), 335–345 (1986)
Hauff, C., Azzopardi, L., Hiemstra, D.: The combination and evaluation of query performance prediction methods. In: ECIR, pp. 301–312 (2009)
He, B., Ounis, I.: Inferring query performance using pre-retrieval predictors. In: Apostolico, A., Melucci, M. (eds.) SPIRE 2004. LNCS, vol. 3246, pp. 43–54. Springer, Heidelberg (2004)
Josang, A.: A logic for uncertain probabilities. Int. J. Uncertain. Fuzziness Knowl.-Based Syst. 9(3), 279–311 (2001)
Lalmas, M.: Information retrieval and Dempster-Shafer’s theory of evidence. In: Hunter, A., Parsons, S. (eds.) Applications of Uncertainty Formalisms. LNCS (LNAI), vol. 1455, pp. 157–176. Springer, Heidelberg (1998)
Lau, R.Y.K., Bruza, P.D., Song, D.: Towards a belief-revision-based adaptive and context-sensitive information retrieval system. ACM Trans. Inf. Syst. 26(2) (2008)
Logan, B., Reece, S., Sparck Jones, K.: Modelling information retrieval agents with belief revision. In: SIGIR, pp. 91–100 (1994)
Losada, D.E., Barreiro, A.: A logical model for information retrieval based on propositional logic and belief revision. Comput. J. 44(5), 410–424 (2001)
Mothe, J., Tanguy, L.: Linguistic features to predict query difficulty - a case study on previous TREC campaigns. In: SIGIR Workshop on Predicting Query Difficulty: Methods and Applications (2005)
Nie, J.-Y.: Towards a probabilistic modal logic for semantic-based information retrieval. In: SIGIR, pp. 140–151 (1992)
Plachouras, V., Ounis, I.: Dempster-Shafer theory for a query-biased combination of evidence on the web. Inf. Retr. 8(2), 197–218 (2005)
Shafer, G.: A Mathematical Theory of Evidence. Princeton University Press, Princeton (1976)
Shi, L., Nie, J.-Y., Cao, G.: Relating dependent indexes using Dempster-Shafer theory. In: CIKM, pp. 429–438 (2008)
Smets, P.: What is Dempster-Shafer’s model? Wiley, Chichester (1994)
Tomlinson, S.: Robust, web, and terabyte retrieval with Hummingbird SearchServer at TREC 2004. In: TREC (2004)
Tsikrika, T., Lalmas, M.: Combining evidence for web retrieval using the inference network model: an experimental study. Inf. Process. Manage. 40(5), 751–772 (2004)
Voorhees, E.M., Harman, D.K.: TREC: Experiment and Evaluation in Information Retrieval. MIT Press, Cambridge (2005)
Yom-Tov, E., Fine, S., Carmel, D., Darlow, A.: Learning to estimate query difficulty: including applications to missing content detection and distributed information retrieval. In: SIGIR, pp. 512–519 (2005)
Zhou, Y., Croft, W.B.: Ranking robustness: a novel framework to predict query performance. In: CIKM, pp. 567–574 (2006)
Zhou, Y., Croft, W.B.: Query performance prediction in web search environments. In: SIGIR, pp. 543–550 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lioma, C., Blanco, R., Mochales Palau, R., Moens, MF. (2009). A Belief Model of Query Difficulty That Uses Subjective Logic . In: Azzopardi, L., et al. Advances in Information Retrieval Theory. ICTIR 2009. Lecture Notes in Computer Science, vol 5766. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04417-5_9
Download citation
DOI: https://doi.org/10.1007/978-3-642-04417-5_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04416-8
Online ISBN: 978-3-642-04417-5
eBook Packages: Computer ScienceComputer Science (R0)