Synonyms
Relevance evaluation of IR systems
Definition
In information retrieval (IR), effectiveness is defined as the relevance of retrieved information to a given query. System effectiveness evaluation typically focuses on the problem of document retrieval: retrieving a ranked list of documents for each input query. Effectiveness is then measured with respect to an environment of interest consisting of the populations of documents, queries, and relevance judgments defining which of these documents are relevant to which queries. Sampling methodologies are employed for each of these populations to estimate a relevance metric, typically a function of precision (the ratio of relevant documents retrieved to the total number of documents retrieved) and recall (the ratio of relevant documents retrieved to the total number of relevant documents for that query). Conclusions about which systems outperform others are drawn from common experimental design, typically focusing on a random sample of...
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Recommended Reading
Buckley C, Voorhees EM. Evaluating evaluation measure stability. In: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval; 2000.p. 33–40.
Clarke CLA, Craswell N, Soboroff I. Overview of the TREC 2004 Terabyte track. In: Proceedings of the 15th Text Retrieval Conference; 2006.
Cleverdon CW, Mills J, Keen EM. Factors determining the performance of indexing systems. vol. 1: Cranfield CERES: Aslib Cranfield Research Project, College of Aeronautics, Cranfield. vol. 1: Design, vol. 2: Results, 1996.
Hawking D, Craswell N. Overview of the TREC 2001 web track. In: Proceedings of the 10th Text Retrieval Conference; 2001.
Jensen EC, Beitzel SM, Chowdhury A, Frieder O. On repeatable evaluation of search services in dynamic environments. ACM Trans Inf Syst. 2007;26(1):1.
Sanderson M, Zobel J. Information retrieval system evaluation: effort, sensitivity, and reliability. In: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval; 2005. p. 162–9.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Section Editor information
Rights and permissions
Copyright information
© 2018 Springer Science+Business Media, LLC, part of Springer Nature
About this entry
Cite this entry
Jensen, E.C., Beitzel, S.M., Frieder, O. (2018). Effectiveness Involving Multiple Queries. In: Liu, L., Özsu, M.T. (eds) Encyclopedia of Database Systems. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-8265-9_477
Download citation
DOI: https://doi.org/10.1007/978-1-4614-8265-9_477
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-8266-6
Online ISBN: 978-1-4614-8265-9
eBook Packages: Computer ScienceReference Module Computer Science and Engineering