Standard Effectiveness Measures
Evaluation in information retrieval
The standard effectiveness evaluation in information retrieval centers around determining how relevant are the documents retrieved to users’ information needs. A user query represents the users’ information need. A user information need is hidden, depending on what the user already knows, what the user wants to find out about, and what is the users constraint (time, format, price, recent, location). Recall and precision are the basis of relevance-based effectiveness measures. Other commonly used evaluation measures based on recall and precision include the F-measure, the 11-point precision-recall curve, and the average precision.
The first formal evaluation of information retrieval (IR) systems was conducted in the late 1950s by Cyrid Cleverdon at the College of Aeronautics in Cranfield, England . These studies are often referred to as the Cranfield experiments and are the foundation of future research on...
- 1.Baeza-Yates R, Ribeiro-Neto B. Modern information retrieval. New York: Addison Wesley; 1999.Google Scholar
- 2.Cleverdon CW. The significance of the cranfield tests on index languages. In: Proceedings of the 14th Annual International Conference on Research and Development in Information Retrieval; 1991. p. 3–12.Google Scholar
- 3.Voorhees EM, Harman DK, editors. TREC: experiment and evaluation in information retrieval. Cambridge, MA: MIT; 2005.Google Scholar