Standard Effectiveness Measures

Synonyms

Evaluation in information retrieval

Definition

The standard effectiveness evaluation in information retrieval centers around determining how relevant are the documents retrieved to users’ information needs. A user query represents the users’ information need. A user information need is hidden, depending on what the user already knows, what the user wants to find out about, and what is the users constraint (time, format, price, recent, location). Recall and precision are the basis of relevance-based effectiveness measures. Other commonly used evaluation measures based on recall and precision include the F-measure, the 11-point precision-recall curve, and the average precision.

Historical Background

The first formal evaluation of information retrieval (IR) systems was conducted in the late 1950s by Cyrid Cleverdon at the College of Aeronautics in Cranfield, England [2]. These studies are often referred to as the Cranfield experiments and are the foundation of future research on...