An Overview of Evaluation Methods in TREC Ad Hoc Information Retrieval and TREC Question Answering

  • Simone Teufel
Part of the Text, Speech and Language Technology book series (TLTB, volume 37)

This chapter gives an overview of the current evaluation strategies and problems in the fields of information retrieval (IR) and question answering (QA), as instantiated in the Text Retrieval Conference (TREC).Whereas IR has a long tradition as a task, QA is a relatively new task which had to quickly develop its evaluation metrics, based on experiences gained in IR. This chapter will contrast the two tasks, their difficulties, and their evaluation metrics. We will end this chapter by pointing out limitations of the current evaluation strategies and potential future developments.

Keywords

Information retrieval Question answering Extrinsic evaluation Intrinsic evaluation Precision Recall Accuracy MRR Composite evaluation metrics. 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer 2007

Authors and Affiliations

  • Simone Teufel
    • 1
  1. 1.Computer LaboratoryUniversity of CambridgeUK

Personalised recommendations