An Overview of Evaluation Methods in TREC Ad Hoc Information Retrieval and TREC Question Answering
This chapter gives an overview of the current evaluation strategies and problems in the fields of information retrieval (IR) and question answering (QA), as instantiated in the Text Retrieval Conference (TREC).Whereas IR has a long tradition as a task, QA is a relatively new task which had to quickly develop its evaluation metrics, based on experiences gained in IR. This chapter will contrast the two tasks, their difficulties, and their evaluation metrics. We will end this chapter by pointing out limitations of the current evaluation strategies and potential future developments.
KeywordsInformation retrieval Question answering Extrinsic evaluation Intrinsic evaluation Precision Recall Accuracy MRR Composite evaluation metrics.
Unable to display preview. Download preview PDF.