Skip to main content
SpringerLink
Log in
Book cover

Encyclopedia of Database Systems pp 1338–1348Cite as

Evaluation Metrics for Structured Text Retrieval

  • Reference work entry
  • First Online:

Synonyms

Evaluation of XML retrieval effectiveness; Performance metrics

Definition

Anevaluation metric is used to evaluate the effectiveness of information retrieval systems and to justify theoretical and/or pragmatic developments of these systems. It consists of a set of measures that follow a common underlying evaluation methodology.

There are many metrics that can be used to evaluate the effectiveness of structured text retrieval systems. These metrics are based on different evaluation assumptions, incorporate different hypotheses of the expected user behavior, and implement their own evaluation methodologies to handle the level of overlap among the units of retrieval.

Historical Background

Over the past 5 years, the initiative for the evaluation of XML retrieval (INEX) has investigated various aspects of structured text retrieval, by particularly focusing on XML retrieval. Major advances, both in terms of approaches to XML retrieval and evaluation of XML retrieval, have been made...

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   4,499.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   6,499.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Recommended Reading

  1. Gövert N, Fuhr N, Lalmas M, Kazai G. Evaluating the effectiveness of content-oriented XML retrieval methods. Inf Retr. 2006;9(6):699–722.

    Article  Google Scholar 

  2. Järvelin K, Kekäläinen J. Cumulated gain-based evaluation of IR techniques. ACM Trans Inf Syst. 2002;20(4):422–46.

    Article  Google Scholar 

  3. Kazai G. Choosing an ideal recall-base for the evaluation of the focused task: sensitivity analysis of the XCG evaluation measures. In: Proceedings of the Comparative Evaluation of XML Information Retrieval Systems: Fifth Workshop of the Initiative for the Evaluation of XML Retrieval; 2007. p. 35–44.

    Google Scholar 

  4. Kazai G, Lalmas M. Notes on what to measure in INEX. In: Proceedings of the INEX 2005 Workshop on Element Retrieval Methodology; 2005. p. 22–38.

    Google Scholar 

  5. Kazai G, Lalmas M. eXtended Cumulated Gain measures for the evaluation of content-oriented XML retrieval. ACM Trans Inf Syst. 2006;24(4):503–42.

    Article  Google Scholar 

  6. Kazai G, Lalmas M, de Vries AP. The overlap problem in content-oriented XML retrieval evaluation. In: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval; 2004. p. 72–9.

    Google Scholar 

  7. Pehcevski J. Evaluation of effective XML information retrieval. RMIT University, Melbourne, Australia, Ph.D. thesis, 2006.

    Google Scholar 

  8. Pehcevski J, Thom JA. HiXEval: highlighting XML retrieval evaluation. In: Advances in XML Information Retrieval and Evaluation: Proceedings of the Fourth Workshop of the Initiative for the Evaluation of XML Retrieval; 2006. p. 43–57.

    Google Scholar 

  9. Piwowarski B, Dupret G. Evaluation in (XML) information retrieval: expected precision-recall with user modelling (EPRUM). In: Proceedings of the 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval; 2006. p. 260–7.

    Google Scholar 

  10. Piwowarski B, Gallinari P. Expected ratio of relevant units: a measure for structured information retrieval. In: Proceedings of the 2nd International Workshop of the Initiative for the Evaluation of XML Retrieval; 2003. p. 158–66.

    Google Scholar 

  11. Piwowarski B, Gallinari P, Dupret G. Precision recall with user modelling (PRUM): application to structured information retrieval. ACM Trans Inf Syst. 2007;25(1):1–37.

    Article  Google Scholar 

  12. Raghavan V, Bollmann P, Jung G. A critical investigation of recall and precision. ACM Trans Inf Syst. 1989;7(3):205–29.

    Article  Google Scholar 

  13. de Vries A, Kazai G, Lalmas M. Tolerance to irrelevance: a user-effort evaluation of retrieval systems without predefined retrieval unit. In: Proceedings of the 7th International Conference on Computer-Assisted Information Retrieval; 2004. p. 463–73.

    Google Scholar 

  14. Woodley A. and Geva S. XCG overlap at INEX 2004. In: Proceedings of the 4th International Workshop of the Initiative for the Evaluation of XML Retrieval; 2005. p. 25–39.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. INRIA Paris-Rocquencourt, Le Chesnay Cedex, France

    Jovan Pehcevski

  2. University of Glasgow, Glasgow, UK

    Benjamin Piwowarski

Authors
  1. Jovan Pehcevski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Benjamin Piwowarski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Benjamin Piwowarski .

Editor information

Editors and Affiliations

  1. Georgia Institute of Technology College of Computing, Atlanta, GA, USA

    Ling Liu

  2. University of Waterloo School of Computer Science, Waterloo, ON, Canada

    M. Tamer Özsu

Section Editor information

  1. University of Amsterdam, Amsterdam, The Netherlands

    Jaap Kamps

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Science+Business Media, LLC, part of Springer Nature

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Pehcevski, J., Piwowarski, B. (2018). Evaluation Metrics for Structured Text Retrieval. In: Liu, L., Özsu, M.T. (eds) Encyclopedia of Database Systems. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-8265-9_152

Download citation

Publish with us

Policies and ethics

Search

Navigation