Definition
Bpref is a preference-based information retrieval measure that considers whether relevant documents are ranked above irrelevant ones. It is designed to be robust to missing relevance judgments, such that it gives the same experimental outcome with incomplete judgments that Mean Average Precision would with complete judgments.
Key Points
In a test collection where all relevant documents have been identified, experiments using bpref and MAP should give the same outcome, for example both systems should agree that system A is better than system B. However, if the relevance judgments are incomplete, for example where only half the pool has been judged, MAP becomes unstable and may incorrectly show that system B is better than system A. The bpref measure was developed to maintain the correct ordering of systems (A better than B) even with incomplete judgments.
Given a ranked list of search results and a set of R known relevant documents and N known irrelevant documents, bpref...
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Recommended Reading
Buckley C, Voorhees EM. Retrieval evaluation with incomplete information. In: Proceedings of 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval; 2004. p. 25–32.
Yilmaz E, Aslam JA. Estimating average precision with incomplete and imperfect judgments. In: Proceedings of International Conference on Information and Knowledge Management; 2006. p. 102–11.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Section Editor information
Rights and permissions
Copyright information
© 2018 Springer Science+Business Media, LLC, part of Springer Nature
About this entry
Cite this entry
Craswell, N. (2018). Bpref. In: Liu, L., Özsu, M.T. (eds) Encyclopedia of Database Systems. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-8265-9_489
Download citation
DOI: https://doi.org/10.1007/978-1-4614-8265-9_489
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-8266-6
Online ISBN: 978-1-4614-8265-9
eBook Packages: Computer ScienceReference Module Computer Science and Engineering