Abstract
Spatial diversity is a relatively new branch of research in the context of spatial information retrieval. Although the assumption that spatially diversified results may meet users’ needs better seems reasonable, there has been little hard evidence in the literature indicating so. In this paper, we will show the potentials of spatial diversity by not only the traditional evaluation metrics (precision and cluster recall), but also through a user preference study using Amazon Mechanical Turk. The encouraging results from the latter prove that users do have strong preference on spatially diversified results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alonso, O., Rose, D.E., Stewart, B.: Crowdsourcing for relevance evaluation. SIGIR Forum 42(2), 9–15 (2008)
Arni, T., Tang, J., Sanderson, M., Clough, P.: Creating a test collection to evaluate diversity in image retrieval. In: Proceedings of the Workshop on Beyond Binary Relevance: Preferences, Diversity, and Set-Level Judgments, held at SIGIR (2008)
Carbonell, J.G., Goldstein, J.: The use of mmr, diversity-based reranking for reordering documents and producing summaries. In: SIGIR 1998, pp. 335–336. ACM Press, Melbourne (1998)
Clarke, C.L.A., Kolla, M., Cormack, G.V., Vechtomova, O., Ashkan, A., Büttcher, S., MacKinnon, I.: Novelty and diversity in information retrieval evaluation. In: SIGIR 2008: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, pp. 659–666. ACM Press, Singapore (2008)
Clough, P., Joho, H., Purves, R.: Judging the spatial relevance of documents for GIR. In: Lalmas, M., MacFarlane, A., Rüger, S.M., Tombros, A., Tsikrika, T., Yavlinsky, A. (eds.) ECIR 2006. LNCS, vol. 3936, pp. 548–552. Springer, Heidelberg (2006)
Dakka, W., Ipeirotis, P.G.: Automatic extraction of useful facet hierarchies from text databases. In: IEEE 24th International Conference on Data Engineering (ICDE 2008), April 2008, pp. 466–475. IEEE, Los Alamitos (2008)
Deselaers, T., Gass, T., Dreuw, P., Ney, H.: Jointly optimising relevance and diversity in image retrieval. In: ACM International Conference on Image and Video Retrieval 2009 (CIVR 2009), Santorini, Greece, 08/07/2009, ACM (2009)
Grubinger, M., Clough, P., Mller, H., Deselaers, T.: The iapr tc-12 benchmark: A new evaluation resource for visual information systems. In: International Workshop OntoImage 2006 Language Resources for Content-Based Image Retrieval held in conjunction with LREC 2006, Genoa, Italy, pp. 13–23 (2006)
Paramita, M.L., Tang, J., Sanderson, M.: Generic and spatial approaches to image search results diversification. In: 31st European Conference on Information Retrieval (ECIR), pp. 603–610 (2009)
Robertson, S.: On gmap: and other transformations. In: Conference on Information and Knowledge Management, Virginia, USA, pp. 78–83 (2006)
Sorokin, A., Forsyth, D.: Utility data annotation with amazon mechanical turk. In: Proceedings of the First IEEE Workshop on Internet Vision at CVPR 2008, pp. 1–8 (2008)
Tang, J., Sanderson, M.: The tripod project technical report - modeling diversity in spatial information retrieval using spatial coverage. Technical report, University of Sheffield (2009)
van Kreveld, M., Reinbacher, I., Arampatzis, A., van Zwol, R.: Distributed ranking methods for geographic information retrieval. In: Proceedings of the 20th European Workshop on Computational Geometry, pp. 231–243 (2004)
Zhai, C.X., Cohen, W.W., Lafferty, J.: Beyond independent relevance: Methods and evaluation metrics for subtopic retrieval. In: SIGIR 2006, Toronto, Canada, pp. 10–17 (2003)
Zhang, B., Li, H., Liu, Y., Ji, L., Xi, W., Fan, W., Chen, Z., Ma, W.-Y.: Improving web search results using affinity graph. In: SIGIR 2005, pp. 504–511. ACM, New York (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tang, J., Sanderson, M. (2010). Evaluation and User Preference Study on Spatial Diversity. In: Gurrin, C., et al. Advances in Information Retrieval. ECIR 2010. Lecture Notes in Computer Science, vol 5993. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12275-0_18
Download citation
DOI: https://doi.org/10.1007/978-3-642-12275-0_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12274-3
Online ISBN: 978-3-642-12275-0
eBook Packages: Computer ScienceComputer Science (R0)