Abstract
In this paper we address the issue of optimizing the actual social photo retrieval technology in terms of users’ requirements. Typical users are interested in taking possession of accurately relevant-to-the-query and non-redundant images so they can build a correct exhaustive perception over the query. We propose to tackle this issue by combining two approaches previously considered non-overlapping: machine image analysis for a pre-filtering of the initial query results followed by crowd-sourcing for a final refinement. In this mechanism, the machine part plays the role of reducing the time and resource consumption allowing better crowd-sourcing results. The machine technique ensures representativeness in images by performing a re-ranking of all images according to the most common image in the initial noisy set; additionally, diversity is ensured by clustering the images and selecting the best ranked images among the most representative in each cluster. Further, the crowd-sourcing part enforces both representativeness and diversity in images, objectives that are, to a certain extent, out of reach by solely the automated machine technique. The mechanism was validated on more than 25,000 photos retrieved from several common social media platforms, proving the efficiency of this approach.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Rae, A., Kelm, P.: Working Notes for the Placing Task at MediaEval. In: MediaEval 2012 Workshop, Pisa, Italy, October 4-5. CEUR-WS.org (2012) ISSN 1613-0073
Jordan, C., Watters, C.: Extending the Rocchio Relevance Feedback Algorithm to Provide Contextual Retrieval. In: Favela, J., Menasalvas, E., Chávez, E. (eds.) AWIC 2004. LNCS (LNAI), vol. 3034, pp. 135–144. Springer, Heidelberg (2004)
Elsas, J.L., Donmez, P., Callan, J., Carbonell, J.G.: Pairwise Document Classification for Relevance Feedback. In: TREC 2009 (2009)
Quinn, A.J., Bederson, B.B.: Human computation: a survey and taxonomy of a growing field. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2011), pp. 1403–1412. ACM, New York (2011)
Soleymani, M., Larson, M.: Crowd-sourcing for affective annotation of video: Development of a viewer-reported boredom corpus. In: SIGIR Workshop on Crowd-sourcing for Search Evaluation (2010)
Taneva, B., Kacimi, M., Weikum, G.: Gathering and ranking photos of named entities with high precision, high recall, and diversity. In: ACM on Web Search and Data Mining, USA, pp. 431–440 (2010)
Zhu, X., Goldberg, A., Gael, J.V., Andrzejewski, D.: Improving Diversity in Ranking using Absorbing Random Walks, pp. 97–104 (2007)
Deselaers, T., Gass, T., Dreuw, P., Ney, H.: Jointly optimising relevance and diversity in image retrieval. In: ACM on Image and Video Retrieval, USA, pp. 39:1–39:8 (2009)
Huang, Z., Hu, B., Cheng, H., Shen, H., Liu, H., Zhou, X.: Mining near-duplicate graph for cluster-based reranking of web video search results. ACM Trans. Inf. Syst. 28, 22:1–22:27 (2010)
Radu, A.-L., Stöttinger, J., Ionescu, B., Menéndez, M., Giunchiglia, F.: Representativeness and Diversity in Photos via Crowd-Sourced Media Analysis. In: 10th International Workshop on Adaptive Multimedia Retrieval, AMR 2012, Copenhagen, Denmark, October 24-25 (2012)
Nowak, S., Rüger, S.: How reliable are annotations via crowd-sourcing? a study about inter-annotator agreement for multi-label image annotation. In: Int. Conf. on Multimedia Information Retrieval (2010)
Kittur, A., Chi, E.H., Suh, B.: Crowd-sourcing user studies with Mechanical Turk. In: SIGCHI Conf. on Human Factors in Computing Systems, Italy, pp. 453–456 (2008)
Crandall, D.J., Backstrom, L., Huttenlocher, D., Kleinberg, J.: Mapping the world’s photos. In: Int. Conf. on World Wide Web, pp. 761–770 (2009)
Eickhoff, C., de Vries, A.P.: Increasing Cheat Robustness of Crowd-sourcing Tasks (2012)
Rudinac, S., Hanjalic, A., Larson, M.: Finding representative and diverse community contributed images to create visual summaries of geographic areas. In: ACM Int. Conf. on Multimedia, pp. 1109–1112 (2011)
Van De Weijer, J., Schmid, C.: Applying Color Names to Image Description. In: IEEE Int. Conf. on Image Processing, USA, p. 493 (2007)
Kennedy, L.S., Naaman, M.: Generating diverse and representative image search results for landmarks. In: Int. Conf. on World Wide Web, China, pp. 297–306 (2008)
Randolph, J.J., Bednarik, R., Myller, N.: Author Note: Free-Marginal Multirater Kappa (multirater kfree): An Alternative to Fleiss’ Fixed - Marginal Multirater Kappa
Noble, J.A.: Minority voices of crowd-sourcing: why we should pay attention to every member of the crowd. In: ACM Conf. on Computer Supported Cooperative Work Companion, USA, pp. 179–182 (2012)
Li, J., Ma, Q., Asano, Y., Yoshikawa, M.: Re-ranking by multi-modal relevance feedback for content-based social image retrieval. In: Sheng, Q.Z., Wang, G., Jensen, C.S., Xu, G. (eds.) APWeb 2012. LNCS, vol. 7235, pp. 399–410. Springer, Heidelberg (2012)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Radu, AL., Ionescu, B., Menéndez, M., Stöttinger, J., Giunchiglia, F., De Angeli, A. (2014). A Hybrid Machine-Crowd Approach to Photo Retrieval Result Diversification. In: Gurrin, C., Hopfgartner, F., Hurst, W., Johansen, H., Lee, H., O’Connor, N. (eds) MultiMedia Modeling. MMM 2014. Lecture Notes in Computer Science, vol 8325. Springer, Cham. https://doi.org/10.1007/978-3-319-04114-8_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-04114-8_3
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-04113-1
Online ISBN: 978-3-319-04114-8
eBook Packages: Computer ScienceComputer Science (R0)