Skip to main content
SpringerLink
Log in

Exploiting visual saliency for increasing diversity of image retrieval results

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Diversification of search results allows for better and faster search, gaining knowledge about different perspectives and viewpoints on retrieved information sources. Recently various methods for diversification of image retrieval results have been proposed, mainly using textual information or techniques imported from the natural language processing domain. However, images contain much more information than their textual descriptions and the use of visual features deserves special attention in this context. Visual saliency provides information about parts of the image perceived as most important, which are instinctively targeted by humans when shooting a photo or looking at a picture. For this reason we propose to exploit such information to improve diversification of search results. To this purpose, we introduce a saliency-based method to re-rank the results of a query and we show that it can achieve significantly better performances as compared to the baseline approach. Experimental validation conducted on a number of queries applied to various datasets demonstrates the potential of the use of saliency information for the diversification of image retrieval results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. http://www.imageclef.org/

  2. http://www.multimediaeval.org/

  3. http://coewww.rutgers.edu/riul/research/code/EDISON/

References

  1. Carbonell J, Goldstein J (1998) The use of mmr, diversity-based reranking for reordering documents and producing summaries. In: ACM SIGIR Conference on Research and Development in Information Retrieval

  2. Clarke C, Craswell N, Soboroff I, Ashkan A (2011) A comparative analysis of cascade measures for novelty and diversity. In: ACM International Conference on Web search and data mining

  3. Dang-Nguyen DT, Piras L, Giacinto G, Boato G (2014) Retrieval of diverse images by pre-filtering and hierarchical clustering. In: MediaEval Benchmarking Initiative for Multimedia Evaluation

  4. Deselaers T, Gass T, Dreuw P, Ney H (2009) Jointly optimising relevance and diversity in image retrieval. In: ACM International Conference on Image and Video Retrieval

  5. Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A (2008) The PASCAL Visual Object Classes Challenge 2008 (VOC2008) Results, http://www.pascal-network.org/challenges/VOC/voc2008/workshop/index.html

  6. Gelasca E, Tomasic D, Ebrahimi T (2005) Which colors best catch your eyes: a subjective study of color saliency. In: International Workshop on Video Processing and Quality Metrics for Consumer Electronics

  7. Griffin G, Holub A, Perona P (2007) Caltech-256 object category dataset. Tech. Rep. 7694, California Institute of Technology, http://authors.library.caltech.edu/7694

  8. Hoiem D, Efros A, Hebert M (2005) Geometric context from a single image. IEEE International Conference of Computer Vision

  9. Huang J, Yang X, Fang X, Lin W, Zhang R (2011) Integrating visual saliency and consistency for re-ranking image search results. IEEE Trans Multimed 13(4):653–661

    Article  Google Scholar 

  10. Ionescu B, Popescu A, Lupu M, Ginsca A L, Mueller H (2014) Retrieving diverse social images at mediaeval 2014: Challenge, dataset and evaluation. In: MediaEval Benchmarking Initiative for Multimedia Evaluation

  11. Kennedy LS, Naaman M (2008) Generating diverse and representative image serach results for landmarks. In: ACM International Conference on World Wide Web

  12. Koehler K, Guo F, Zhang S, Eckstein M P (2014) What do saliency models predict. J Vis 14(3): 1–27

    Article  Google Scholar 

  13. van Leuken RH, Garcia L, Olivares X, van Zwol R (2009) Visual diversification of image search results. In: ACM International Conference on World Wide Web

  14. Leung T, Malik J (1999) Recognizing surfaces using three-dimensional textons. In: International Conference on Computer Vision, pp 1010–1017

  15. Li H, Tang J, Li G, Chua TS (2008) Word2image: Towards visual interpreting of words. In: ACM International Conference on Multimedia

  16. Li J, Ji Z, Zhang J, Su YT (2012) Generating diverse and relevant image searching results with divrank. In: International Conference on Machine Learning and Cybernetics

  17. Liu T, Yuan Z, Sun J, Wang J, Zheng N, Tang X, Shum H Y (2011) Learning to detect a salient object. IEEE Trans Pattern Anal Mach Intell 33(2):353–367

    Article  Google Scholar 

  18. Muratov O, Zontone P, Boato G, De Natale FGB (2011) A segment-based image saliency detection. In: IEEE International Conference on Acoustics, Speech and Signal Processing

  19. Paramita ML, Sanderson M, Clough P (2009) Diversity in photo retrieval: Overview of the imageclefphoto task 2009. In: International Conference on Cross-language evaluation forum: multimedia experiments

  20. Ramanathan S, Katti H, Sebe N, Kankanhalli M, Chua TS (2010) An eye fixation database for saliency detection in images. In: European Conference on Computer Vision

  21. Ramanathan S, Yanulevskaya V, Sebe N (2011) Can computers learn from humans to see better?: Inferring scene semantics from viewers’ eye movements. In: ACM International Conference on Multimedia, pp 33–42

  22. Riche N, Duvinage M, Mancas M, Gosselin B, Dutoit T (2013) Saliency and human fixations: State-of-the-art and study of comparison metrics. In: IEEE International Conference on Computer Vision

  23. Rudinac S, Hanjalic A, Larson M (2013) Generating visual summaries of geographic areas using community-contributed images. IEEE Trans Multimed 15(4):921–932

    Article  Google Scholar 

  24. Sanderson M, Tang J, Arni T, Clough P (2009) What else is there? search diversity examined. In: European Conference on Information Retrieval

  25. Shi R, Liu Z, Du H, Zhang X, Shen L (2012) Region diversity maximization for salient object detection. IEEE Signal Proc Lett 19(4):215–218

    Article  Google Scholar 

  26. Song K, Tian Y, Gao W, Huang T (2006) Diversifying the image retrieval results. In: ACM International Conference on Multimedia

  27. Tollari S, Mulhem P, Ferecatu M, Glotin H, Detyniecki M, Gallinari P, Sahbi H, Zhao ZQ (2009) A comparative study of diversity methods for hybrid text and image retrieval approaches. In: Workshop of the Cross-Language Evaluation Forum

  28. Wang M, Yang K, Hua XS, Zhang HJ (2010) Towards a relevant and diverse search of social images. IEEE Trans Multimed 12(8):829–842

    Article  Google Scholar 

  29. Yu Z, Lu L, Guo Y, Fan R, Liu M, Wang W (2014) Content-aware photo collage using circle packing. IEEE Trans Vis Comput Graph 20(2):182–195

    Article  Google Scholar 

  30. Zontone P, Boato G, De Natale FGB, De Rosa A, Barni M, Piva A, Hare JS, Dupplaw D, Lewis PH (2009) Image diversity analysis: context, opinion and bias. In: Living Web Workshop

  31. Zontone P, Boato G, Hare J, Lewis P, Siersdorfer S, Minack E (2010) Image and collateral text in support of auto-annotation and sentiment analysis. In: Workshop on Graph-based Methods for Natural Language Processing, pp 88–92

  32. Zwol RV, Murdock V, Pueyo LG, Ramirez G (2008) G.: Diversifying image search with user generated content. In: ACM International Conference on Multimedia Information Retrieval

Download references

Acknowledgments

This work was supported by the Deanship of Scientific Research of the King Saud University through the International Research Group under Project IRG14-20.

Author information

Authors and Affiliations

  1. Department of Information Engineering and Computer Science, University of Trento, via Sommarive 9, Trento, Italy

    Giulia Boato, Duc-Tien Dang-Nguyen, Oleg Muratov & Francesco G. B. De Natale

  2. College of Computer and Information Sciences, King Saud University, Riyadh, 11543, Saudi Arabia

    Naif Alajlan

Authors
  1. Giulia Boato
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Duc-Tien Dang-Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Oleg Muratov
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Naif Alajlan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Francesco G. B. De Natale
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Giulia Boato.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Boato, G., Dang-Nguyen, DT., Muratov, O. et al. Exploiting visual saliency for increasing diversity of image retrieval results. Multimed Tools Appl 75, 5581–5602 (2016). https://doi.org/10.1007/s11042-015-2526-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-2526-4

Keywords

Search

Navigation