Content Based Image Retrieval Based on Modelling Human Visual Attention

  • Alex Papushoy
  • Adrian G. Bors
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9256)


In this paper we propose to employ human visual attention models for content based image retrieval. This approach is called query by saliency content retrieval (QSCR) and considers visual saliency at both local and global image levels. Each image, from a given database, is segmented and specific features are evaluated locally for each of its regions. The global saliency is evaluated based on edge distribution and orientation. During the retrieval stage, the most similar images are retrieved by using an optimization approach such as the Earth Moving Distance (EMD) algorithm. The proposed method ranks the similarity between the query image and a set of given images based on their similarity in the features associated with the salient regions.


Content based image retrieval Human visual attention models Earth moving distance Local and global saliency 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Achanta, R., Hemami, S., Estrada, F., Susstrunk, S.: Frequency-tuned salient region detection. In: Proc. IEEE Int. Conf. on Computer Vision and Pattern Recognition, pp. 1597–1604 (2009)Google Scholar
  2. 2.
    Chen, Y., Wang, J. Z.: Image categorization by learning and reasoning with images. J. of Machine Learning Research, 913–939 (2004)Google Scholar
  3. 3.
    Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. on Pattern Analysis and Machine Intelligence 24(5), 603–619 (2002)CrossRefGoogle Scholar
  4. 4.
    Datta, R., Joshi, D., Li, J., Wan, J.Z.: Image retrieval: Ideas, influences, and trends of the new age. ACM Computing Surveys 40(2), 5:1–5:60 (2008)CrossRefGoogle Scholar
  5. 5.
    Feng, S., Xu, D., Yang, X.: Attention-driven salient edge(s) and region(s) extraction with application to CBIR. Signal Processing 90(1), 1–15 (2010)CrossRefzbMATHGoogle Scholar
  6. 6.
    Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Proc. Advances in Neural Information Processing Systems (NIPS), vol. 19, pp. 545–552 (2007)Google Scholar
  7. 7.
    Itti, L., Koch, C.: Computational modelling of visual attention. Nature Reviews Neuroscience 2(3), 194–203 (2001)CrossRefGoogle Scholar
  8. 8.
    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(11), 1254–1259 (1998)CrossRefGoogle Scholar
  9. 9.
    Itti, L., Ullman, S.: Shifts in selective visual attention: Towards the underlying neural circuitry. Human Neurobiology 4(4), 219–227 (1985)Google Scholar
  10. 10.
    Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. on Syst., Man and Cyber. 9(1), 62–66 (1979)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Papushoy, A., Bors, A.G.: Image retrieval based on query by saliency content. Digital Signal Processing 36, 156–173 (2015)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Rahmani, R., Goldman, S.A., Zhang, H., Cholleti, S.R., Fritts, J.E.: Localized content-based image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(11), 1902–1912 (2008)CrossRefGoogle Scholar
  13. 13.
    Rubner, Y., Tomasi, C., Guibas, L.J.: The earth mover distance as a metric for image retrieval. Int. Journal of Computer Vision 40(2), 99–121 (2000)CrossRefzbMATHGoogle Scholar
  14. 14.
    Rui, Y., Huang, T., Ortega, M., Mehrotra, S.: Relevance feedback: A power tool in interactive content-based image retrieval. IEEE Trans. on Circuits and Systems for Video Technology 8(5), 644–655 (1998)CrossRefGoogle Scholar
  15. 15.
    Smeulders, A., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Trans. on Pattern Analysis and Machine Intelligence 22(12), 1349–1380 (2000)CrossRefGoogle Scholar
  16. 16.
    Wang, J., Li, J., Wiederhold, G.: Simplicity: Semantics-sensitive integrated matching for picture libraries. IEEE Trans. on Pattern Analysis and Machine Intelligence 23(9), 947–963 (2001)CrossRefGoogle Scholar
  17. 17.
    Zhang, L., Tong, M.H., Marks, T.K., Shan, H., Cottrell, G.W.: SUN: A Bayesian framework for saliency using natural statistics. Jour. of Vision 8(7), 32.1–32.20 (2008)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Alex Papushoy
    • 1
  • Adrian G. Bors
    • 1
  1. 1.Department of Computer ScienceUniversity of YorkYorkUK

Personalised recommendations