Salient Object Detection: A Benchmark

  • Ali Borji
  • Dicky N. Sihite
  • Laurent Itti
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7573)


Several salient object detection approaches have been published which have been assessed using different evaluation scores and datasets resulting in discrepancy in model comparison. This calls for a methodological framework to compare existing models and evaluate their pros and cons. We analyze benchmark datasets and scoring techniques and, for the first time, provide a quantitative comparison of 35 state-of-the-art saliency detection models. We find that some models perform consistently better than the others. Saliency models that intend to predict eye fixations perform lower on segmentation datasets compared to salient object detection algorithms. Further, we propose combined models which show that integration of the few best models outperforms all models over other datasets. By analyzing the consistency among the best models and among humans for each scene, we identify the scenes where models or humans fail to detect the most salient object. We highlight the current issues and propose future research directions.


Large Object Salient Object Model Ranking Salient Region Saliency Detection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Judd, T., Ehinger, K., Durand, F.: Learning to predict where humans look. In: ICCV (2009)Google Scholar
  2. 2.
    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. PAMI 20(11), 1254–1259 (1998)CrossRefGoogle Scholar
  3. 3.
    Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: NIPS (2006)Google Scholar
  4. 4.
    Bruce, N.D.B., Tsotsos, J.K.: Saliency based on information maximization. In: NIPS (2005)Google Scholar
  5. 5.
    Hou, X., Zhang, L.: Saliency detection: A spectral residual approach. In: CVPR (2007)Google Scholar
  6. 6.
    Hou, X., Zhang, L.: Dynamic attention: Searching for coding length increments. In: NIPS (2008)Google Scholar
  7. 7.
    Garcia-Diaz, A., Fdez-Vidal, X.R., Pardo, X.M., Dosil, R.: Decorrelation and Distinctiveness Provide with Human-Like Saliency. In: Blanc-Talon, J., Philips, W., Popescu, D., Scheunders, P. (eds.) ACIVS 2009. LNCS, vol. 5807, pp. 343–354. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  8. 8.
    Seo, H.J., Milanfar, P.: Static and space-time visual saliency detection by self-resemblance. Journal of Vision 9, 1–27 (2009)CrossRefGoogle Scholar
  9. 9.
    Walther, D., Koch, C.: Modeling attention to salient proto-objects. Neural Net. (2006)Google Scholar
  10. 10.
    Zhang, L., Tong, M.H., Marks, T.K., Shan, H., Cottrell, G.W.: SUN: A Bayesian framework for saliency using natural statistics. JOV (2008)Google Scholar
  11. 11.
    Tatler, B.W.: The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor bases and image feature distributions. J. Vision 14(7) (2007)Google Scholar
  12. 12.
    Zhai, Y., Shah, M.: Visual attention detection in video sequences using spatiotemporal cues. ACM Multimedia (2006)Google Scholar
  13. 13.
    Cheng, M.M., Zhang, G.X., Mitra, N.J., Huang, X., Hu, S.M.: Global contrast based salient region detection. In: CVPR (2011)Google Scholar
  14. 14.
    Achanta, R., Estrada, F.J., Wils, P., Süsstrunk, S.: Salient Region Detection and Segmentation. In: Gasteratos, A., Vincze, M., Tsotsos, J.K. (eds.) ICVS 2008. LNCS, vol. 5008, pp. 66–75. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  15. 15.
    Goferman, S., Zelnik-Manor, L., Tal, A.: Context-aware saliency detection. In: CVPR (2010)Google Scholar
  16. 16.
    Achanta, R., Hemami, S., Estrada, F., Süsstrunk, S.: Frequency-tuned salient region detection. In: CVPR (2009)Google Scholar
  17. 17.
    Wang, J., Sun, J., Quan, L., Tang, X., Shum, H.Y.: Picture collage. In: CVPR (2006)Google Scholar
  18. 18.
    Wang, M., Konrad, J., Ishwar, P., Jing, Y., Rowley, H.: Image saliency: from intrinsic to extrinsic context. In: CVPR (2011)Google Scholar
  19. 19.
    Rosin, P.L.: A simple method for detecting salient regions. Pattern Rec. (2009)Google Scholar
  20. 20.
    Goferman, S., Tal, A., Zelnik-Manor, L.: Puzzle-like collage. In: EuroGraphics (2010)Google Scholar
  21. 21.
    Zhang, W., Wu, Q.M.J., Wang, G., Yin, H.: An adaptive computational model for salient object detection. IEEE Trans. on Multimedia 12(4) (2010)Google Scholar
  22. 22.
    Feng, J., Wei, Y., Tao, L., Zhang, C., Sun, J.: Salient object detection by composition. In: ICCV (2011)Google Scholar
  23. 23.
    Mehrani, P., Veksler, O.: Saliency segmentation based on learning and graph cut. In: BMVC (2010)Google Scholar
  24. 24.
    Lu, Y., Zhang, W., Lu, H., Xue, X.: Salient object detection using concavity context. In: ICCV (2011)Google Scholar
  25. 25.
    Klein, D.A., Frintrop, S.: Center-surround divergence of feature statistics for salient object detection. In: ICCV (2011)Google Scholar
  26. 26.
    Wang, L., Xue, J., Zheng, N., Hua, G.: Automatic Salient object extraction with contextual cue. In: ICCV (2011)Google Scholar
  27. 27.
    Chang, K.Y., Liu, T.L., Chen, H.T., Lai, S.H.: Fusing generic objectness and visual saliency for salient object detection. In: ICCV (2011)Google Scholar
  28. 28.
    Khuwuthyakorn, P., Robles-Kelly, A., Zhou, J.: Object of Interest Detection by Saliency Learning. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part II. LNCS, vol. 6312, pp. 636–649. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  29. 29.
    Rahtu, E., Kannala, J., Salo, M., Heikkilä, J.: Segmenting Salient Objects from Images and Videos. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 366–379. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  30. 30.
    Kanan, C., Cottrell, G.: Robust classification of objects, faces, and flowers using natural image. In: CVPR (2010)Google Scholar
  31. 31.
    Jiang, H., Wang, J., Yuan, Z., Liu, T., Zheng, N., Li, S.: Automatic salient object segmentation based on context and shape prior. In: BMVC (2011)Google Scholar
  32. 32.
    Li, J., Tian, Y., Huang, T., Gao, W.: A dataset and evaluation methodology for visual saliency in video. In: Int. Conf. on Multimedia and Expo., pp. 442–445 (2009)Google Scholar
  33. 33.
    Liu, T., Sun, J., Zheng, N., Tang, X., Shum, H.: Learning to detect a salient object. In: CVPR (2007)Google Scholar
  34. 34.
    Gopalakrishnan, V., Hu, Y., Rajan, D.: Random walks on graphs to model saliency in images. In: CVPR (2009)Google Scholar
  35. 35.
    Marchesotti, L., Cifarelli, C., Csurka, G.: A framework for visual saliency detection with applications to image thumbnailing. In: ICCV (2009)Google Scholar
  36. 36.
    Li, J., Levine, M.D., An, X., He, H.: Saliency detection based on frequency and spatial domain analysis. In: BMVC (2011)Google Scholar
  37. 37.
    Holtzman-Gazit, M., Zelnik-Manor, L., Yavneh, I.: Salient edges: A multi scale approach. In: ECCV, Workshop on Vision for Cognitive Tasks (2010)Google Scholar
  38. 38.
    Luo, Y., Yuan, J., Xue, P., Tian, Q.: Saliency Density Maximization for Object Detection and Localization. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part III. LNCS, vol. 6494, pp. 396–408. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  39. 39.
    Deng, Q., Luo, Y.: Edge-based method for detecting salient objects. Opt. Eng. 50 (2011)Google Scholar
  40. 40.
    Valenti, R., Sebe, N., Gevers, T.: Image saliency by isocentric curvedness and color. In: ICCV (2009)Google Scholar
  41. 41.
    Zhang, Q., Liu, H., Shen, J., Gu, G., Xiao, H.: An improved computational approach for salient region detection. Journal of Computers (2010)Google Scholar
  42. 42.
    Li, H., Ngan, K.N.: A co-saliency model of image pairs. IEEE Trans. Image Process (2011)Google Scholar
  43. 43.
    Ge, F., Wang, S.: New benchmark for image segmentation evaluation. Journal of Electronic Imaging 16(3) (2007)Google Scholar
  44. 44.
    Estrada, F.J., Jepson, A.D.: Benchmarking image segmentation algorithms. IJCV (2009)Google Scholar
  45. 45.
    Ancuti, C.O., Ancuti, C., Bekaert, P.: CVPR (2011)Google Scholar
  46. 46.
    Movahedi, V., Elder, J.H.: Design and perceptual validation of performance measures for salient object segmentation. In: POCV (2010)Google Scholar
  47. 47.
    Alpert, S., Galun, M., Basri, R., Brandt, A.: Image segmentation by probabilistic bottom-up aggregation and cue integration. In: CVPR (2007)Google Scholar
  48. 48.
    Ge, F., Wang, S., Liu, T.: Image-segmentation evaluation from the perspective of salient object extraction. In: CVPR (2006)Google Scholar
  49. 49.
    Mishra, A.K., Aloimonos, Y., Fah, C.L., Kassim, A.: Active visual segmentation. IEEE Trans. PAMI (2011)Google Scholar
  50. 50.
    Siagian, C., Koch, C.: Salient segmentation using contours and region growing (submitted)Google Scholar
  51. 51.
    Ma, Y.F., Zhang, H.J.: Contrast-based image attention analysis by using fuzzy growing. ACM Multimedia, 374–381 (2003)Google Scholar
  52. 52.
    Liu, Z., Xue, Y., Yan, H., Zhang, Z.: Efficient saliency detection based on Gaussian models. IET Image Processing 5(2), 122–131 (2011)CrossRefGoogle Scholar
  53. 53.
    Liu, Z., Xue, Y., Shen, L., Zhang, Z.: Nonparametric saliency detection using kernel density estimation. In: ICIP, pp. 253–256 (2010)Google Scholar
  54. 54.
    Li, J., Tian, Y., Huang, T., Gao, W.: Probabilistic multi-task learning for visual saliency estimation in video. IJCV 90(2), 150–165 (2010)CrossRefGoogle Scholar
  55. 55.
    Achanta, R., Susstrunk, S.: Saliency detection for content-aware image resizing. In: ICIP (2009)Google Scholar
  56. 56.
    Guo, C., Zhang, L.: A novel multiresolution spatiotemporal saliency detection model and Its applications in image and video compression. IEEE Trans. on Image Processing (2010)Google Scholar
  57. 57.
    Huang, T.H., Cheng, K.Y., Chuang, Y.Y.: A collaborative benchmark for region of interest detection algorithms. In: CVPR (2009)Google Scholar
  58. 58.
    Masciocchi, C.M., Mihalas, S., Parkhurst, D., Niebur, E.: Everyone knows what is interesting: salient locations which should be fixated. Journal of Vision (2009)Google Scholar
  59. 59.
    Itti, L.: Automatic Foveation for Video Compression using a neurobiological model of visual attention. IEEE Trans. Image Process (2004)Google Scholar
  60. 60.
    Ma, Y., Hua, X., Lu, L., Zhang, H.: A generic framework of user aattention model and its application in video summarization. IEEE Trans. Multimedia (2005)Google Scholar
  61. 61.
    Borji, A., Itti, L.: State-of-the-art in visual attention modeling. IEEE Trans. Pattern Anal. Mach. Intell (2012)Google Scholar
  62. 62.
    Gao, D., Vasconcelos, N.: Discriminant saliency for visual recognition from cluttered scenes. In: NIPS (2004)Google Scholar
  63. 63.
    Kienzle, W., Franz, M.O., Schölkopf, B., Wichmann, F.A.: Center-surround patterns emerge as optimal predictors for human saccade targets. J. Vision (2009)Google Scholar
  64. 64.
    Judd, T.: Understanding and predicting where people look. Phd Thesis, MIT (2011)Google Scholar
  65. 65.
    Itti, L., Dhavale, N., Pighin, F.: SPIE (2003)Google Scholar
  66. 66.
    Koehler, K., Guo, F., Zhang, S., Eckstein, M.: Vision Science Society (2011)Google Scholar
  67. 67.
    Borji, A., Sihite, D.N., Itti, L.: Quantitative analysis of human-model agreement in visual saliency modeling: a comparative study. IEEE Trans. Image Processing (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Ali Borji
    • 1
  • Dicky N. Sihite
    • 1
  • Laurent Itti
    • 1
  1. 1.Department of Computer ScienceUniversity of Southern CaliforniaUSA

Personalised recommendations