Local Saliency Estimation and Global Homogeneity Refinement for Video Saliency Detection

  • Rahma KalboussiEmail author
  • Mehrez AbdellaouiEmail author
  • Ali DouikEmail author
Conference paper
Part of the Smart Innovation, Systems and Technologies book series (SIST, volume 98)


Saliency detection aims to segment the object of interest from the rest of the scene. While there has been a big number of saliency detection methods in still images, video saliency is in its early stages. In this paper, we propose a two stages video saliency detection method using local saliency estimation and global homogeneity refinement. Starting from a patch, the problem of saliency detection is modeled as a growing region which starts from a patch with high saliency information to the background. Local saliency is measured by combining spatial priors presented by local surrounding contrast with temporal information issued from the motion estimation feature. Temporal and spatial information are fused and then used to label each patch as foreground and background patches and produce the final saliency maps. Finally, Global homogeneity refinement is used to refine the saliency results by evaluating the foreground and background probabilities ratio propagated from the patches. Experiments have proved that the proposed method outperforms state-of-the-art methods over two benchmark datasets.


  1. 1.
    Avidan, S., Shamir, A.: Seam carving for content-aware image resizing. ACM Trans. Graph. (TOG) 26, 10 (2007)CrossRefGoogle Scholar
  2. 2.
    Bi, S., Li, G., Yu, Y.: Person re-identification using multiple experts with random subspaces. J. Image Graph. 2 (2014)Google Scholar
  3. 3.
    Borji, A., Cheng, M.-M., Jiang, H., Li, J.: Salient object detection: a benchmark. IEEE Trans. Image Process. 24, 5706–5722 (2015)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Fang, Y., Lin, W., Chen, Z., Tsai, C.-M., Lin, C.-W.: A video saliency detection model in compressed domain. IEEE Trans. Circ. Syst. Video Technol. 24, 27–38 (2014)CrossRefGoogle Scholar
  5. 5.
    Fu, H., Cao, X., Tu, Z.: Cluster-based co-saliency detection. IEEE Trans. Image Process. 22, 3766–3778 (2013)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Fukuchi, K., Miyazato, K., Kimura, A., Takagi, S., Yamato, J.: Saliency-based video segmentation with graph cuts and sequentially updated priors. In: 2009 IEEE International Conference on Multimedia and Expo, pp. 638–641. IEEE (2009)Google Scholar
  7. 7.
    Gao, D., Mahadevan, V., Vasconcelos, N.: The discriminant center-surround hypothesis for bottom-up saliency. In: Advances in Neural Information Processing Systems, pp. 497–504 (2008)Google Scholar
  8. 8.
    Gao, D., Vasconcelos, N.: Bottom-up saliency is a discriminant process. In: IEEE 11th International Conference on Computer Vision, ICCV 2007, pp. 1–6. IEEE (2007)Google Scholar
  9. 9.
    Goferman, S., Zelnik-Manor, L., Tal, A.: Context-aware saliency detection. IEEE Trans. Pattern Anal. Mach. Intell. 34, 1915–1926 (2012)CrossRefGoogle Scholar
  10. 10.
    Harel, J., Koch, C., Perona, P., et al.: Graph-based visual saliency. In: NIPS, vol. 1, p. 5 (2006)Google Scholar
  11. 11.
    Itti, L., Baldi, P.: A principled approach to detecting surprising events in video. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 631–637. IEEE (2005 )Google Scholar
  12. 12.
    Kim, H., Kim, Y., Sim, J.-Y., Kim, C.-S.: Spatiotemporal saliency detection for video sequences based on random walk with restart. IEEE Transa. Image Process. 24, 2552–2564 (2015)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Lee, S.-H., Kim, J.-H., Choi, K.P., Sim, J.-Y., Kim, C.-S.: Video saliency detection based on spatiotemporal feature learning. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 1120–1124. IEEE (2014)Google Scholar
  14. 14.
    Li, F., Kim, T., Humayun, A., Tsai, D., Rehg, J.M.: Video segmentation by tracking many figure-ground segments. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2192–2199 (2013)Google Scholar
  15. 15.
    Lucas, B.D., Kanade, T., et al.: An iterative image registration technique with an application to stereo vision. In: IJCAI, vol. 81, pp. 674–679 (1981)Google Scholar
  16. 16.
    Mahadevan, V., Vasconcelos, N.: Spatiotemporal saliency in dynamic scenes. IEEE Trans. Pattern Anal. Mach. Intell. 32, 171–177 (2010)CrossRefGoogle Scholar
  17. 17.
    Mancas, M., Riche, N., Leroy, J., Gosselin, B.: Abnormal motion selection in crowds using bottom-up saliency. In: 2011 18th IEEE International Conference on Image Processing (ICIP), pp. 229–232. IEEE (2011)Google Scholar
  18. 18.
    Mauthner, T., Possegger, H., Waltner, G., Bischof, H.: Encoding based saliency detection for videos and images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2494–2502 (2015)Google Scholar
  19. 19.
    Navalpakkam, V., Itti, L.: An integrated model of top-down and bottom-up attention for optimizing detection speed. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), vol. 2, pp. 2049–2056. IEEE (2006)Google Scholar
  20. 20.
    Rahtu, E., Kannala, J., Salo, M., Heikkilä, J.: Segmenting salient objects from images and videos. In: Computer Vision-ECCV 2010, pp. 366–379 (2010)CrossRefGoogle Scholar
  21. 21.
    Seo, H.J., Milanfar, P.: Static and space-time visual saliency detection by self-resemblance. J. Vis. 9, 15–15 (2009)CrossRefGoogle Scholar
  22. 22.
    Singh, A., Chu, C.-H.H., Pratt, M.: Learning to predict video saliency using temporal superpixels. In: 4th International Conference on Pattern Recognition Applications and Methods, pp. 201–209 (2015)Google Scholar
  23. 23.
    Treisman, A.M., Gelade, G.: A feature-integration theory of attention. Cogn. psychol. 12, 97–136 (1980)CrossRefGoogle Scholar
  24. 24.
    Wang, W., Shen, J., Porikli, F.: Saliency-aware geodesic video object segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3395–3402 (2015)Google Scholar
  25. 25.
    Wu, R., Yu, Y., Wang, W.: Scale: supervised and cascaded Laplacian eigenmaps for visual object recognition based on nearest neighbors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 867–874 (2013)Google Scholar
  26. 26.
    Yeh, H.-H., Liu, K.-H., Chen, C.-S.: Salient object detection via local saliency estimation and global homogeneity refinement. Pattern Recogn. 47, 1740–1750 (2014)CrossRefGoogle Scholar
  27. 27.
    Zhong, S., Liu, Y., Ren, F., Zhang, J., Ren, T.: Video saliency detection via dynamic consistent spatio-temporal attention modelling. In: AAAI, pp. 1063–1069 (2013)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Networked Objects Control and Communication Systems LaboratoryPôle technologique de SousseSousseTunisia
  2. 2.University of SousseSousseTunisia
  3. 3.Higher Institute of Computer Sciences and Communications Technology, Hammam SousseSousseTunisia

Personalised recommendations