Skip to main content
SpringerLink
Log in

Spatio-temporal saliency detection using objectness measure

  • Original Article
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Spatio-temporal saliency detection has gradually gained much attention in various computer vision applications such as intelligent video advertising and visual tracking. In this paper, we present a new approach based on the spatial and temporal information of the input video frame which aims to find the similar salient objects is proposed. First, objectness measure is performed to highlight the regions that may contain the object of interest. Then, for each candidate, newly proposed motion distinctiveness cues and static features including contrast measure and spatial distance are used to compute saliency maps. Experiments over two widely benchmark datasets, using several evaluation metrics such as mean absolute error, F score and area under the ROC curve measures, show the efficiency of our saliency approach compared to recent state-of-the-art methods. More interestingly, our attended scenes locations are coherent with the ground truth video frames. On SegTrack v2 and Fukuchi datasets, our proposed method yielded an overall mean absolute error, respectively, of 0.0669 and 0.0794. These results indicate the potential of our proposed framework in detecting motion salient objects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Itti, L.: Visual salience. Scholarpedia 2(9), 3327 (2007)

    Article  Google Scholar 

  2. Treisman, A.M., Gelade, G.: A feature-integration theory of attention. Cogn. Psychol. 12(1), 97–136 (1980)

    Article  Google Scholar 

  3. Koch, C., Ullman, S.: Shifts in selective visual attention: towards the underlying neural circuitry. In: Matters of Intelligence, pp. 115–141. Springer (1987)

  4. Itti, L., Koch, C.: Computational modelling of visual attention. Nat. Rev. Neurosci. 2(3), 194–203 (2001)

    Article  Google Scholar 

  5. Itti, L., Koch, C.: A saliency-based search mechanism for overt and covert shifts of visual attention. Vis. Res. 40(10), 1489–1506 (2000)

    Article  Google Scholar 

  6. Chang, J., Wei, D., Fisher, J.W.: A video representation using temporal superpixels. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2051–2058 (2013)

  7. Singh, A., Chu, C.H.H., Pratt, M.A.: Learning to predict video saliency using temporal superpixels. In: ICPRAM (2), pp. 201–209 (2015)

  8. Zhou, F., Bing Kang, S., Cohen, M.F.: Time-mapping using space-time saliency. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3358–3365 (2014)

  9. Mauthner, T., Possegger, H., Waltner, G., Bischof, H.: Encoding based saliency detection for videos and images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2494–2502 (2015)

  10. Shanableh, T.: Saliency detection in MPEG and HEVC video using intra-frame and inter-frame distances. Signal Image Video Process. 10(4), 703–709 (2016)

    Article  Google Scholar 

  11. Hamel, S., Guyader, N., Pellerin, D., Houzet, D.: Contribution of color in saliency model for videos. Signal Image Video Process. 10(3), 423–429 (2016)

    Article  Google Scholar 

  12. Annum, R., Riaz, M.M., Ghafoor, A.: Saliency detection using contrast enhancement and texture smoothing operations. Signal Image Video Process. 12(3), 505–511 (2018)

    Article  Google Scholar 

  13. Bhattacharya, S., Venkatsh, K., Gupta, S.: Background estimation and motion saliency detection using total variation-based video decomposition. Signal Image Video Process. 11(1), 113–121 (2017)

    Article  Google Scholar 

  14. Imamoglu, N., Shimoda, W., Zhang, C., Fang, Y., Kanezaki, A., Yanai, K., Nishida, Y.: An integration of bottom-up and top-down salient cues on RGB-D data: saliency from objectness versus non-objectness. Signal Image Video Process. 12(2), 307–314 (2018)

    Article  Google Scholar 

  15. Şaykol, E., Gudukbay, U., Ulusoy, O.: Integrated querying of images by color, shape, and texture content of salient objects. In: International Conference on Advances in Information Systems, pp. 363–371 (2004)

  16. Bastan, M., Gudukbay, U., Ulusoy, O.: Segmentation-based extraction of important objects from video for object-based indexing. In: Multimedia and Expo, 2008 IEEE International Conference on Multimedia and Expo, pp. 1357–1360 (2008)

  17. Şaykol, E., Gudukbay, U., Ulusoy, O.: A database model for querying visual surveillance videos by integrating semantic and low-level features. In: International Workshop on Multimedia Information Systems, pp. 163–176 (2005)

  18. Goferman, S., Manor, Z.L., Tal, A.: Context-aware saliency detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(10), 1915–1926 (2012)

    Article  Google Scholar 

  19. Koffka, K.: Principles of Gestalt Psychology, vol. 44. Routledge, Abingdon-on-Thames (2013)

    Book  Google Scholar 

  20. Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)

  21. Jiang, H., Wang, J., Yuan, Z., Liu, T., Zheng, N., Li, S.: Automatic salient object segmentation based on context and shape prior. In: British Machine Vision Conference–BMVC, vol. 6, p. 9 (2011)

  22. Mancas, M., Riche, N., Leroy, J., Gosselin, B.: Abnormal motion selection in crowds using bottom-up saliency. In: 18th IEEE International Conference on Image Processing, pp. 229–232 (2011)

  23. Rahtu, E., Kannala, J., Salo, M., Heikkilä, J.: Segmenting salient objects from images and videos. In: Computer Vision—ECCV, pp. 366–379 (2010)

  24. Itti, L., Baldi, P.: A principled approach to detecting surprising events in video. In: IEEE International Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 631–637 (2005)

  25. Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Advances in Neural Information Processing Systems, pp. 545–552 (2007)

  26. Achanta, R., Shaji, A., Smith, K.: Aurelienlucchi, pascal fua, and sabine s usstrunk, “slic superpixels”. Technical report, EPFL technical report 149300 (2010)

  27. Alexe, B., Deselaers, T., Ferrari, V.: Measuring the objectness of image windows. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2189–2202 (2012)

    Article  Google Scholar 

  28. Srivatsa, S., Babu, V.: Salient object detection via objectness measure. In: Proceedings of the IEEE International Conference on Image Processing, pp. 4481–4485 (2015)

  29. Lucas, B.D., Kanade, T., et al.: An iterative image registration technique with an application to stereo vision (1981)

  30. Kalboussi, R., Abdellaoui, M., Douik, A.: Video saliency detection based on Boolean map theory. In: International Conference on Image Analysis and Processing, pp. 119–128 (2017)

  31. Zhu, W., Liang, S., Wei, Y., Sun, J.: Saliency optimization from robust background detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2814–2821 (2014)

  32. Tsai, D., Flagg, M., Nakazawa, A., Rehg, J.M.: Motion coherent tracking using multi-label MRF optimization. Int. J. Comput. Vis. 100(2), 190–202 (2012)

    Article  MathSciNet  Google Scholar 

  33. Fukuchi, K., Miyazato, K., Kimura, A., Takagi, S., Yamato, J.: Saliency-based video segmentation with graph cuts and sequentially updated priors. In: IEEE International Conference on Multimedia and Expo, pp. 638–641 (2009)

  34. Borji, A., Cheng, M.M., Jiang, H., Li, J.: Salient object detection: a benchmark. IEEE Trans. Image Process. 24(12), 5706–5722 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  35. Cheng, M.M., Mitra, N.J., Huang, X., Torr, P.H., Hu, S.M.: Global contrast based salient region detection. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 569–582 (2015)

    Article  Google Scholar 

  36. Achanta, R., Hemami, S., Estrada, F., Susstrunk, S.: Frequency-tuned salient region detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1597–1604 (2009)

  37. Perazzi, F., Krähenbühl, P., Pritch, Y., Hornung, A.: Saliency filters: contrast based filtering for salient region detection. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 733–740 (2012)

  38. Fawcett, T.: An introduction to ROC analysis. Pattern Recognit. Lett. 27(8), 861–874 (2006)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Networked Objects Control and Communication Systems Laboratory, National Engineering School of Sousse, University of Sousse, Sousse, Tunisia

    Khawla Brahim, Rahma Kalboussi, Mehrez Abdellaoui & Ali Douik

Authors
  1. Khawla Brahim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rahma Kalboussi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mehrez Abdellaoui
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ali Douik
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Khawla Brahim.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Brahim, K., Kalboussi, R., Abdellaoui, M. et al. Spatio-temporal saliency detection using objectness measure. SIViP 13, 1055–1062 (2019). https://doi.org/10.1007/s11760-019-01445-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-019-01445-0

Keywords

Search

Navigation