Advertisement

Predictive Video Saliency Detection

  • Qian Li
  • Shifeng Chen
  • Beiwei Zhang
Part of the Communications in Computer and Information Science book series (CCIS, volume 321)

Abstract

Human Visual System has the characters of focusing on salient regions when seeing images or videos. The method which finds the irregularity and unpredictability of images or videos by simulating human visual system is called saliency detection. In this paper, we propose a novel video saliency detection method based on temporal consistency. The traditional video detection approaches fall into two main groups. One processes a video frame by frame independently without considering motion information, and the other regards optical flow only as a part of features without taking account of consistency of video saliency between consecutive frames. In the proposed method, the temporal consistency constraint is enforced by using motion vectors. By constructing correspondences via motion information, the saliency map of each frame can be predicted by the result of its previous frame. By combining the predicted results and the traditional approaches, our algorithm can achieve better video saliency maps.

Keywords

Still image saliency detection Motion saliency detection Predictive saliency model Video Analysis 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Achanta, R., Hemami, S., Estrada, F., Susstrunk, S.: Frequency-tuned salient region detection. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009., pp. 1597–1604. IEEE (2009)Google Scholar
  2. 2.
    Cheng, M.M., Zhang, G.X., Mitra, N.J., Huang, X., Hu, S.M.: Global contrast based salient region detection. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 409–416. IEEE (2011)Google Scholar
  3. 3.
    Cui, X., Liu, Q., Metaxas, D.: Temporal spectral residual: fast motion saliency detection. In: Proceedings of the ACM International Conference on Multimedia (2009)Google Scholar
  4. 4.
    Guo, C., Ma, Q., Zhang, L.: Spatio-temporal saliency detection using phase spectrum of quaternion fourier transform. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–8. IEEE (2008)Google Scholar
  5. 5.
    Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Advances in Neural Information Processing Systems, vol. 19, p. 545 (2007)Google Scholar
  6. 6.
    Hou, X., Zhang, L.: Saliency detection: A spectral residual approach. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2007, pp. 1–8. IEEE (2007)Google Scholar
  7. 7.
    Itti, L.: Automatic foveation for video compression using a neurobiological model of visual attention. IEEE Transactions on Image Processing 13(10), 1304–1318 (2004)CrossRefGoogle Scholar
  8. 8.
    Itti, L., Baldi, P.: A principled approach to detecting surprising events in video. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 631–637. IEEE (2005)Google Scholar
  9. 9.
    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(11), 1254–1259 (1998)CrossRefGoogle Scholar
  10. 10.
    Li, H., Ngan, K.N.: Saliency model-based face segmentation and tracking in head-and-shoulder video sequences. Journal of Visual Communication and Image Representation 19(5), 320–333 (2008)CrossRefGoogle Scholar
  11. 11.
    Liu, C., Yuen, P., Qiu, G.: Object motion detection using information theoretic spatio-temporal saliency. Pattern Recognition 42(11), 2897–2906 (2009)zbMATHCrossRefGoogle Scholar
  12. 12.
    Mittal, A., Paragios, N.: Motion-based background subtraction using adaptive kernel density estimation. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 2, pp. II–302. IEEE (2004)Google Scholar
  13. 13.
    Slowe, T.E., Marsic, I.: Saliency-based visual representation for compression. In: Proceedings of International Conference on Image Processing 1997, vol. 2, pp. 554–557. IEEE (1997)Google Scholar
  14. 14.
    Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition 1999, vol. 2, IEEE (1999)Google Scholar
  15. 15.
    Sun, D., Roth, S., Black, M.: Secrets of optical flow estimation and their principles. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2432–2439. IEEE (2010)Google Scholar
  16. 16.
    Tuzel, O., Porikli, F., Meer, P.: A bayesian approach to background modeling. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, CVPR Workshops, pp. 58–58. IEEE (2005)Google Scholar
  17. 17.
    Wolf, L., Guttmann, M., Cohen-Or, D.: Non-homogeneous content-driven video-retargeting. In: IEEE 11th International Conference on Computer Vision, ICCV 2007, pp. 1–6. IEEE (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Qian Li
    • 1
    • 2
  • Shifeng Chen
    • 1
    • 2
  • Beiwei Zhang
    • 1
    • 2
  1. 1.Computer and ScienceNanJing University of Finance and EconomicsNanjingChina
  2. 2.Shenzhen Key Lab for Computer Vision and Pattern Recognition, Shenzhen Institutes of Advanced TechnologyChinese Academy of SciencesShenzhenChina

Personalised recommendations