Saliency Detection Using Joint Temporal and Spatial Decorrelation

  • Hamed Rezazadegan Tavakoli
  • Esa Rahtu
  • Janne Heikkilä
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7944)


This article presents a scene-driven (i.e. bottom-up) visual saliency detection technique for videos. The proposed method utilizes non-negative matrix factorization (NMF) to replicate neural responses of primary visual cortex neurons in spatial domain. In temporal domain, principal component analysis (PCA) was applied to imitate the effect of stimulus change experience during neural adaptation phenomena. We apply the proposed saliency model to background subtraction problem. The proposed method does not rely on any background model and is purely unsupervised. In experimental results, it will be shown that the proposed method competes well with some of the state-of-the-art background subtraction techniques especially in dynamic scenes.


Independent Component Analysis Sparse Code Salient Region Equal Error Rate Saliency Detection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Borji, A.: Boosting bottom-up and top-down visual features for saliency estimation. In: IEEE Conference on Computer Vision and Pattern Recognition (2012)Google Scholar
  2. 2.
    Rahtu, E., Kannala, J., Blaschko, M.B.: Learning a category independent object detection cascade. In: IEEE International Conference on Computer Vision (2011)Google Scholar
  3. 3.
    Rutishauser, U., Walther, D., Koch, C., Perona, P.: Is bottom-up attention useful for object recognition? In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 2, pp. II-37–II-44 (2004)Google Scholar
  4. 4.
    Kanan, C., Cottrell, G.: Robust classification of objects, faces, and flowers using natural image statistics. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2472–2479 (2010)Google Scholar
  5. 5.
    Achanta, R., Estrada, F., Wils, P., Süsstrunk, S.: Salient Region Detection and Segmentation. In: Gasteratos, A., Vincze, M., Tsotsos, J.K. (eds.) ICVS 2008. LNCS, vol. 5008, pp. 66–75. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  6. 6.
    Rahtu, E., Kannala, J., Salo, M., Heikkilä, J.: Segmenting salient objects from images and videos. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 366–379. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  7. 7.
    Frintrop, S.: General object tracking with a component-based target descriptor. In: 2010 IEEE International Conference on Robotics and Automation (ICRA), pp. 4531–4536 (2010)Google Scholar
  8. 8.
    Guo, C., Zhang, L.: A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression. Trans. Img. Proc. 19(1), 185–198 (2010)CrossRefGoogle Scholar
  9. 9.
    Lu, T., Yuan, Z., Huang, Y., Wu, D., Yu, H.: Video retargeting with nonlinear spatial-temporal saliency fusion. In: Proceedings of the 2010 IEEE 17th International Conference on Image Processing (2010)Google Scholar
  10. 10.
    Jacobson, N., Lee, Y.L., Mahadevan, V., Vasconcelos, N., Nguyen, T.: A novel approach to fruc using discriminant saliency and frame segmentation. IEEE Transactions on Image Processing 19(11), 2924–2934 (2010)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Marchesotti, L., Cifarelli, C., Csurka, G.: A framework for visual saliency detection with applications to image thumbnailing. In: IEEE 12th International Conference on Computer Vision, pp. 2232–2239 (2009)Google Scholar
  12. 12.
    Goferman, S., Zelnik-Manor, L., Tal, A.: Context-aware saliency detection. In: 2010 IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 2376–2383 (2010)Google Scholar
  13. 13.
    Bouwmans, T.: Recent advanced statistical background modeling for foreground detection: A systematic survey. Recent Patents on Computer Science 4(3), 147–176 (2011)Google Scholar
  14. 14.
    Calderara, S., Melli, R., Prati, A., Cucchiara, R.: Reliable background suppression for complex scenes. In: Proceedings of the 4th ACM International Workshop on Video Surveillance and Sensor Networks, pp. 211–214 (2006)Google Scholar
  15. 15.
    Heikkilä, J., Silvn, O.: A real-time system for monitoring of cyclists and pedestrians. Image and Vision Computing 22(7), 563–570 (2004)CrossRefGoogle Scholar
  16. 16.
    Stauffer, C., Grimson, W.: Adaptive background mixture models for real-time tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. xxiii+637+663 (1999)Google Scholar
  17. 17.
    Zivkovic, Z., van der Heijden, F.: Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recogn. Lett. 27(7), 773–780 (2006)CrossRefGoogle Scholar
  18. 18.
    Elgammal, A., Harwood, D., Davis, L.: Non-parametric model for background subtraction. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1843, pp. 751–767. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  19. 19.
    Mahadevan, V., Vasconcelos, N.: Spatiotemporal saliency in dynamic scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(1), 171–177 (2010)CrossRefGoogle Scholar
  20. 20.
    Gao, D., Mahadevan, V., Vasconcelos, N.: On the plausibility of the discriminant center-surround hypothesis for visual saliency. Journal of Vision 8(7) (2008)Google Scholar
  21. 21.
    Itti, L., Baldi, P.: Bayesian surprise attracts human attention. Vision Research 49(10), 1295–1306 (2009)CrossRefGoogle Scholar
  22. 22.
    Olshausen, B., Field, D.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381(6583), 607–609 (1996)CrossRefGoogle Scholar
  23. 23.
    Tsotsos, J.K., Bruce, N.D.B.: Saliency based on information maximization. In: Weiss, Y., Schölkopf, B., Platt, J. (eds.) Advances in Neural Information Processing Systems, vol. 18, pp. 155–162. MIT Press (2006)Google Scholar
  24. 24.
    Zhang, L., Tong, M.H., Marks, T.K., Shan, H., Cottrell, G.W.: Sun: A bayesian framework for saliency using natural statistics. Journal of Vision 8(7) (2008)Google Scholar
  25. 25.
    Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401(6755), 788–791 (1999)CrossRefGoogle Scholar
  26. 26.
    Hoyer, P.O.: Modeling receptive fields with non-negative sparse coding. Neurocomputing 52–54, 547–552 (2003)Google Scholar
  27. 27.
    Hoyer, P.O.: Non-negative matrix factorization with sparseness constraints. J. Mach. Learn. Res. 5, 1457–1469 (2004)MathSciNetzbMATHGoogle Scholar
  28. 28.
    Rajapakse, M., Wyse, L.: Nmf vs ica for face recognition. In: Guo, M. (ed.) ISPA 2003. LNCS, vol. 2745, pp. 605–610. Springer, Heidelberg (2003)Google Scholar
  29. 29.
    Olmos, A., Kingdom, F.A.A.: A biologically inspired algorithm for the recovery of shading and reflectance images. Perception 33, 1463–1473 (2004)CrossRefGoogle Scholar
  30. 30.
    Barnich, O., Van Droogenbroeck, M.: Vibe: A universal background subtraction algorithm for video sequences. IEEE Transactions on Image Processing 20(6), 1709–1724 (2011)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Maddalena, L., Petrosino, A.: The sobs algorithm: What are the limits. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 21–26 (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Hamed Rezazadegan Tavakoli
    • 1
  • Esa Rahtu
    • 1
  • Janne Heikkilä
    • 1
  1. 1.Center for Machine Vision ResearchUniversity of OuluFinland

Personalised recommendations