Segmenting Salient Objects from Images and Videos

  • Esa Rahtu
  • Juho Kannala
  • Mikko Salo
  • Janne Heikkilä
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6315)


In this paper we introduce a new salient object segmentation method, which is based on combining a saliency measure with a conditional random field (CRF) model. The proposed saliency measure is formulated using a statistical framework and local feature contrast in illumination, color, and motion information. The resulting saliency map is then used in a CRF model to define an energy minimization based segmentation approach, which aims to recover well-defined salient objects. The method is efficiently implemented by using the integral histogram approach and graph cut solvers. Compared to previous approaches the introduced method is among the few which are applicable to both still images and videos including motion cues. The experiments show that our approach outperforms the current state-of-the-art methods in both qualitative and quantitative terms.


Video Sequence Salient Object Conditional Random Field Saliency Detection Conditional Random Field Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Yarbus, A.: Eye movements and vision. Plenum, New York (1967)Google Scholar
  2. 2.
    Avidan, S., Shamir, A.: Seam carving for content-aware image resizing. ACM Transactions of Graphics 26 (2007)Google Scholar
  3. 3.
    Marchesotti, L., Cifarelli, C., Csurka, G.: A framework for visual saliency detection with applications to image thumbnailing. In: ICCV (2009)Google Scholar
  4. 4.
    Han, J., Ngan, K.N., Li, M., Zhang, H.: Unsupervised extraction of visual attention objects in color images. IEEE Trans. Circuits Syst. Video Techn. 16, 141–145 (2006)CrossRefGoogle Scholar
  5. 5.
    Ko, B., Nam, J.: Object-of-interest image segmentation based on human attention and semantic region clustering. J. Opt. Soc. Am. 23, 2462–2470 (2006)CrossRefGoogle Scholar
  6. 6.
    Rutishauser, U., Walther, D., Koch, C., Perona, P.: Is bottom-up attention useful for object recognition. In: CVPR (2004)Google Scholar
  7. 7.
    Yang, L., Zheng, N., Yang, J., Chen, M., Cheng, H.: A biased sampling strategy for object categorization. In: ICCV (2009)Google Scholar
  8. 8.
    Sheikh, Y., Shah, M.: Bayesian modeling of dynamic scenes for object detection. IEEE TPAMI 27, 1778–1792 (2005)Google Scholar
  9. 9.
    Monnet, A., Mittal, A., Paragios, N., Ramesh, V.: Background modeling and subtraction of dynamic scenes. In: CVPR, vol. 2, pp. 1305–1312 (2003)Google Scholar
  10. 10.
    Heikkilä, M., Pietikäinen, M.: A texture-based method for modeling the background and detecting moving objects. IEEE TPAMI 28, 657–662 (2006)Google Scholar
  11. 11.
    Ren, Y., Chua, C., Ho, Y.: Motion detection with nonstationary background. Machine Vision and Applications 13, 332–343 (2003)CrossRefGoogle Scholar
  12. 12.
    Sheikh, Y., Javed, O., Kanade, T.: Background Subtraction for Freely Moving Cameras. In: ICCV (2009)Google Scholar
  13. 13.
    Lampert, C., Blaschko, M., Hofmann, T.: Efficient subwindow search: A branch and bound framework for object localization. IEEE TPAMI 31, 2129–2142 (2009)Google Scholar
  14. 14.
    Desai, C., Ramanan, D., Fowlkes, C.: Discriminative models for multi-class object layout. In: ICCV (2009)Google Scholar
  15. 15.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR, vol. 1 (2005)Google Scholar
  16. 16.
    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE TPAMI 20, 1254–1259 (1998)Google Scholar
  17. 17.
    Itti, L., Koch, C.: A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Research 40, 1489–1506 (2000)CrossRefGoogle Scholar
  18. 18.
    Ma, Y., Zhang, H.: Contrast-based image attention analysis by using fuzzy growing. In: ACM Intl. Conf. on Multimedia, pp. 59–68 (2003)Google Scholar
  19. 19.
    Hu, Y., Xie, X., Ma, W., Chia, L., Rajan, D.: Salient region detection using weighted feature maps based on the human visual attention model. In: Aizawa, K., Nakamura, Y., Satoh, S. (eds.) PCM 2004. LNCS, vol. 3332, pp. 993–1000. Springer, Heidelberg (2004)Google Scholar
  20. 20.
    Achanta, R., Estrada, F.J., Wils, P., Süsstrunk, S.: Salient region detection and segmentation. In: Gasteratos, A., Vincze, M., Tsotsos, J.K. (eds.) ICVS 2008. LNCS, vol. 5008, pp. 66–75. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  21. 21.
    Rahtu, E., Heikkilä, J.: A simple and efficient saliency detector for background subtraction. In: IEEE Intl. Workshop on Visual Surveillance, pp. 1137–1144 (2009)Google Scholar
  22. 22.
    Mahadevan, V., Vasconcelos, N.: Spatiotemporal saliency in highly dynamic scenes. IEEE TPAMI 32, 171–177 (2010)Google Scholar
  23. 23.
    Hou, X., Zhang, L.: Saliency detection: A spectral residual approach. In: CVPR (2007)Google Scholar
  24. 24.
    Guo, C., Ma, Q., Zhang, L.: Spatio-temporal saliency detection using phase spectrum of quaternion fourier transform. In: CVPR (2008)Google Scholar
  25. 25.
    Gao, D., Vasconcelos, N.: Bottom-up saliency is a discriminant process. In: ICCV (2007)Google Scholar
  26. 26.
    Achanta, R., Hemami, S.S., Estrada, F.J., Süsstrunk, S.: Frequency-tuned salient region detection. In: CVPR (2009)Google Scholar
  27. 27.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE TPAMI 23, 1222–1239 (2001)Google Scholar
  28. 28.
    Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? IEEE TPAMI 26, 147–159 (2004)Google Scholar
  29. 29.
    Rother, C., Kolmogorov, V., Blake, A.: ”Grabcut”: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23, 309–314 (2004)CrossRefGoogle Scholar
  30. 30.
    Criminisi, A., Cross, G., Blake, A., Kolmogorov, V.: Bilayer segmentation of live video. In: CVPR, vol. 1, pp. 53–60 (2006)Google Scholar
  31. 31.
    Szummer, M., Kohli, P., Hoiem, D.: Learning CRFs using graph cuts. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 582–595. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  32. 32.
    Werlberger, M., Trobin, W., Pock, T., Wedel, A., Cremers, D., Bischof, H.: Anisotropic Huber-L1 optical flow. In: BMVC (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Esa Rahtu
    • 1
  • Juho Kannala
    • 1
  • Mikko Salo
    • 2
  • Janne Heikkilä
    • 1
  1. 1.Machine Vision GroupUniversity of OuluFinland
  2. 2.Department of Mathematics and StatisticsUniversity of HelsinkiFinland

Personalised recommendations