Advertisement

Overview

  • Jianming Zhang
  • Filip Malmberg
  • Stan Sclaroff
Chapter

Abstract

Visual saliency computation is about detecting and understanding pertinent regions and elements in a visual scene. Given limited computational resources, the human visual system relies on saliency computation to quickly grasp important information from the excessive input from the visual world. Modeling visual saliency computation can help computer vision systems to filter out irrelevant information and thus make them fast and smart.

References

  1. 1.
    Achanta, R., Hemami, S., Estrada, F., and Susstrunk, S. Frequency-tuned salient region detection. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2009).Google Scholar
  2. 3.
    Anoraganingrum, D. Cell segmentation with median filter and mathematical morphology operation. In International Conference on Image Analysis and Processing (1999).Google Scholar
  3. 8.
    Ba, J., Mnih, V., and Kavukcuoglu, K. Multiple object recognition with visual attention. In International Conference on Learning Representations (ICLR) (2015).Google Scholar
  4. 10.
    Baylis, G. C., Driver, J., et al. Shape-coding in IT cells generalizes over contrast and mirror reversal, but not figure-ground reversal. Nature Neuroscience 4 (2001), 937–942.CrossRefGoogle Scholar
  5. 11.
    Berg, T. L., and Berg, A. C. Finding iconic images. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (2009).Google Scholar
  6. 12.
    Borji, A., Cheng, M.-M., Jiang, H., and Li, J. Salient object detection: A benchmark. IEEE Transactions on Image Processing (TIP) 24, 12 (2015), 5706–5722.MathSciNetCrossRefGoogle Scholar
  7. 13.
    Borji, A., and Itti, L. Exploiting local and global patch rarities for saliency detection. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012).Google Scholar
  8. 19.
    Bruce, N. D., and Tsotsos, J. K. Saliency, attention, and visual search: An information theoretic approach. Journal of Vision 9, 3 (2009), 5.CrossRefGoogle Scholar
  9. 30.
    Chen, T., Cheng, M.-M., Tan, P., Shamir, A., and Hu, S.-M. Sketch2photo: internet image montage. ACM Transactions on Graphics (TOG) 28, 5 (2009), 124.Google Scholar
  10. 31.
    Chen, X., and Gupta, A. Webly supervised learning of convolutional networks. In IEEE International Conference on Computer Vision (ICCV) (2015).Google Scholar
  11. 32.
    Cheng, M., Mitra, N., Huang, X., Torr, P., and Hu, S. Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 37, 3 (March 2015), 569–582.CrossRefGoogle Scholar
  12. 34.
    Cheng, M.-M., Mitra, N. J., Huang, X., Torr, P. H. S., and Hu, S.-M. Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 37, 3 (2015), 569–582.CrossRefGoogle Scholar
  13. 37.
    Chia, A. Y.-S., Zhuo, S., Gupta, R. K., Tai, Y.-W., Cho, S.-Y., Tan, P., and Lin, S. Semantic colorization with internet images. ACM Transactions on Graphics (TOG) 30, 6 (2011), 156.Google Scholar
  14. 38.
    Choi, J., Jung, C., Lee, J., and Kim, C. Determining the existence of objects in an image and its application to image thumbnailing. Signal Processing Letters 21, 8 (2014), 957–961.CrossRefGoogle Scholar
  15. 40.
    Ciesielski, K. C., Strand, R., Malmberg, F., and Saha, P. K. Efficient algorithm for finding the exact minimum barrier distance. Computer Vision and Image Understanding (CVIU) 123 (2014), 53–64.CrossRefGoogle Scholar
  16. 47.
    DeCarlo, D., and Santella, A. Stylization and abstraction of photographs. ACM Transactions on Graphics (TOG) 21, 3 (2002), 769–776.Google Scholar
  17. 61.
    Garcia-Diaz, A., Fdez-Vidal, X. R., Pardo, X. M., and Dosil, R. Saliency from hierarchical adaptation through decorrelation and variance normalization. Image and Vision Computing 30, 1 (2012), 51–64.CrossRefGoogle Scholar
  18. 69.
    Han, J., Ngan, K. N., Li, M., and Zhang, H.-J. Unsupervised extraction of visual attention objects in color images. IEEE Trans. Circuits and Systems for Video Technology 16, 1 (2006), 141–145.CrossRefGoogle Scholar
  19. 76.
    Huang, H., Zhang, L., and Zhang, H.-C. Arcimboldo-like collage using internet images. ACM Transactions on Graphics (TOG) 30, 6 (2011), 155.Google Scholar
  20. 79.
    Itti, L., Koch, C., and Niebur, E. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 20, 11 (1998), 1254–1259.CrossRefGoogle Scholar
  21. 84.
    Jiang, B., Zhang, L., Lu, H., Yang, C., and Yang, M.-H. Saliency detection via absorbing Markov chain. In IEEE International Conference on Computer Vision (ICCV) (2013).Google Scholar
  22. 85.
    Jiang, H., Wang, J., Yuan, Z., Wu, Y., Zheng, N., and Li, S. Salient object detection: A discriminative regional feature integration approach. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013).Google Scholar
  23. 87.
    Judd, T., Ehinger, K., Durand, F., and Torralba, A. Learning to predict where humans look. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2009).Google Scholar
  24. 89.
    Karpathy, A., and Fei-Fei, L. Deep visual-semantic alignments for generating image descriptions. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015).Google Scholar
  25. 90.
    Kaufman, E., Lord, M., Reese, T., and Volkmann, J. The discrimination of visual number. The American Journal of Psychology (1949), 498–525.CrossRefGoogle Scholar
  26. 92.
    Kellnhofer, P., Didyk, P., Myszkowski, K., Hefeeda, M. M., Seidel, H.-P., and Matusik, W. Gazestereo3d: seamless disparity manipulations. ACM Transactions on Graphics (TOG) 35, 4 (2016), 68.Google Scholar
  27. 94.
    Kimchi, R., and Peterson, M. A. Figure-ground segmentation can occur without attention. Psychological Science 19, 7 (2008), 660–668.CrossRefGoogle Scholar
  28. 95.
    Koch, C., and Ullman, S. Shifts in selective visual attention: towards the underlying neural circuitry. In Matters of intelligence. Springer, 1987, pp. 115–141.Google Scholar
  29. 96.
    Koffka, K. Principles of Gestalt psychology. Harcourt, Brace New York, 1935.Google Scholar
  30. 98.
    Kourtzi, Z., and Kanwisher, N. Representation of perceived object shape by the human lateral occipital complex. Science 293, 5534 (2001), 1506–1509.CrossRefGoogle Scholar
  31. 102.
    Larochelle, H., and Hinton, G. E. Learning to combine foveal glimpses with a third-order Boltzmann machine. In Advances in Neural Information Processing Systems (NIPS) (2010).Google Scholar
  32. 104.
    Lee, Y. J., Ghosh, J., and Grauman, K. Discovering important people and objects for egocentric video summarization. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012).Google Scholar
  33. 112.
    Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., and Shum, H.-Y. Learning to detect a salient object. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 33, 2 (2011), 353–367.Google Scholar
  34. 113.
    Lu, S., Mahadevan, V., and Vasconcelos, N. Learning optimal seeds for diffusion-based salient object detection. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014).Google Scholar
  35. 118.
    Mahadevan, V., and Vasconcelos, N. Saliency-based discriminant tracking. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2009).Google Scholar
  36. 120.
    Malmberg, F., Strand, R., Zhang, J., and Sclaroff, S. The Boolean map distance: Theory and efficient computation. In International Conference on Discrete Geometry for Computer Imagery (2017), Springer, pp. 335–346.Google Scholar
  37. 125.
    Mathe, S., and Sminchisescu, C. Dynamic eye movement datasets and learnt saliency models for visual action recognition. In European Conference on Computer Vision (ECCV) (2012).CrossRefGoogle Scholar
  38. 126.
    Mazza, V., Turatto, M., and Umilta, C. Foreground–background segmentation and attention: A change blindness study. Psychological Research 69, 3 (2005), 201–210.CrossRefGoogle Scholar
  39. 127.
    Mnih, V., Heess, N., Graves, A., et al. Recurrent models of visual attention. In Advances in Neural Information Processing Systems (NIPS) (2014).Google Scholar
  40. 129.
    Nath, S. K., Palaniappan, K., and Bunyak, F. Cell segmentation using coupled level sets and graph-vertex coloring. In Medical Image Computing and Computer-Assisted Intervention (MICCAI) (2006).CrossRefGoogle Scholar
  41. 131.
    Ninassi, A., Le Meur, O., Le Callet, P., and Barba, D. Does where you gaze on an image affect your perception of quality? applying visual attention to image quality metric. In IEEE International Conference on Image Processing (ICIP) (2007).Google Scholar
  42. 133.
    Palmer, S. E. Vision Science: Photons to Phenomenology. The MIT press, 1999.Google Scholar
  43. 145.
    Rubin, E. Figure and ground. Readings in Perception (1958), 194–203.Google Scholar
  44. 146.
    Rubinstein, M., Shamir, A., and Avidan, S. Improved seam carving for video retargeting. ACM Transactions on Graphics (TOG) 27, 3 (2008), 16.Google Scholar
  45. 150.
    Rutishauser, U., Walther, D., Koch, C., and Perona, P. Is bottom-up attention useful for object recognition? In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2004).Google Scholar
  46. 152.
    Scharfenberger, C., Waslander, S. L., Zelek, J. S., and Clausi, D. A. Existence detection of objects in images for robot vision using saliency histogram features. In International Conference on Computer and Robot Vision (2013).Google Scholar
  47. 158.
    Shen, X., and Wu, Y. A unified approach to salient object detection via low rank matrix recovery. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012).Google Scholar
  48. 166.
    Strand, R., Ciesielski, K. C., Malmberg, F., and Saha, P. K. The minimum barrier distance. Computer Vision and Image Understanding (CVIU) 117, 4 (2013), 429–437.CrossRefGoogle Scholar
  49. 168.
    Sugano, Y., and Bulling, A. Self-calibrating head-mounted eye trackers using egocentric visual saliency. In Annual ACM Symposium on User Interface Software & Technology (2015).Google Scholar
  50. 169.
    Sugano, Y., Matsushita, Y., and Sato, Y. Appearance-based gaze estimation using visual saliency. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 35, 2 (2013), 329–341.CrossRefGoogle Scholar
  51. 187.
    Wei, Y., Wen, F., Zhu, W., and Sun, J. Geodesic saliency using background priors. In European Conference on Computer Vision (ECCV) (2012).CrossRefGoogle Scholar
  52. 189.
    Wolfe, J. M., and Horowitz, T. S. What attributes guide the deployment of visual attention and how do they do it? Nature Reviews Neuroscience 5, 6 (2004), 495–501.CrossRefGoogle Scholar
  53. 191.
    Xiong, B., and Grauman, K. Detecting snap points in egocentric video with a web photo prior. In European Conference on Computer Vision (ECCV) (2014).Google Scholar
  54. 194.
    Yang, C., Zhang, L., Lu, H., Ruan, X., and Yang, M.-H. Saliency detection via graph-based manifold ranking. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013).Google Scholar
  55. 199.
    Zhang, J., Ma, S., Sameki, M., Sclaroff, S., Betke, M., Lin, Z., Shen, X., Price, B., and Mĕch, R. Salient object subitizing. International Journal of Computer Vision 124, 2 (2017), 169–186.Google Scholar
  56. 200.
    Zhang, J., Ma, S., Sameki, M., Sclaroff, S., Betke, M., Lin, Z., Shen, X., Price, B., and Měch, R. Salient object subitizing. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015).Google Scholar
  57. 201.
    Zhang, J., and Sclaroff, S. saliency detection: a Boolean map approach. In IEEE International Conference on Computer Vision (ICCV) (2013).Google Scholar
  58. 202.
    Zhang, J., and Sclaroff, S. Exploiting surroundedness for saliency detection: a Boolean map approach. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 38, 5 (2016), 889–902.CrossRefGoogle Scholar
  59. 203.
    Zhang, J., Sclaroff, S., Lin, Z., Shen, X., Price, B., and Měch, R. Minimum barrier salient object detection at 80 fps. In IEEE International Conference on Computer Vision(ICCV) (2015).Google Scholar
  60. 207.
    Zhu, J.-Y., Wu, J., Wei, Y., Chang, E., and Tu, Z. Unsupervised object class discovery via saliency-guided multiple class learning. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012).Google Scholar
  61. 208.
    Zhu, W., Liang, S., Wei, Y., and Sun, J. Saliency optimization from robust background detection. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014).Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Jianming Zhang
    • 1
  • Filip Malmberg
    • 2
  • Stan Sclaroff
    • 3
  1. 1.Adobe Inc.San JoseUSA
  2. 2.Centre for Image AnalysisUppsala UniversityUppsalaSweden
  3. 3.Department of Computer ScienceBoston UniversityBostonUSA

Personalised recommendations