Video Saliency Modulation in the HSI Color Space for Drawing Gaze

  • Tao Shi
  • Akihiro Sugimoto
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8333)


We propose a method for drawing gaze to a given target in videos, by modulating the value of pixels based on the saliency map. The change of pixel values is described by enhancement maps, which are weighted combination of center-surround difference maps of intensity channel and two color opponency channels. Enhancement maps are applied to each video frame in the HSI color space to increase saliency in the target region, and to decrease that in the background. The TLD tracker is employed for tracking the target over frames. Saliency map is used to control the strength of modulation. Moreover, a pre-enhancement step is introduced for accelerating computation, and a post-processing module helps to eliminate flicker. Experimental results show that this method is effective in drawing attention of subjects, but the problem of flicker may rise in minor cases.


visual focus of attention saliency video modulation gaze navigation 


  1. 1.
    Bailey, R., McNamara, A., Sudarsanam, N., Grimm, C.: Subtle gaze direction. ACM Trans. on Graphics 28(4), 1–14 (2009)CrossRefGoogle Scholar
  2. 2.
    Greenspan, H., Belongie, S., Goodman, R., Perona, P., Rakshit, S., Anderson, C.: Overcomplete steerable pyramid filters and rotation invariance. In: Proc. of IEEE Conf. on CVPR, pp. 222–228 (1994)Google Scholar
  3. 3.
    Hagiwara, A., Sugimoto, A., Kawamoto, K.: Saliency-based image editing for guiding visual attention. In: Proc. of the 1st Int. Workshop on Pervasive Eye Tracking & Mobile Eye-Based Interaction, pp. 43–48 (2011)Google Scholar
  4. 4.
    Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. Advances in Neural Information Processing Systems 19, 545–552 (2007)Google Scholar
  5. 5.
    Huang, C., Liu, Q., Yu, S.: Regions of interest extraction from color image based on visual saliency. The Journal of Supercomputing 58(1), 20–33 (2010)CrossRefGoogle Scholar
  6. 6.
    Itti, L., Koch, C., Niebur, E.: A Model of saliency-based visual attention for rapid scene analysis. IEEE Trans. on PAMI 20(11), 1254–1259 (1998)CrossRefGoogle Scholar
  7. 7.
    Itti, L., Dhavale, N., Pighin, F.: Realistic avatar eye and head animation using a neurobiological model of visual attention. In: Proc. of SPIE. Applications and Science of Neural Networks, Fuzzy Systems, and Evolutionary Computation VI, vol. 5200, pp. 64–78 (2004)Google Scholar
  8. 8.
    Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking-Learning-Detection. IEEE Trans. on PAMI 6(1), 1–14 (2011)Google Scholar
  9. 9.
    Koch, C., Ullman, S.: Shifts in selective visual attention: towards the underlying neural circuitry. Matters of Intelligence 188, 115–141 (1987)CrossRefGoogle Scholar
  10. 10.
    Ledley, R., Buas, M., Golab, T.: Fundamentals of true-color image processing. In: Proc. of the 10th ICPR, pp. 791–795 (1990)Google Scholar
  11. 11.
    Mendez, E., Feiner, S., Schmalstieg, D.: Focus and context in mixed reality by modulating first order salient features. In: Proc. of the 10th Int. Symposium on Smart Graphics, pp. 232–243 (2010)Google Scholar
  12. 12.
    Reichardt, W.: Evaluation of optical motion information by movement detectors. Journal of comparative physiology. A, Sensory, Neural, and Behavioral Physiology 161(4), 533–547 (1987)Google Scholar
  13. 13.
    Veas, E., Mendez, E., Feiner, S., Schmalstieg, D.: Directing attention and influencing memory with visual saliency modulation. In: Proc. of the SIGCHI Conf. on Human Factors in Computing Systems, pp. 1471–1480 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Tao Shi
    • 1
  • Akihiro Sugimoto
    • 1
  1. 1.National Institute of InformaticsChiyoda-kuJapan

Personalised recommendations