Advertisement

Saliency Detection with Flash and No-flash Image Pairs

  • Shengfeng He
  • Rynson W. H. Lau
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8691)

Abstract

In this paper, we propose a new saliency detection method using a pair of flash and no-flash images. Our approach is inspired by two observations. First, only the foreground objects are significantly brightened by the flash as they are relatively nearer to the camera than the background. Second, the brightness variations introduced by the flash provide hints to surface orientation changes. Accordingly, the first observation is explored to form the background prior to eliminate background distraction. The second observation provides a new orientation cue to compute surface orientation contrast. These photometric cues from the two observations are independent of visual attributes like color, and they provide new and robust distinctiveness to support salient object detection. The second observation further leads to the introduction of new spatial priors to constrain the regions rendered salient to be compact both in the image plane and in 3D space. We have constructed a new flash/no-flash image dataset. Experiments on this dataset show that the proposed method successfully identifies salient objects from various challenging scenes that the state-of-the-art methods usually fail.

Keywords

Saliency detection Flash photography Background elimination Surface orientation 

Supplementary material

978-3-319-10578-9_8_MOESM1_ESM.pdf (7.1 mb)
Electronic Supplementary Material (PDF 7,274 KB)

References

  1. 1.
    Achanta, R., Hemami, S., Estrada, F., Susstrunk, S.: Frequency-tuned salient region detection. In: CVPR, pp. 1597–1604 (2009)Google Scholar
  2. 2.
    Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Susstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE TPAMI, 2274–2282 (2012)Google Scholar
  3. 3.
    Agrawal, A., Raskar, R., Nayar, S., Li, Y.: Removing photography artifacts using gradient projection and flash-exposure sampling. ACM TOG 24(3), 828–835 (2005)CrossRefGoogle Scholar
  4. 4.
    Borji, A., Sihite, D.N., Itti, L.: Salient object detection: A benchmark. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part II. LNCS, vol. 7573, pp. 414–429. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  5. 5.
    Chang, K., Liu, T., Chen, H., Lai, S.: Fusing generic objectness and visual saliency for salient object detection. In: ICCV (2011)Google Scholar
  6. 6.
    Chen, T., Cheng, M., Tan, P., Shamir, A., Hu, S.: Sketch2photo: Internet image montage. ACM TOG 28(5), 124:1–124:10 (2009)Google Scholar
  7. 7.
    Cheng, M., Zhang, G., Mitra, N., Huang, X., Hu, S.: Global contrast based salient region detection. In: CVPR, pp. 409–416 (2011)Google Scholar
  8. 8.
    Eisemann, E., Durand, F.: Flash photography enhancement via intrinsic relighting. ACM TOG 23(3), 673–678 (2004)CrossRefGoogle Scholar
  9. 9.
    Enns, J., Rensink, R.: Sensitivity to three-dimensional orientation in visual search. Psychological Science 1, 323–326 (1990)CrossRefGoogle Scholar
  10. 10.
    Goferman, S., Zelnik-Manor, L., Tal, A.: Context-aware saliency detection. In: CVPR (2010)Google Scholar
  11. 11.
    Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: NIPS, pp. 545–552 (2007)Google Scholar
  12. 12.
    He, Z., Nakayama, K.: Visual attention to surfaces in 3-d space. Proc. National Academy of Sciences 92, 11155–11159 (1995)CrossRefGoogle Scholar
  13. 13.
    Itti, L., Koch, C.: Computational modelling of visual attention. Nature Reviews Neuroscience 2(3), 194–203 (2001)CrossRefGoogle Scholar
  14. 14.
    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE TPAMI 20(11), 1254–1259 (1998)CrossRefGoogle Scholar
  15. 15.
    Jiang, H., Wang, J., Yuan, Z., Liu, T., Zheng, N.: Automatic salient object segmentation based on context and shape prior. In: BMVC (2011)Google Scholar
  16. 16.
    Koch, C., Ullman, S.: Shifts in Selective Visual Attention: Towards the Underlying Neural Circuitry. Human Neurobiology 4, 219–227 (1985)Google Scholar
  17. 17.
    Krishnan, D., Fergus, R.: Dark flash photography. ACM TOG 28(3), 96:1–96:11 (2009)Google Scholar
  18. 18.
    Liao, M., Wang, L., Yang, R., Gong, M.: Light fall-off stereo. In: CVPR, pp. 1–8 (2007)Google Scholar
  19. 19.
    Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., Shum, H.: Learning to detect a salient object. IEEE TPAMI 33(2), 353–367 (2011)CrossRefGoogle Scholar
  20. 20.
    Ma, Y., Zhang, H.: Contrast-based image attention analysis by using fuzzy growing. ACM Multimedia, 374–381 (2003)Google Scholar
  21. 21.
    Margolin, R., Tal, A., Zelnik-Manor, L.: What makes a patch distinct? In: CVPR (2013)Google Scholar
  22. 22.
    Niu, Y., Geng, Y., Li, X., Liu, F.: Leveraging stereopsis for saliency analysis. In: CVPR (2012)Google Scholar
  23. 23.
    Niu, Y., Liu, F., Li, X., Gleicher, M.: Warp propagation for video resizing. In: CVPR (2010)Google Scholar
  24. 24.
    Nothdurft, H.: Salience from feature contrast: additivity across dimensions. Vision Research 40(10), 1183–1201 (2000)CrossRefGoogle Scholar
  25. 25.
    Parkhurst, D., Law, K., Niebur, E.: Modeling the role of salience in the allocation of overt visual attention. Vision Research 42(1), 107–123 (2002)CrossRefGoogle Scholar
  26. 26.
    Perazzi, F., Krähenbühl, P., Pritch, Y., Hornung, A.: Saliency filters: Contrast based filtering for salient region detection. In: CVPR, pp. 733–740 (2012)Google Scholar
  27. 27.
    Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., Toyama, K.: Digital photography with flash and no-flash image pairs. ACM TOG 23(3), 664–672 (2004)CrossRefGoogle Scholar
  28. 28.
    Raskar, R., Tan, K., Feris, R., Yu, J., Turk, M.: Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging. ACM TOG 23(3), 679–688 (2004)CrossRefGoogle Scholar
  29. 29.
    Reinagel, P., Zador, A.: Natural scene statistics at the centre of gaze. In: Network: Computation in Neural Systems, pp. 341–350 (1999)Google Scholar
  30. 30.
    Reynolds, J., Desimone, R.: Interacting roles of attention and visual salience in V4. Neuron 37(5), 853–863 (2003)CrossRefGoogle Scholar
  31. 31.
    Sharma, G., Jurie, F., Schmid, C.: Discriminative spatial saliency for image classification. In: CVPR (2012)Google Scholar
  32. 32.
    Shi, K., Wang, K., Lu, J., Lin, L.: PISA: Pixelwise image saliency by aggregating complementary appearance contrast measures with spatial priors. In: CVPR (2013)Google Scholar
  33. 33.
    Sun, J., Li, Y., Kang, S., Shum, H.: Flash matting. ACM TOG 25(3), 772–778 (2006)CrossRefGoogle Scholar
  34. 34.
    Sun, J., Sun, J., Kang, S., Xu, Z., Tang, X., Shum, H.: Flash cut: Foreground extraction with flash and no-flash image pairs. In: CVPR (2007)Google Scholar
  35. 35.
    Toet, A.: Computational versus psychophysical bottom-up image saliency: A comparative evaluation study. IEEE TPAMI 33(11), 2131–2146 (2011)CrossRefGoogle Scholar
  36. 36.
    Valenti, R., Sebe, N., Gevers, T.: Image saliency by isocentric curvedness and color. In: ICCV (2009)Google Scholar
  37. 37.
    Wei, Y., Wen, F., Zhu, W., Sun, J.: Geodesic saliency using background priors. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part III. LNCS, vol. 7574, pp. 29–42. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  38. 38.
    Wexler, M., Ouarti, N.: Depth affects where we look. Current Biology 18(23), 1872–1876 (2008)CrossRefGoogle Scholar
  39. 39.
    Wolfe, J., Horowitz, T.: What attributes guide the deployment of visual attention and how do they do it? Nature Reviews Neuroscience 5(6), 495–501 (2004)CrossRefGoogle Scholar
  40. 40.
    Yan, Q., Xu, L., Shi, J., Jia, J.: Hierachical saliency detection. In: CVPR (2013)Google Scholar
  41. 41.
    Yang, C., Zhang, L., Lu, H., Ruan, X., Yang, M.H.: Saliency detection via graph-based manifold ranking. In: CVPR (2013)Google Scholar
  42. 42.
    Zhou, C., Troccoli, A., Pulli, K.: Robust stereo with flash and no-flash image pairs. In: CVPR, pp. 342–349 (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Shengfeng He
    • 1
  • Rynson W. H. Lau
    • 1
  1. 1.Department of Computer ScienceCity University of Hong KongHong KongChina

Personalised recommendations