Diffuse-Specular Separation and Depth Recovery from Image Sequences

  • Stephen Lin
  • Yuanzhen Li
  • Sing Bing Kang
  • Xin Tong
  • Heung-Yeung Shum
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2352)


Specular reflections present difficulties for many areas of computer vision such as stereo and segmentation. To separate specular and diffuse reflection components, previous approaches generally require accurate segmentation, regionally uniform reflectance or structured lighting. To overcome these limiting assumptions, we propose a method based on color analysis and multibaseline stereo that simultaneously estimates the separation and the true depth of specular reflections. First, pixels with a specular component are detected by a novel form of color histogram differencing that utilizes the epipolar constraint. This process uses relevant data from all the stereo images for robustness, and addresses the problem of color occlusions. Based on the Lambertian model of diffuse reflectance, stereo correspondence is then employed to compute for specular pixels their corresponding diffuse components in other views. The results of color-based detection aid the stereo correspondence, which determines both separation and true depth of specular pixels. Our approach integrates color analysis and multibaseline stereo in a synergistic manner to yield accurate separation and depth, as demonstrated by our results on synthetic and real image sequences.


Color Histogram Epipolar Geometry Continuity Constraint Scene Point True Depth 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    D.N. Bhat and S.K. Nayar. Binocular stereo in the presence of specular reflection. In ARPA, pages II:1305–1315, 1994.Google Scholar
  2. 2.
    D.N. Bhat and S.K. Nayar. Stereo in the presence of specular reflection. In ICCV, pages 1086–1092, 1995.Google Scholar
  3. 3.
    A.F. Bobick and S.S. Intille. Large occlusion stereo. IJCV, 33(3):1–20, Sept. 1999.Google Scholar
  4. 4.
    P.E. Debevec and J. Malik. Recovering high dynamic range radiance maps from photographs. Computer Graphics (SIGGRAPH), 31:369–378, 1997.Google Scholar
  5. 5.
    H. Jin, A. Yezzi, and S. Soatto. Variational multiframe stereo in the presence of specular reflections. Technical Report TR01-0017, UCLA, 2001.Google Scholar
  6. 6.
    S.B. Kang, R. Szeliski, and J. Chai. Handling occlusions in dense multi-view stereo. In CVPR, pages 103–110, Dec. 2001.Google Scholar
  7. 7.
    G.J. Klinker, S.A. Shafer, and T. Kanade. A physical approach to color image understanding. IJCV, 4(1):7–38, Jan. 1990.Google Scholar
  8. 8.
    L. Lam and C.Y. Suen. Application of majority voting to pattern recognition: An analysis of its behaviour and performance. IEEE Trans. on Systems, Man, and Cyberbetics, 27(5):553–568, 1997.CrossRefGoogle Scholar
  9. 9.
    S.W. Lee and R. Bajcsy. Detection of specularity using color and multiple views. Image and Vision Computing, 10:643–653, 1992.CrossRefGoogle Scholar
  10. 10.
    D.C. Marr and T. Poggio. A computational theory of human stereo vision. In Lucia M. Vaina, editor, From the Retina to the Neocortex: Selected Papers of David Marr, pages 263–290. Birkhäuser, Boston, MA, 1991.Google Scholar
  11. 11.
    Y. Nakamura, T. Matsura, K. Satoh, and Y. Ohta. Occlusion detectable stereo-occlusion patterns in camera matrix. In CVPR, pages 371–378, 1996.Google Scholar
  12. 12.
    S.K. Nayar, X. Fang, and T.E. Boult. Removal of specularities using color and polarization. In CVPR, pages 583–590, 1993.Google Scholar
  13. 13.
    M. Okutomi and T. Kanade. A multiple baseline stereo. IEEE PAMI, 15, 1993.Google Scholar
  14. 14.
    Y. Sato and K. Ikeuchi. Temporal-color space analysis of reflection. J. of the Opt. Soc. of America A, 11, 1994.Google Scholar
  15. 15.
    S. Shafer. Using color to separate reflection components. Color Research and Applications, 10, 1985.Google Scholar
  16. 16.
    F. Tong and B. V. Funt. Specularity removal for shape from shading. In Proc. Vision Interface, pages 98–103, 1988.Google Scholar
  17. 17.
    L.B. Wolff and T.E. Boult. Constraining object features using a polarization rflectance model. IEEE PAMI, 13, 1991.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Stephen Lin
    • 1
  • Yuanzhen Li
    • 1
    • 2
  • Sing Bing Kang
    • 3
  • Xin Tong
    • 1
  • Heung-Yeung Shum
    • 1
  1. 1.Microsoft Research, AsiaUSA
  2. 2.Chinese Academy of ScienceChina
  3. 3.Microsoft ResearchUSA

Personalised recommendations