Abstract
Light-field cameras have now become available in both consumer and industrial applications, and recent papers have demonstrated practical algorithms for depth recovery from a passive single-shot capture. However, current light-field depth estimation methods are designed for Lambertian objects and fail or degrade for glossy or specular surfaces. Because light-field cameras have an array of micro-lenses, the captured data allows modification of both focus and perspective viewpoints. In this paper, we develop an iterative approach to use the benefits of light-field data to estimate and remove the specular component, improving the depth estimation. The approach enables light-field data depth estimation to support both specular and diffuse scenes. We present a physically-based method that estimates one or multiple light source colors. We show our method outperforms current state-of-the-art diffuse and specular separation and depth estimation algorithms in multiple real world scenarios.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Gortler, S., Grzeszczuk, R., Szeliski, R., Cohen, M.: The lumigraph. In: ACM SIGGRAPH (1996)
Levoy, M., Hanrahan, P.: Light field rendering. In: ACM SIGGRAPH (1996)
Ng, R., Levoy, M., Bredif, M., Duval, G., Horowitz, M., Hanrahan, P.: Light field photographhy with a hand-held plenoptic camera. CSTR 2005–02 (2005)
Tao, M., Hadap, S., Malik, J., Ramamoorthi, R.: Depth from combining defocus and correspondence using light-field cameras. In: ICCV (2013)
Wanner, S., Goldluecke, B.: Globally consistent depth labeling of 4D light fields. In: CVPR (2012)
Shafer, S.: Using color to separate reflection components. Color research and applications (1985)
Mallick, S., Zickler, T., Kriegman, D., Belhumeur, P.: Beyond lambert: reconstructing specular surfaces using color. In: CVPR (2005)
Yoon, K., Choi, Y., Kweon, I.: Fast separation of reflection components using a specularity-invariant image representation. In: IEEE Image Processing (2006)
Wantanabe, M., Nayar, S.: Rational filters for passive depth from defocus. IJCV (1998)
Xiong, Y., Shafer, S.: Depth from focusing and defocusing. In: CVPR (1993)
Horn, B., Schunck, B.: Determining optical flow. Artificial Intelligence (1981)
Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Imaging Understanding Workshop (1981)
Min, D., Lu, J., Do, M.: Joint histogram based cost aggregation for stereo matching. PAMI (2013)
Kim, C., Zimmer, H., Pritch, Y., Sorkine-Hornung, A., Gross, M.: Scene reconstruction from high spatio-angular resolution light fields. In: SIGGRAPH (2013)
Lin, S., Li, Y., Kang, S.B., Tong, X., Shum, H.-Y.: Diffuse-Specular Separation and Depth Recovery from Image Sequences. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part III. LNCS, vol. 2352, pp. 210–224. Springer, Heidelberg (2002)
Jin, H., Soatto, S., Yezzi, A.J.: Multi-view stereo beyond lambert. In: CVPR (2003)
Mallick, S.P., Zickler, T.E., Belhumeur, P.N., Kriegman, D.J.: Specularity Removal in Images and Videos: A PDE Approach. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part I. LNCS, vol. 3951, pp. 550–563. Springer, Heidelberg (2006)
Park, J.: Efficient color representation for image segmentation under nonwhite illumination. SPIE (2003)
Bajscy, R., Lee, S., Leonardis, A.: Detection of diffuse and specular interface reflections and inter-reflections by color image segmentation. IJCV (1996)
Tan, R., Ikeuchi, K.: Separating reflection components of textured surfaces using a single image. PAMI (2005)
Tan, R., Quan, L., Lin, S.: Separation of highlight reflections fromEtextured surfaces. CVPR (2006)
Yang, Q., Wang, S., Ahuja, N.: Real-Time Specular Highlight Removal Using Bilateral Filtering. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 87–100. Springer, Heidelberg (2010)
Kim, H., Jin, H., Hadap, S., Kweon, I.: Specular reflection separation using dark channel prior. In: CVPR (2013)
Artusi, A., Banterle, F., Chetverikov, D.: A survey of specularity removal methods. Computer Graphics Forum (2011)
Sato, Y., Ikeuchi, K.: Temporal-color space analysis of reflection. JOSAA (1994)
Nishino, K., Zhang, Z., Ikeuchi, K.: Determining reflectance parameters and illumination distribution from a sparse set of images for view-dependent image synthesis. ICCV (2001)
Finlayson, G., Schaefer, G.: Solving for color constancy using a constrained dichromatic reflection model. IJCV (2002)
Tan, R., Nishino, K., Ikeuchi, K.: Color constancy through inverse-intensity chromaticity space. JOSA (2004)
Janoch, A., Karayev, S., Jia, Y., Barron, J., Fritz, M., Saenko, K., Darrell, T.: A category-level 3D object dataset: putting the kinect to work. In: ICCV (2011)
Pharr, M., Humphreys, G.: Physically-based rendering: from theory to implementation. Elsevier Science and Technology Books (2004)
Sun, D., Roth, S., Black, M.: Secrets of optical flow estimation and their principles. In: CVPR (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Tao, M.W., Wang, TC., Malik, J., Ramamoorthi, R. (2015). Depth Estimation for Glossy Surfaces with Light-Field Cameras. In: Agapito, L., Bronstein, M., Rother, C. (eds) Computer Vision - ECCV 2014 Workshops. ECCV 2014. Lecture Notes in Computer Science(), vol 8926. Springer, Cham. https://doi.org/10.1007/978-3-319-16181-5_41
Download citation
DOI: https://doi.org/10.1007/978-3-319-16181-5_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-16180-8
Online ISBN: 978-3-319-16181-5
eBook Packages: Computer ScienceComputer Science (R0)