Skip to main content
SpringerLink
Log in

Shape and Refractive Index from Single-View Spectro-Polarimetric Images

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

In this paper, we address the problem of the simultaneous recovery of the shape and refractive index of an object from a spectro-polarimetric image captured from a single view. Here, we focus on the diffuse polarisation process occuring at dielectric surfaces due to subsurface scattering and transmission from the object surface into the air. The diffuse polarisation of the reflection process is modelled by the Fresnel transmission theory. We present a method for estimating the azimuth angle of surface normals from the spectral variation of the phase of polarisation. Moreover, we estimate the zenith angle of surface normals and index of refraction simultaneously in a well-posed optimisation framework. We achieve well-posedness by introducing two additional constraints to the problem, including the surface integrability and the material dispersion equation. This yields an iterative solution which is computationally efficient due to the use of closed-form solutions for both the zenith angle and the refractive index in each iteration. To demonstrate the effectiveness of our approach, we show results of shape recovery and surface rendering for both real-world and synthetic imagery.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Algorithm 2
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  • Atkinson, G., & Hancock, E. R. (2005). Recovery of surface height using polarization from two views. In CAIP (pp. 162–170).

    Google Scholar 

  • Atkinson, G. A., & Hancock, E. R. (2005). Multi-view surface reconstruction using polarization. In IEEE International Conference on Computer Vision (ICCV’05) (Vol. 1, pp. 309–316). Washington: IEEE Comput. Soc.

    Chapter  Google Scholar 

  • Atkinson, G., & Hancock, E. (2006). Recovery of surface orientation from diffuse polarization. IEEE Transactions on Image Processing, 15(6), 1653–1664.

    Article  Google Scholar 

  • Atkinson, G., & Hancock, E. R. (2007a). Surface reconstruction using polarization and photometric stereo. In CAIP (pp. 466–473).

    Google Scholar 

  • Atkinson, G. A., & Hancock, E. R. (2007b). Shape estimation using polarization and shading from two views. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(11), 2001–2017.

    Article  Google Scholar 

  • Belhumeur, P. N., Kriegman, D. J., & Yuille, A. L. (1999). The bas-relief ambiguity. International Journal of Computer Vision, 35(1), 33–44.

    Article  Google Scholar 

  • Born, M., & Wolf, E. (1999). Principles of optics: electromagnetic theory of propagation, interference and diffraction of light (7th ed.). Cambridge: Cambridge University Press.

    Google Scholar 

  • Chen, H., & Wolff, L. (1998). Polarization phase based method for material classification in computer vision. International Journal of Computer Vision, 28(1), 73–83.

    Article  Google Scholar 

  • Coleman, T. F., & Li, Y. (1996). A reflective Newton method for minimizing a quadratic function subject to bounds on some of the variables. SIAM Journal on Optimization, 6(4), 1040–1058.

    Article  MATH  MathSciNet  Google Scholar 

  • Denes, L. J., Gottlieb, M. S., & Kaminsky, B. (1998). Acousto-optic tunable filters in imaging applications. Optical Engineering, 37, 1262.

    Article  Google Scholar 

  • Drbohlav, O., & Šára, R. (2001). Unambiguous determination of shape from photometric stereo with unknown light sources. In International conference on computer vision (Vol. 1, pp. 581–586). Los Alamitos: IEEE Comput. Soc.

    Google Scholar 

  • Frankot, R. T., & Chellappa, R. (1988). A method for enforcing integrability in shape from shading algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(4), 439–451.

    Article  MATH  Google Scholar 

  • Gonzalez, R. C., & Woods, R. E. (1992). Digital image processing (2nd ed.). Boston: Addison-Wesley/Longman.

    Google Scholar 

  • Goudail, F., Terrier, P., Takakura, Y., Bigué, L., Galland, F., & DeVlaminck, V. (2004). Target detection with a liquid-crystal-based passive stokes polarimeter. Applied Optics, 43(2), 274–282.

    Article  Google Scholar 

  • Gupta, N., Dahmani, R., & Choy, S. (2002). Acousto-optic tunable filter based visible- to near-infrared spectropolarimetric imager. Optical Engineering, 41(5), 1033–1038.

    Article  Google Scholar 

  • Hall, J. S. (1951). Some polarization measurements in astronomy. Journal of the Optical Society of America, 41(12), 963–966.

    Article  Google Scholar 

  • Harris, S. E., & Wallace, R. W. (1969). Acousto-optic tunable filter. Journal of the Optical Society of America, 59(6), 744–747.

    Article  Google Scholar 

  • Hawryshyn, C. W. (2000). Ultraviolet polarization vision in fishes: possible mechanisms for coding e-vector. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 355(1401), 1187–1190.

    Article  Google Scholar 

  • Hecht, E. (2002). Optics (4th ed.). Reading: Addison-Wesley.

    Google Scholar 

  • Kasarova, S. N., Sultanova, N. G., Ivanov, C. D., & Nikolov, I. D. (2007). Analysis of the dispersion of optical plastic materials. Optical Materials, 29(11), 1481–1490.

    Article  Google Scholar 

  • Mandel, L., & Wolf, E. (1995). Optical coherence and quantum optics. Cambridge: Cambridge University Press.

    Google Scholar 

  • Marshall, N. J., Land, M. F., King, C. A., & Cronin, T. W. (1991). The compound eyes of mantis shrimps (Crustacea, Hoplocarida, Stomatopoda). I. Compound eye structure: the detection of polarized light. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 334(1269), 33–56.

    Article  Google Scholar 

  • Miyazaki, D., Saito, M., Sato, Y., & Ikeuchi, K. (2002). Determining surface orientations of transparent objects based on polarization degrees in visible and infrared wavelengths. Journal of the Optical Society of America. A, Online, 19(4), 687–694.

    Article  Google Scholar 

  • Miyazaki, D., Tan, R. T., Hara, K., & Ikeuchi, K. (2003). Polarization-based inverse rendering from a single view. In IEEE international conference on computer vision (Vol. 2, p. 982).

    Chapter  Google Scholar 

  • Miyazaki, D., Kagesawa, M., & Ikeuchi, K. (2004). Transparent surface modeling from a pair of polarization images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(1), 73–82.

    Article  Google Scholar 

  • Nayar, S. K., Fang, X.-S., & Boult, T. (1997). Separation of reflection components using color and polarization. International Journal of Computer Vision, 21(3), 163–186.

    Article  Google Scholar 

  • Nocedal, J., & Wright, S. J. (2006). Numerical optimization (2nd ed.). New York: Springer.

    MATH  Google Scholar 

  • Oren, M., & Nayar, S. K. (1995). Generalization of the lambertian model and implications for machine vision. International Journal of Computer Vision, 14(3), 227–251.

    Article  Google Scholar 

  • Rahmann, S. (1999). Inferring 3D scene structure from a single polarization image. In SPIE proceedings on polarization and color techniques in industrial inspection (Vol. 3826, pp. 22–33).

    Chapter  Google Scholar 

  • Rahmann, S. (2000). Polarization images: a geometric interpretation for shape analysis. International Conference on Pattern Recognition, 3, 538–542.

    Google Scholar 

  • Rahmann, S., & Canterakis, N. (2001). Reconstruction of specular surfaces using polarization imaging. In IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 149–155).

    Google Scholar 

  • Sadjadi, F. A., & Chun, C. S. L. (2004). Remote sensing using passive infrared stokes parameters. Optical Engineering, 43, 2283–2291.

    Article  Google Scholar 

  • Saito, M., Kashiwagi, H., Sato, Y., & Ikeuchi, K. (1999). Measurement of surface orientations of transparent objects using polarization in highlight. In IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 1381).

    Google Scholar 

  • Schlick, C. (1994). An inexpensive brdf model for physically-based rendering. Computer Graphics Forum, 13(3), 233–246.

    Article  Google Scholar 

  • Sellmeier, W. (1871). Zur erklrung der abnormen farbenfolge im spectrum einiger substanzen. Annalen der Physik, 219(6), 272–282.

    Article  Google Scholar 

  • Shannon, C. E. (1949). Communication in the presence of noise. Proceedings of the Institute of Radio Engineers, 37(1), 10–21.

    MathSciNet  Google Scholar 

  • Stiles, W. S., & Burch, J. M. (1959). N.P.L. colour-matching investigation: final report (1958). Optica Acta 6, 1–26.

    Article  Google Scholar 

  • Thilak, V., Voelz, D. G., & Creusere, C. D. (2007). Polarization-based index of refraction and reflection angle estimation for remote sensing applications. Applied Optics, 46(30), 7527–7536.

    Article  Google Scholar 

  • Torrance, K., & Sparrow, E. (1967). Theory for off-specular reflection from roughened surfaces. Journal of the Optical Society of America, 57(9), 1105–1112.

    Article  Google Scholar 

  • Torrance, K. E., Sparrow, E. M., & Birkebak, R. C. (1966). Polarization, directional distribution, and off-specular peak phenomena in light reflected from roughened surfaces. Journal of the Optical Society of America, 56(7), 916–924.

    Article  Google Scholar 

  • Wolff, L. B. (1990). Polarization-based material classification from specular reflection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(11), 1059–1071.

    Article  Google Scholar 

  • Wolff, L. B. (1994). Diffuse-reflectance model for smooth dielectric surfaces. Journal of the Optical Society of America, 11(11), 2956–2968.

    Article  MathSciNet  Google Scholar 

  • Wolff, L. B. (1997). Polarization vision: a new sensory approach to image understanding. Image and Vision Computing, 15(2), 81–93.

    Article  MathSciNet  Google Scholar 

  • Wolff, L. B., & Andreou, A. G. (1995). Polarization camera sensors. Image and Vision Computing, 13(6), 497–510.

    Article  Google Scholar 

  • Wolff, L. B., & Boult, T. E. (1989). Polarization/radiometric based material classification. In Computer vision and pattern recognition (pp. 387–395).

    Google Scholar 

  • Wolff, L., & Boult, T. (1991). Constraining object features using a polarization reflectance model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(7), 635–657.

    Article  Google Scholar 

  • Wolff, L. B., Mancini, T. A., Pouliquen, P., & Andreou, A. G. (1997). Liquid crystal polarization camera. IEEE Transactions on Robotics and Automation, 13(2), 195–203.

    Article  Google Scholar 

  • Zhu, Q., & Shi, J. (2006). Shape from shading: recognizing the mountains through a global view. In IEEE computer society conference on computer vision and pattern recognition (Vol. 2, pp. 1839–1846). Los Alamitos: IEEE Comput. Soc.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National ICT Australia (NICTA), Locked Bag 8001, Canberra, ACT, 2601, Australia

    Cong Phuoc Huynh & Antonio Robles-Kelly

  2. Research School of Engineering, Australian National University, Canberra, ACT, 0200, Australia

    Antonio Robles-Kelly

  3. School of Inf. Tech. and Electrical Eng., UNSW@ADFA, Canberra, ACT, 2600, Australia

    Antonio Robles-Kelly

  4. Department of Computer Science, University of York, Heslington, York, YO10 5DD, UK

    Edwin R. Hancock

Authors
  1. Cong Phuoc Huynh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antonio Robles-Kelly
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Edwin R. Hancock
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cong Phuoc Huynh.

Additional information

NICTA is funded by the Australian Government as represented by the Department of Broadband, Communications and the Digital Economy and the Australian Research Council through the ICT Centre of Excellence program.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Huynh, C.P., Robles-Kelly, A. & Hancock, E.R. Shape and Refractive Index from Single-View Spectro-Polarimetric Images. Int J Comput Vis 101, 64–94 (2013). https://doi.org/10.1007/s11263-012-0546-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-012-0546-3

Keywords

Search

Navigation