Advertisement

Direct Variational Perspective Shape from Shading with Cartesian Depth Parametrisation

  • Yong Chul JuEmail author
  • Daniel Maurer
  • Michael Breuß
  • Andrés Bruhn
Conference paper
Part of the Mathematics and Visualization book series (MATHVISUAL)

Abstract

Most of today’s state-of-the-art methods for perspective shape from shading are modelled in terms of partial differential equations (PDEs) of Hamilton-Jacobi type. To improve the robustness of such methods w.r.t. noise and missing data, first approaches have recently been proposed that seek to embed the underlying PDE into a variational framework with data and smoothness term. So far, however, such methods either make use of a radial depth parametrisation that makes the regularisation hard to interpret from a geometrical viewpoint or they consider indirect smoothness terms that require additional consistency constraints to provide valid solutions. Moreover the minimisation of such frameworks is an intricate task, since the underlying energy is typically non-convex. In this chapter we address all three of the aforementioned issues. First, we propose a novel variational model that operates directly on the Cartesian depth. In contrast to existing variational methods for perspective shape from shading this refers to both the data and the smoothness term. Moreover, we employ a direct second-order regulariser with edge-preservation property. This direct regulariser yields by construction valid solutions without requiring additional consistency constraints. Finally, we also propose a novel coarse-to-fine minimisation framework based on an alternating explicit scheme. This framework allows us to avoid local minima during the minimisation and thus to improve the accuracy of the reconstruction. Experiments show the good quality of our model as well as the usefulness of the proposed numerical scheme.

Keywords

Time Step Size Explicit Scheme Data Term Radial Depth Integrability Constraint 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

This work has been partially funded by the Deutsche Forschungsgemeinschaft (DFG) as a joint project (BR 2245/3-1, BR 4372/1-1).

References

  1. 1.
    Abdelrahim, A.S.: Three-Dimensional Modeling of the Human Jaw/Teeth Using Optics and Statistics. PhD thesis, Department of Electrical and Computer Engineering, University of Louisville, Louisville (2014)Google Scholar
  2. 2.
    Abdelrahim, A.S., Abdelrahman, M.A., Abdelmunim, H., Farag, A., Miller, M.: Novel image-based 3D reconstruction of the human jaw using shape from shading and feature descriptors. In: Proceedings of the British Machine Vision Conference, pp. 1–11 (2011)Google Scholar
  3. 3.
    Abdelrahim, A.S., Abdelmunim, H., Graham, J., Farag, A.: Novel variational approach for the perspective shape from shading problem using calibrated images. In: Proceedings of the IEEE International Conference on Image Processing, pp. 2563–2566 (2013)Google Scholar
  4. 4.
    Ahmed, A., Farag, A.: A new formulation for shape from shading for non-Lambertian surfaces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1817–1824 (2006)Google Scholar
  5. 5.
    Barles, G.: Solutions de viscosité des équations de Hamilton-Jacobi. Mathématiques & Applications, vol. 17. Springer (1994)Google Scholar
  6. 6.
    Basha, T., Moses, Y. Kiryati, N.: Multi-view scene flow estimation: a view centered variational approach. Int. J. Comput. Vis. 101 (1), 6–21 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Bors, A.G., Hancock, E.R., Wilson, R.C.: Terrain analysis using radar shape-from-shading. IEEE Trans. Pattern Anal. Mach. Intell. 25 (8), 974–992 (2003)CrossRefGoogle Scholar
  8. 8.
    Breuß, M., Cristiani, E., Durou, J.-D., Falcone, M., Vogel, O.: Numerical algorithms for perspective shape from shading. Kybernetika 46 (2), 207–225 (2010)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Breuß, M., Cristiani, E., Durou, J.-D., Falcone, M., Vogel, O.: Perspective shape from shading: ambiguity analysis and numerical approximations. SIAM J. Imaging Sci. 5 (1), 311–342 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Brooks, M.J., Horn, B.K.P.: Shape and source from shading. In: Proceedings of the International Joint Conference in Artificial Intelligence, pp. 932–936 (1985)Google Scholar
  11. 11.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Proceedings of the European Conference on Computer Vision. LNCS, vol. 3024, pp. 25–36 (2004)zbMATHGoogle Scholar
  12. 12.
    Camilli, F., Prados, E.: Viscosity solution. In: Ikeuchi, K. (ed.) The Encyclopedia of Computer Vision. Springer, New York (2014)Google Scholar
  13. 13.
    Charbonnier, P., Blanc-Féraud, L., Aubert, G., Barlaud, M.: Deterministic edge-preserving regularization in computed imaging. IEEE Trans. Image Process. 6 (2), 298–311 (1997)CrossRefGoogle Scholar
  14. 14.
    Crandall, M.G., Lions, P.-L.: Viscosity solutions of Hamilton-Jacobi equations. Trans. Am. Math. Soc. 277 (1), 1–42 (1983)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Courant, R., Hilbert, D.: Methods of Mathematical Physics. Interscience Publishers, Inc., New York (1953)zbMATHGoogle Scholar
  16. 16.
    Courteille, F., Crouzil, A., Durou, J.-D., Gurdjos, P.: Towards shape from shading under realistic photographic conditions. In: Proceedings of the IEEE International Conference on Pattern Recognition, pp. 277–280 (2004)Google Scholar
  17. 17.
    Courteille, F., Crouzil, A., Durou, J.-D., Gurdjos, P.: 3D-spline reconstruction using shape from shading: spline from shading. Image Vis. Comput. 26 (4), 466–479 (2008)CrossRefzbMATHGoogle Scholar
  18. 18.
    Demetz, O., Stoll, M., Volz, S., Weickert, J., Bruhn, A.: Learning brightness transfer functions for the joint recovery of illumination changes and optical flow. In: Proceedings of the European Conference on Computer Vision. LNCS, vol. 8689, pp. 455–471 (2014)Google Scholar
  19. 19.
    Diggelen, J.V.: A photometric investigation of the slopes and heights of the ranges of hills in the maria of the moon. Bull. Astron. Inst. Neth. XI (423), 283–289 (1951)Google Scholar
  20. 20.
    Durou, J.-D., Falcone, M., Sagona, M.: Numerical methods for shape-from-shading: a new survey with benchmarks. Comput. Vis. Image Underst. 109 (1), 22–43 (2008)CrossRefGoogle Scholar
  21. 21.
    Estellers, V., Thiran, J.-P., Gabrani, M.: Surface reconstruction from microscopic images in optical lithography. IEEE Trans. Image Process. 23 (8), 3560–3573 (2014)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Frankot, R.T., Chellappa, R.: A method for enforcing integrability in shape from shading algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 10 (4), 439–451 (1988)CrossRefzbMATHGoogle Scholar
  23. 23.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2004)CrossRefzbMATHGoogle Scholar
  24. 24.
    Horn, B.K.P.: Shape from Shading: A Method for Obtaining the Shape of a Smooth Opaque Object from One View. PhD thesis, Department of Electrical Engineering, MIT, Cambridge (1970)Google Scholar
  25. 25.
    Horn, B.K.P.: Robot Vision. MIT, Cambridge (1986)Google Scholar
  26. 26.
    Horn, B.K.P., Brooks, M.J.: The variational approach to shape from shading. Comput. Vis. Graph. Image Process. 33, 174–208 (1986)CrossRefzbMATHGoogle Scholar
  27. 27.
    Horn, B.K.P., Brooks, M.J.: Shape from Shading. Artificial Intelligence Series. MIT, Cambridge (1989)zbMATHGoogle Scholar
  28. 28.
    http://www.blender.org. Last visited on 05 May 2015
  29. 29.
    Ikeuchi, K., Horn, B.K.P.: Numerical shape from shading and occluding boundaries. Artif. Intell. 17 (1–3), 141–184 (1981)CrossRefGoogle Scholar
  30. 30.
    Ju, Y.C., Breuß, M., Bruhn, A.: Variational perspective shape from shading. In: Proceedings of the International Conference on Scale Space and Variational Methods in Computer Vision. LNCS, vol. 9087, pp. 538–550 (2015)MathSciNetGoogle Scholar
  31. 31.
    Kimmel, R., Siddiqi, K, Kimia, B.B., Bruckstein, A.M.: Shape from shading: level set propagation and viscosity solutions. Int. J. Comput. Vis. 16, 107–133 (1995)CrossRefGoogle Scholar
  32. 32.
    Lysaker, M., Lundervold, A., Tai, X.-C.: Noise removal using fourth-order partial differential equation with applications to medical magnetic resonance images in space and time. IEEE Trans. Image Process. 12 (12), 1057–1590 (2003)CrossRefzbMATHGoogle Scholar
  33. 33.
    Mecca, R., Falcone, M.: Uniqueness and approximation of a photometric shape-from-shading model. SIAM J. Imaging Sci. 6, 616–659 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Mecca, R., Wetzler, A., Bruckstein, A.M., Kimmel, R.: Near field photometric stereo with point light sources. SIAM J. Imaging Sci. 7 (4), 2732–2770 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Okatani, T., Deguchi, K.: Shape reconstruction from an endoscope image by shape from shading technique for a point light source at the projection center. Comput. Vis. Image Underst. 66, 119–131 (1997)CrossRefGoogle Scholar
  36. 36.
    Oliensis, J.: Shape from shading as a partially well-constrained problem. Comput. Vis. Graph. Image Process: Image Underst. 54 (2), 163–183 (1991)zbMATHGoogle Scholar
  37. 37.
    Oren, M., Nayar, S.: Generalization of the Lambertian model and implications for machine vision. Int. J. Comput. Vis. 14 (3), 227–251 (1995)CrossRefGoogle Scholar
  38. 38.
    Phong, B.T.: Illumination for computer-generated pictures. Commun. ACM 18 (6), 311–317 (1975)CrossRefGoogle Scholar
  39. 39.
    Prados, E., Faugeras, O.: “Perspective shape from shading” and viscosity solutions. In: Proceedings of the IEEE International Conference Computer Vision, pp. 826–831 (2003)Google Scholar
  40. 40.
    Prados, E., Faugeras, O.: Shape from shading: a well-posed problem? In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 870–877 (2005)Google Scholar
  41. 41.
    Prados, E., Camilli, F., Faugeras, O.: A unifying and rigorous shape from shading method adapted to realistic data and applications. J. Math. Imaging Vis. 25 (3), 307–328 (2006)MathSciNetCrossRefGoogle Scholar
  42. 42.
    Rindfleisch, T.: Photometric method for lunar topography. Photogramm. Eng. 32 (2), 262–277 (1966)Google Scholar
  43. 43.
    Robert, L., Deriche, R.: Dense depth map reconstruction: a minimization and regularization approach which preserves discontinuities. In: Proceedings of the European Conference on Computer Vision. LNCS, vol. 1064, pp. 439–451 (1996)Google Scholar
  44. 44.
    Rouy, E., Tourin, A.: A viscosity solution approach to shape-from-shading. SIAM J. Numer. Anal. 29 (3), 867–884 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  45. 45.
    Sethian, J.: Level Set Methods and Fast Marching Methods. Cambridge University Press, Cambridge (1999)zbMATHGoogle Scholar
  46. 46.
    Tankus, A., Sochen, N, Yeshurun, Y.: A new perspective [on] shape-from-shading. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 862–869 (2003)Google Scholar
  47. 47.
    Tankus, A., Sochen, N., Yeshurun, Y.: Shape-from-shading under perspective projection. Int. J. Comput. Vis. 63 (1), 21–43 (2005)CrossRefGoogle Scholar
  48. 48.
    The Stanford 3D Scanning Repository, http://graphics.stanford.edu/data/3Dscanrep/. Last visited on 05 May 2015
  49. 49.
    Vogel, O., Bruhn, A., Weickert, J., Didas, S.: Direct shape-from-shading with adaptive higher order regularisation. In: Proceedings of the International Conference on Scale Space and Variational Methods in Computer Vision. LNCS, vol. 4485, pp. 871–882 (2007)Google Scholar
  50. 50.
    Vogel, O., Breuß, M., Weickert, J.: Perspective shape from shading with non-Lambertian reflectance. In: Proceedings of the German Conference on Pattern Recognition. LNCS, vol. 5096, pp. 517–526 (2008)MathSciNetGoogle Scholar
  51. 51.
    Vogel, O., Breuß, M., Leichtweis, T., Weickert, J.: Fast shape from shading for Phong-type surfaces. In: Proceedings of the International Conference on Scale Space and Variational Methods in Computer Vision. LNCS, vol. 5567, pp. 733–744 (2009)Google Scholar
  52. 52.
    Wang, G.H., Han, J.Q., Zhang, X.M.: Three-dimensional reconstruction of endoscope images by a fast shape from shading method. Meas. Sci. Technol. 20 (12) (2009)Google Scholar
  53. 53.
    Wu, C., Narasimhan, S., Jaramaz, B.: A multi-image shape-from-shading framework for near-lighting perspective endoscopes. Int. J. Comput. Vis. 86, 211–228 (2010). http://iopscience.iop.org/article/10.1088/0957-0233/20/12/125801/meta;jsessionid=2362840D33B53BD14BC41A3CE06C16D8.c5.iopscience.cld.iop.org MathSciNetCrossRefGoogle Scholar
  54. 54.
    Zhang, R., Tsai, P.-S., Cryer, J.E., Shah, M.: Shape from shading: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 21 (8), 690–706 (1999)CrossRefzbMATHGoogle Scholar
  55. 55.
    Zhang, L., Yip, A.M., Tan, C.T.: Shape from shading based on Lax-Friedrichs fast sweeping and regularization techniques with applications to document image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)Google Scholar
  56. 56.
    Zhang, L., Yip, A.M., Tan, C.T.: A restoration framework for correcting photometric and geometric distortions in camera-based document images. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1–8 (2007)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Yong Chul Ju
    • 1
    Email author
  • Daniel Maurer
    • 1
  • Michael Breuß
    • 2
  • Andrés Bruhn
    • 1
  1. 1.Institute for Visualization and Interactive SystemsUniversity of StuttgartStuttgartGermany
  2. 2.Institute for Applied Mathematics and Scientific ComputingBrandenburg University of TechnologCottbusGermany

Personalised recommendations