Advertisement

Introducing More Physics into Variational Depth–from–Defocus

  • Nico PerschEmail author
  • Christopher Schroers
  • Simon Setzer
  • Joachim Weickert
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8753)

Abstract

Given an image stack that captures a static scene with different focus settings, variational depth–from–defocus methods aim at jointly estimating the underlying depth map and the sharp image. We show how one can improve existing approaches by incorporating important physical properties. Most formulations are based on an image formation model (forward operator) that explains the varying amount of blur depending on the depth. We present a novel forward operator: It approximates the thin–lens camera model from physics better than previous ones used for this task, since it preserves the maximum–minimum principle w.r.t. the unknown image intensities. This operator is embedded in a variational model that is minimised with a multiplicative variant of the Euler–Lagrange formalism. This offers two advantages: Firstly, it guarantees that the solution remains in the physically plausible positive range. Secondly, it allows a stable gradient descent evolution without the need to adapt the relaxation parameter. Experiments with synthetic and real–world images demonstrate that our model is highly robust under different initialisations. Last but not least, the experiments show that the physical constraints are essential for obtaining more accurate solutions, especially in the presence of strong depth changes.

Keywords

Focal Plane Point Spread Function Lagrange Formalism Sharp Image Blind Deconvolution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

Our research has been partly funded by the Deutsche Forschungsgemeinschaft (DFG) through a Gottfried Wilhelm Leibniz prize for Joachim Weickert and the Cluster of Excellence Multimodal Computing and Interaction.

References

  1. 1.
    Aguet, F., Van De Ville, D., Unser, M.: Model-based 2.5-D deconvolution for extended depth of field in brightfield microscopy. IEEE Trans. Image Process. 17(7), 1144–1153 (2008)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Barsky, B.A., Kosloff, T.J.: Algorithms for rendering depth of field effects in computer graphics. In: Proceedings of WSEAS International Conference on Computers, pp. 999–1010. World Scientific and Engineering Academy and Society, Heraklion, July 2008Google Scholar
  3. 3.
    Bhasin, S., Chaudhuri, S.: Depth from defocus in presence of partial self occlusion. In: Proceedings of IEEE International Conference on Computer Vision, vol. 1, pp. 488–493. Vancouver, Canada, July 2001Google Scholar
  4. 4.
    Born, M., Wolf, E.: Principles of Optics: Electromagnetic Theory of Propagation, Interference and Diffraction of Light, 4th edn. Pergamon Press, Oxford (1970)Google Scholar
  5. 5.
    Cant, R., Langensieoen, C.: Creating depth of field effects without multiple samples. In: Proceeding of IEEE International Conference on Computer Modelling and Simulation, pp. 159–164. Cambridge, UK, Mar 2012Google Scholar
  6. 6.
    Chan, T.F., Wong, C.K.: Total variation blind deconvolution. IEEE Trans. Image Process. 7, 370–375 (1998)CrossRefGoogle Scholar
  7. 7.
    Chaudhuri, S., Rajagopalan, A.: Depth from Defocus: A Real Aperture Imaging Approach. Springer, Berlin (1999)Google Scholar
  8. 8.
    Cook, R.L., Porter, T., Carpenter, L.: Distributed ray tracing. In: Computer Graphics, SIGGRAPH ’84, pp. 137–145. ACM, Minneapolis, Jul 1984Google Scholar
  9. 9.
    Favaro, P., Osher, S., Soatto, S., Vese, L.: 3D shape from anisotropic diffusion. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR, Madison, USA, Jun 2003Google Scholar
  10. 10.
    Favaro, P., Soatto, S.: Shape and radiance estimation from the information divergence of blurred images. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1842, pp. 755–768. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  11. 11.
    Favaro, P., Soatto, S., Burger, M., Osher, S.: Shape from defocus via diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 30(3), 518–531 (2008)CrossRefGoogle Scholar
  12. 12.
    Gelfand, I.M., Fomin, S.V.: Calculus of Variations. Dover, New York (2000)zbMATHGoogle Scholar
  13. 13.
    Hong, L., Yu, J., Hong, C., Sui, W.: Depth estimation from defocus images based on oriented heat-flows. In: Proceedings of IEEE International Conference on Machine Vision, pp. 212–215. Dubai, UAE (2009)Google Scholar
  14. 14.
    Jin, H., Favaro, P.: A variational approach to shape from defocus. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part II. LNCS, vol. 2351, pp. 18–30. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  15. 15.
    Luenberger, D., Ye, Y.: Linear and Nonlinear Programming, 3rd edn. Springer, New York (2008)zbMATHGoogle Scholar
  16. 16.
    Namboodiri, V.P., Chaudhuri, S.: Use of linear diffusion in depth estimation based on defocus cue. In: Chanda, B., Chandran, S., Davis, L.S. (eds.) Proceedings of Indian Conference on Computer Vision, Graphics and Image Processing, pp. 133–138. Allied Publishers Private Limited, Kolkata (2004)Google Scholar
  17. 17.
    Namboodiri, V.P., Chaudhuri, S.: On defocus, diffusion and depth estimation. Pattern Recogn. Lett. 28(3), 311–319 (2007)CrossRefGoogle Scholar
  18. 18.
    Namboodiri, V., Chaudhuri, S., Hadap, S.: Regularized depth from defocus. In: Proceedings of IEEE International Conference on Image Processing, San Diego, USA, pp. 1520–1523, Oct 2008Google Scholar
  19. 19.
    Pentland, A.P.: A new sense for depth of field. IEEE Trans. Pattern Anal. Mach. Intell. 9(4), 523–531 (1987)CrossRefGoogle Scholar
  20. 20.
    Pharr, M., Humphreys, G.: Physically Based Rendering: From Theory to Implementation. Morgan Kaufmann, San Francisco (2004)Google Scholar
  21. 21.
    Rokita, P.: Fast generation of depth of field effects in computer graphics. Comput. Graph. 17(5), 593–595 (1993)CrossRefGoogle Scholar
  22. 22.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)CrossRefzbMATHGoogle Scholar
  23. 23.
    Subbarao, M.: Parallel depth recovery by changing camera parameters. In: Proceedings of IEEE International Conference on Computer Vision, Washington, USA, pp. 149–155, Dec 1988Google Scholar
  24. 24.
    Sugimoto, S.A., Ichioka, Y.: Digital composition of images with increased depth of focus considering depth information. Appl. Optics 24(14), 2076–2080 (1985)CrossRefGoogle Scholar
  25. 25.
    Tikhonov, A.N.: Solution of incorrectly formulated problems and the regularization method. Sov. Math. Doklady 4, 1035–1038 (1963)Google Scholar
  26. 26.
    Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRefGoogle Scholar
  27. 27.
    Wei, Y., Dong, Z., Wu, C.: Global depth from defocus with fixed camera parameters. In: Proceedings of IEEE International Conference on Mechatronics and Automation, Changchun, China, pp. 1887–1892, Aug 2009Google Scholar
  28. 28.
    Welk, M., Nagy, J.G.: Variational deconvolution of multi-channel images with inequality constraints. In: Martí, J., Benedí, J.M., Mendonça, A.M., Serrat, J. (eds.) IbPRIA 2007. LNCS, vol. 4477, pp. 386–393. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  29. 29.
    Whittaker, E.T.: A new method of graduation. Proc. Edinburgh Math. Soc. 41, 65–75 (1923)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Nico Persch
    • 1
    Email author
  • Christopher Schroers
    • 1
  • Simon Setzer
    • 1
  • Joachim Weickert
    • 1
  1. 1.Mathematical Image Analysis Group, Faculty of Mathematics and Computer ScienceSaarland UniversitySaarbrückenGermany

Personalised recommendations