Advertisement

Learning Shape from Defocus

  • Paolo Favaro
  • Stefano Soatto
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2351)

Abstract

We present a novel method for inferring three-dimensional shape from a collection of defocused images. It is based on the observation that defocused images are the null-space of certain linear operators that depend on the three-dimensional shape of the scene as well as on the optics of the camera. Unlike most current work based on inverting the imaging model to recover the “deblurred” image and the shape of the scene, we approach the problem from a new angle by collecting a number of deblurred images, and estimating the operator that spans their left null space directly. This is done using a singular value decomposition. Since the operator depends on the depth of the scene, we repeat the procedure for a number of different depths. Once this is done, depth can be recovered in real time: the new image is projected onto each null-space, and the depth that results in the smallest residual is chosen. The most salient feature of this algorithm is its robustness: not only can one learn the operators with one camera and then use them to successfully retrieve depth from images taken with another camera, but one can even learn the operators from simulated images, and use them to retrieve depth from real images. Thus we train the system with synthetic patterns, and then use it on real data without knowledge of the optics of the camera. Another attractive feature is that the algorithm does not rely on a discretization or an approximation of the radiance of the scene (the “deblurred” image). In fact, the operator we recover is finite-dimensional, but it arises as the orthogonal projector of a semi-infinite operator that maps square-integrable radiance distributions onto images. Thus, the radiance is never approximated or represented via a finite set of filters. Instead, the rank of the operator learned from real data provides an estimate of the intrinsic dimensionality of the radiance distribution of real images. The algorithm is optimal in the sense of \( \mathcal{L}^2 \) and can be implemented in real time.

Keywords

Input Image Singular Value Decomposition Real Image Depth Level Blind Deconvolution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    A. N. Rajagopalan and S. Chaudhuri. A variational approach to recovering depth from defocused images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19, (no. 10):1158–64, October 1997.CrossRefGoogle Scholar
  2. 2.
    W. Boothby. Introduction to Differentiable Manifolds and Riemannian Geometry. Academic Press, 1986.Google Scholar
  3. 3.
    S. Chaudhuri and A. N. Rajagopalan. Depth from defocus: a real aperture imaging approach. Springer Verlag, 1999.Google Scholar
  4. 4.
    J. Ens and P. Lawrence. An investigation of methods for determining depth from focus. IEEE Trans. Pattern Anal. Mach. Intell., 15:97–108, 1993.CrossRefGoogle Scholar
  5. 5.
    P. Favaro and S. Soatto. Shape and reflectance estimation from the information divergence of blurred images. In European Conference on Computer Vision, pages 755–768, June 2000.Google Scholar
  6. 6.
    G. Golub and V. Pereyra. The differentiation of pseudo-inverses and nonlinear least-squares problems whose variables separate. SIAM J. Numer. Anal., 10(2):413–532, 1973.zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    R. A. Jarvis. A perspective on range finding techniques for computer vision. In IEEE Trans. on Pattern Analysis and Machine Intelligence, volume 2, page 122:139, March 1983.Google Scholar
  8. 8.
    H. Jin and P. Favaro. A variational approach to shape from defocus. In Proc. of the European Conference on Computer Vision, volume in presss, May 2002.Google Scholar
  9. 9.
    S. Nayar and Y. Nakagawa. Shape from focus. IEEE Trans. Pattern Anal. Mach. Intell., 16(8):824–831, 1994.CrossRefGoogle Scholar
  10. 10.
    A. Pentland. A new sense for depth of field. IEEE Trans. PatternAnal. Mach. Intell., 9:523–531, 1987.CrossRefGoogle Scholar
  11. 11.
    S. Soatto and P. Favaro. A geometric approach to blind deconvolution with application to shape from defocus. In Intl. Conf. on Computer Vision and Pattern Recognition, pages 10–17, June 2000.Google Scholar
  12. 12.
    M. Subbarao and G. Surya. Depth from defocus: a spatial domain approach. Intl. J. of Computer Vision, 13:271–294, 1994.CrossRefGoogle Scholar
  13. 13.
    M. Watanabe and S. K. Nayar. Rational filters for passive depth from defocus. Intl. J. of Comp. Vision, 27(3):203–225, 1998.CrossRefGoogle Scholar
  14. 14.
    Y. Xiong and S. Shafer. Depth from focusing and defocusing. In Proc. of the Intl. Conf. of Comp. Vision and Pat. Recogn., pages 68–73, 1993.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Paolo Favaro
    • 1
  • Stefano Soatto
    • 2
  1. 1.Department of Electrical EngineeringWashington UniversitySt. LouisUSA
  2. 2.Department of Computer ScienceUniversity of CaliforniaLos AngelesUSA

Personalised recommendations