Advertisement

Journal of Mathematical Imaging and Vision

, Volume 52, Issue 2, pp 200–217 | Cite as

Super-Resolved Fourier-Slice Refocusing in Plenoptic Cameras

  • F. Pérez
  • A. Pérez
  • M. Rodríguez
  • E. Magdaleno
Article

Abstract

Plenoptic cameras are a new type of sensors that capture the four-dimensional lightfield of a scene. Processing the recorded lightfield, these cameras extend the capabilities of current commercial cameras offering the possibility of focusing the scene after the shot or obtaining 3D information. Conventional photographs focused on certain planes can be obtained through projections of the four-dimensional lightfield onto two spatial dimensions. These photographs can be efficiently computed using the Fourier Slice technique, but their resolution is limited since a plenoptic camera trades off spatial resolution for angular resolution. In order to remove this limitation, several super-resolution methods have been recently developed to increase the spatial resolution of plenoptic cameras. In this paper, we study the super-resolution problem in plenoptic cameras and show how to efficiently compute super-resolved photographs using the Fourier Slice technique. We also show how several existing super-resolution methods can be seen as particular cases of this approach. Experimental results are provided to show the validity of the approach and its extension to super-resolved all-in-focus image computation and 3D processing is studied.

Keywords

Lightfield processing Plenoptic cameras Refocusing Super-resolution Depth estimation 

Notes

Acknowledgments

The authors would like to thank R. Ng, T. Georgiev and Heidelberg University for lightfields that were used in the experimental results. This work has been partially supported by “Ayudas al Fomento de Nuevos Proyectos de Investigación” (Project 2013/0001339) of the University of La Laguna

References

  1. 1.
    Adelson, E.H., Bergen, J.R.: The plenoptic function and the elements of early vision. In: Landy, M., Movshon, J.A. (eds.) In Computation Models of Visual Processing, pp. 3–20. MIT Press, Cambridge (1991)Google Scholar
  2. 2.
    Levoy, M., Hanrahan, P.: “Light field rendering”. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, pp. 31–42 (1996)Google Scholar
  3. 3.
    Gortler, S.J., Grzeszczuk, R., Szelinsli, R., Cohen, M.F.: The Lumigraph. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (1996)Google Scholar
  4. 4.
    Ives, F.: US Patent 725567 (1903)Google Scholar
  5. 5.
    Lipmannn, G.: Epreuves reversibles donnant la sensation du relief. J. Phys. 7(4), 821–825 (1908)Google Scholar
  6. 6.
    Adelson, T., Wang, J.: Single lens stereo with a plenoptic camera. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 99–106 (1992)CrossRefGoogle Scholar
  7. 7.
    Okano, F., Arai, J., Hoshino, H., Yuyama, I.: Three dimensional video system based on integral photography. Opt. Eng. 38(6), 1072–1077 (1999)CrossRefGoogle Scholar
  8. 8.
    Naemura, T., Yoshida, T., Harashima, H.: 3-D computer graphics based on integral photography. Opt. Express 38(6), 255–262 (2001)CrossRefGoogle Scholar
  9. 9.
    Tyson, R.K.: Principles of Adaptive Optics. Academic, New York (1991)Google Scholar
  10. 10.
    Wilburn, B., Joshi, N., Vaish, V., Talvala, E., Antunez, E.: High performance imaging using large camera arrays. Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (1995)Google Scholar
  11. 11.
    Ng, R.: Fourier slice photography. ACM Trans. Graph 24, 735–744 (2005)CrossRefGoogle Scholar
  12. 12.
  13. 13.
    Raytrix: www.raytrix.com
  14. 14.
    Pérez, F., Marichal, J.G., Rodríguez, J.M.: The discrete focal stack transform. In: Proceedings of Eusipco 08 (2008)Google Scholar
  15. 15.
    Lumsdaine, A., Georgiev, T.: The focused plenoptic camera. In: International Conference on Computational Photography (2009)Google Scholar
  16. 16.
    Georgiev, T., Chunev, G., Lumsdaine, A.: « Superresolution with the focused plenoptic camera, » In: Proceedings of SPIE 7873, Computational Imaging IX, (2011)Google Scholar
  17. 17.
    Pérez, F., Lüke, J.P.: Simultaneous estimation of super-resolved depth and all-in-focus images from a plenoptic camera. In: 3D Television (2009)Google Scholar
  18. 18.
    Bishop, T.E., Zanetti, S., Favaro, P.: Light field superresolution. In: International Conference on Computational Photography (2009)Google Scholar
  19. 19.
    Stroebel, L., Compton, J., Current, I., Zakia, R.: Photographic Materials and Processes. Focal, Boston (1996)Google Scholar
  20. 20.
    Kaiser, J. F.: Nonrecursive digital filter design using I0-sinh window function. In: IEEE International Symposiun on Circuits and Systems (1974)Google Scholar
  21. 21.
    Beerends, R.J., Morsche, H.G., Berg, J.C., Vrie, E.M.: Fourier and Laplace Transforms. Cambridge University Press, Cambridge (2003)CrossRefMATHGoogle Scholar
  22. 22.
    Bailey, D.H., Swarztrauber, P.N.: The fractional Fourier transform and applications. SIAM Rev. 33, 389–404 (1991)CrossRefMATHMathSciNetGoogle Scholar
  23. 23.
    Georgiev, T., Lumsdaine, A.: Reducing plenoptic camera artifacts. Comput. Graph. Forum 29(6), 1955–1968 (2010)CrossRefGoogle Scholar
  24. 24.
    Ng, R.: « Digital light field photography. »PhD Thesis Stanford University (2006)Google Scholar
  25. 25.
    Pérez, F., Pérez, A., Rodríguez, M., Magdaleno, E.: « Fourier Slice Super-resolution in plenoptic cameras, »In: International Conference on Computational Photography, Seattle (2012)Google Scholar
  26. 26.
    Park, S.C., Park, M.K., Kang, M.G.: Super-resolution image reconstruction- a technical overview. IEEE Signal Process. Mag. 20(3), 21–36 (2003)CrossRefGoogle Scholar
  27. 27.
    Farsiu, S., Robinson, D., Elad, M., Mylanfar, P.: Advances and challenges in super-resolution. Int. J. Imaging Syst. Technol. 14(2), 47–57 (2004)CrossRefGoogle Scholar
  28. 28.
    Borman, S., Stevenson, R.: Super-resolution from image sequences—a review. In: Midwest Symposium on Circuits and Systems (1998)Google Scholar
  29. 29.
    Elad, M., Hel-Or, Y.: A fast super-resolution algorithm for pure translational motion and common space-invariant blur. IEEE Trans. Image Process. 10(8), 1187–1193 (2001)CrossRefMATHGoogle Scholar
  30. 30.
    Hansen, C.: Discrete Inverse Problems: Insight and Algorithms (Fundamentals of Algorithms). SIAM, Philadelphia (2010)CrossRefGoogle Scholar
  31. 31.
    van der Shaaf, A., van Hateren, J.H.: « Modelling the power spectra of natural images: statistics and information  ». Vision Res. 36(17), 2759–2770 (1996)CrossRefGoogle Scholar
  32. 32.
    Tan, L.: Digital Signal Processing. Academic Press, New York (2007)Google Scholar
  33. 33.
    Dansereau, G., Pizarro, O., Williams, S.B.: « Decoding, calibration and rectification for lenselet-based plenoptic cameras, » In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013)Google Scholar
  34. 34.
    Wang, Z., Bovik, A., Sheik, H., Simoncelli, E.: « Image quality assessment: from error visibility to structural similarity ». IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRefGoogle Scholar
  35. 35.
    Agarwala, A., Dontcheva, M., Agrawala, M., Drucker S., Colburn, A., Curless, B., Salesin, D., Cohen, D.: Interactive digital photomontage. In: SIGGRAPH (2004)Google Scholar
  36. 36.
    Vaquero, D., Gelfand, N., Tico, M., Kulli, K., Turk, M.: Generalized autofocus. In: IEEE Workshop on Applications of Computer Vision (2011)Google Scholar
  37. 37.
    Nayar, K., Nakagawa, Y.: « Shape from focus, ». IEEE Trans. Pattern Anal. Mach. Intell. \(\text{ n }^\circ 16\). 16, 824–831 (1994)Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • F. Pérez
    • 1
  • A. Pérez
    • 1
  • M. Rodríguez
    • 2
  • E. Magdaleno
    • 2
  1. 1.Departamento de Estadística, Investigación Operativa y ComputaciónUniversity of La LagunaLa LagunaSpain
  2. 2.Departamento de Física Fundamental, Experimental, Electrónica y SistemasUniversity of La LagunaLa LagunaSpain

Personalised recommendations