Skip to main content
SpringerLink
Log in

Super-Resolved Fourier-Slice Refocusing in Plenoptic Cameras

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

Plenoptic cameras are a new type of sensors that capture the four-dimensional lightfield of a scene. Processing the recorded lightfield, these cameras extend the capabilities of current commercial cameras offering the possibility of focusing the scene after the shot or obtaining 3D information. Conventional photographs focused on certain planes can be obtained through projections of the four-dimensional lightfield onto two spatial dimensions. These photographs can be efficiently computed using the Fourier Slice technique, but their resolution is limited since a plenoptic camera trades off spatial resolution for angular resolution. In order to remove this limitation, several super-resolution methods have been recently developed to increase the spatial resolution of plenoptic cameras. In this paper, we study the super-resolution problem in plenoptic cameras and show how to efficiently compute super-resolved photographs using the Fourier Slice technique. We also show how several existing super-resolution methods can be seen as particular cases of this approach. Experimental results are provided to show the validity of the approach and its extension to super-resolved all-in-focus image computation and 3D processing is studied.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Adelson, E.H., Bergen, J.R.: The plenoptic function and the elements of early vision. In: Landy, M., Movshon, J.A. (eds.) In Computation Models of Visual Processing, pp. 3–20. MIT Press, Cambridge (1991)

    Google Scholar 

  2. Levoy, M., Hanrahan, P.: “Light field rendering”. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, pp. 31–42 (1996)

  3. Gortler, S.J., Grzeszczuk, R., Szelinsli, R., Cohen, M.F.: The Lumigraph. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (1996)

  4. Ives, F.: US Patent 725567 (1903)

  5. Lipmannn, G.: Epreuves reversibles donnant la sensation du relief. J. Phys. 7(4), 821–825 (1908)

    Google Scholar 

  6. Adelson, T., Wang, J.: Single lens stereo with a plenoptic camera. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 99–106 (1992)

    Article  Google Scholar 

  7. Okano, F., Arai, J., Hoshino, H., Yuyama, I.: Three dimensional video system based on integral photography. Opt. Eng. 38(6), 1072–1077 (1999)

    Article  Google Scholar 

  8. Naemura, T., Yoshida, T., Harashima, H.: 3-D computer graphics based on integral photography. Opt. Express 38(6), 255–262 (2001)

    Article  Google Scholar 

  9. Tyson, R.K.: Principles of Adaptive Optics. Academic, New York (1991)

    Google Scholar 

  10. Wilburn, B., Joshi, N., Vaish, V., Talvala, E., Antunez, E.: High performance imaging using large camera arrays. Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (1995)

  11. Ng, R.: Fourier slice photography. ACM Trans. Graph 24, 735–744 (2005)

    Article  Google Scholar 

  12. Lytro: www.lytro.com

  13. Raytrix: www.raytrix.com

  14. Pérez, F., Marichal, J.G., Rodríguez, J.M.: The discrete focal stack transform. In: Proceedings of Eusipco 08 (2008)

  15. Lumsdaine, A., Georgiev, T.: The focused plenoptic camera. In: International Conference on Computational Photography (2009)

  16. Georgiev, T., Chunev, G., Lumsdaine, A.: « Superresolution with the focused plenoptic camera, » In: Proceedings of SPIE 7873, Computational Imaging IX, (2011)

  17. Pérez, F., Lüke, J.P.: Simultaneous estimation of super-resolved depth and all-in-focus images from a plenoptic camera. In: 3D Television (2009)

  18. Bishop, T.E., Zanetti, S., Favaro, P.: Light field superresolution. In: International Conference on Computational Photography (2009)

  19. Stroebel, L., Compton, J., Current, I., Zakia, R.: Photographic Materials and Processes. Focal, Boston (1996)

    Google Scholar 

  20. Kaiser, J. F.: Nonrecursive digital filter design using I0-sinh window function. In: IEEE International Symposiun on Circuits and Systems (1974)

  21. Beerends, R.J., Morsche, H.G., Berg, J.C., Vrie, E.M.: Fourier and Laplace Transforms. Cambridge University Press, Cambridge (2003)

    Book  MATH  Google Scholar 

  22. Bailey, D.H., Swarztrauber, P.N.: The fractional Fourier transform and applications. SIAM Rev. 33, 389–404 (1991)

    Article  MATH  MathSciNet  Google Scholar 

  23. Georgiev, T., Lumsdaine, A.: Reducing plenoptic camera artifacts. Comput. Graph. Forum 29(6), 1955–1968 (2010)

    Article  Google Scholar 

  24. Ng, R.: « Digital light field photography. »PhD Thesis Stanford University (2006)

  25. Pérez, F., Pérez, A., Rodríguez, M., Magdaleno, E.: « Fourier Slice Super-resolution in plenoptic cameras, »In: International Conference on Computational Photography, Seattle (2012)

  26. Park, S.C., Park, M.K., Kang, M.G.: Super-resolution image reconstruction- a technical overview. IEEE Signal Process. Mag. 20(3), 21–36 (2003)

    Article  Google Scholar 

  27. Farsiu, S., Robinson, D., Elad, M., Mylanfar, P.: Advances and challenges in super-resolution. Int. J. Imaging Syst. Technol. 14(2), 47–57 (2004)

    Article  Google Scholar 

  28. Borman, S., Stevenson, R.: Super-resolution from image sequences—a review. In: Midwest Symposium on Circuits and Systems (1998)

  29. Elad, M., Hel-Or, Y.: A fast super-resolution algorithm for pure translational motion and common space-invariant blur. IEEE Trans. Image Process. 10(8), 1187–1193 (2001)

    Article  MATH  Google Scholar 

  30. Hansen, C.: Discrete Inverse Problems: Insight and Algorithms (Fundamentals of Algorithms). SIAM, Philadelphia (2010)

    Book  Google Scholar 

  31. van der Shaaf, A., van Hateren, J.H.: « Modelling the power spectra of natural images: statistics and information  ». Vision Res. 36(17), 2759–2770 (1996)

    Article  Google Scholar 

  32. Tan, L.: Digital Signal Processing. Academic Press, New York (2007)

    Google Scholar 

  33. Dansereau, G., Pizarro, O., Williams, S.B.: « Decoding, calibration and rectification for lenselet-based plenoptic cameras, » In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013)

  34. Wang, Z., Bovik, A., Sheik, H., Simoncelli, E.: « Image quality assessment: from error visibility to structural similarity ». IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  35. Agarwala, A., Dontcheva, M., Agrawala, M., Drucker S., Colburn, A., Curless, B., Salesin, D., Cohen, D.: Interactive digital photomontage. In: SIGGRAPH (2004)

  36. Vaquero, D., Gelfand, N., Tico, M., Kulli, K., Turk, M.: Generalized autofocus. In: IEEE Workshop on Applications of Computer Vision (2011)

  37. Nayar, K., Nakagawa, Y.: « Shape from focus, ». IEEE Trans. Pattern Anal. Mach. Intell. \(\text{ n }^\circ 16\). 16, 824–831 (1994)

Download references

Acknowledgments

The authors would like to thank R. Ng, T. Georgiev and Heidelberg University for lightfields that were used in the experimental results. This work has been partially supported by “Ayudas al Fomento de Nuevos Proyectos de Investigación” (Project 2013/0001339) of the University of La Laguna

Author information

Authors and Affiliations

  1. Departamento de Estadística, Investigación Operativa y Computación, University of La Laguna, La Laguna, Canary Islands, Spain

    F. Pérez & A. Pérez

  2. Departamento de Física Fundamental, Experimental, Electrónica y Sistemas, University of La Laguna, La Laguna, Canary Islands, Spain

    M. Rodríguez & E. Magdaleno

Authors
  1. F. Pérez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Pérez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Rodríguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. E. Magdaleno
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to F. Pérez.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pérez, F., Pérez, A., Rodríguez, M. et al. Super-Resolved Fourier-Slice Refocusing in Plenoptic Cameras. J Math Imaging Vis 52, 200–217 (2015). https://doi.org/10.1007/s10851-014-0540-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-014-0540-1

Keywords

Search

Navigation