Skip to main content
SpringerLink
Log in

Depth estimation improvement in 3D integral imaging using an edge removal approach

  • Original Article
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

A new depth estimation method for 3D reconstruction in a synthetic aperture integral imaging framework is presented. This method removes the edges of the objects in the elemental images when the objects are in focus. This strategy aims to compensate for the noise that objects focused close to the cameras can introduce into the photo-consistency measure of objects at higher depths. Furthermore, a photo-consistency criterion is applied combining a defocus and a correspondence measure, and a depth regularization method which smooths noisy depth results for the case of object surfaces. The proposed method obtains consistent results for any type of object surfaces as well as very sharp boundaries. Experimental results show that our method reduces the noise in the object edges and gives rise to an improvement in the depth map results in relation to the other methods shown in the comparative analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Lytro redefines photography with light field cameras. Press release. (2011) (Outline). https://www.lytro.com/

  2. Arai J, Okano F, Kawakita M, Okui M, Haino Y, Yoshimura M, Furuya M, Sato M (2010) Integral three-dimensional television using a 33-megapixel imaging system. IEEE J Disp Technol 6(10):422–430

    Article  Google Scholar 

  3. Arimoto H, Javidi B (2001) Integral three-dimensional imaging with digital reconstruction. Opt Lett 26:157–159

    Article  Google Scholar 

  4. Azevedo TCS, Tavares JMRS, Vaz MAP (2009) 3d object reconstruction from uncalibrated images using and off-the-shelf camera. In: Tavares J, Jorge RN (eds) Advances in computational vision and medical image processing: methods and applications. Springer, Berlin

    Google Scholar 

  5. Azevedo TCS, Tavares JMRS, Vaz MAP (2010) Three-dimensional reconstruction and characterization of human external shapes from two-dimensional images using volumetric methods. Comput Methods Biomech Biomed Eng 13(3):359–369

    Article  Google Scholar 

  6. Bae S, Duran F (2007) Defocus magnification. Eurographics 26(3):571–579

    Google Scholar 

  7. Benton SA, Bove VM (2008) Holographic imaging. Wiley, Hoboken

    Book  Google Scholar 

  8. Boykov Y, Kolmogorov V (2004) An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans Pattern Anal Mach Intell 26(9):1124–1137

    Article  MATH  Google Scholar 

  9. Boykov Y, Veksler O, Zabih R (2001) Fast approximate energy minimization via graph cuts. IEEE Trans Pattern Anal Mach Intell 23(11):1222–1239

    Article  Google Scholar 

  10. Burckhardt CB (1968) Optimum parameters and resolution limitation of integral photography. J Opt Soc Am 58:71–76

    Article  Google Scholar 

  11. Cho M, Javidi B (2010) Three-dimensional visualization of objects in turbid water using integral imaging. J Display Technol 6(10):544–547

    Article  Google Scholar 

  12. Cho M, Mahalanobis A, Javidi B (2011) 3D passive photon counting automatic target recognition using advanced correlation filters. Opt Lett 36(6):861–863

    Article  Google Scholar 

  13. Daneshpanah M, Javidi B (2009) Profilometry and optical slicing by passive three-dimensional imaging. Opt Lett 34(7):1105–1107

    Article  Google Scholar 

  14. Espinós-Morató H, Latorre-Carmona P, Sotoca JM, Pla F, Javidi B (2017) Combining defocus and photoconsistency for depth map estimation in 3D integral imaging. In: 8th Iberian conference on pattern recognition and image analysis (IbPRIA), pp 114–121

  15. Furukawa Y, Hernandez C (2015) Multi-view stereo: a tutorial. Found Trends Comput Graph Vision 9:1–148

    Article  Google Scholar 

  16. Hong SH, Jang JS, Javidi B (2004) Three-dimensional volumetric object reconstruction using computational integral imaging. Opt Express 3(3):483–491

    Article  Google Scholar 

  17. Hongen L, Hata N, Nakajima S, Iwahara M, Sakuma I, Dohi T (2004) Surgical navigation by autostereoscopic image overlay of integral videography. IEEE Trans Inf Technol Biomed 8:114–121

    Article  Google Scholar 

  18. Igarashi Y, Murata H, Ueda M (1978) 3-D display system using a computer generated integral photograph. Jpn J Appl Phys 17(9):1683–1684

    Article  Google Scholar 

  19. Ives HE (1931) Optical properties of a Lippmann lenticuled sheet. J Opt Soc Am 21:171–176

    Article  Google Scholar 

  20. Jang JS, Javidi B (2002) Three-dimensional synthetic aperture integral imaging. Opt Lett 27:1144–1146

    Article  Google Scholar 

  21. Javidi B, Hong SH, Matoba O (2006) Multidimensional optical sensor and imaging system. Appl Opt 45:2986–2994

    Article  Google Scholar 

  22. Javidi B, Okano F, Son JY (2008) Three-dimensional imaging, visualization, and display technology. Springer, Berlin

    Google Scholar 

  23. Javidi B, Shen X, Markman A, Latorre-Carmona P, Martínez-Uso A, Sotoca JM, Pla F, Martinez-Corral M, Saavedra G, Huang YP, Stern A (2017) Multidimensional optical sensing and imaging systems (MOSIS): from macro to micro scales. Proc IEEE 105(5):850–875

    Article  Google Scholar 

  24. Kurulakos K, Seitz S (2000) A theory of shape by space carving. Int J Comput Vision 38(3):199–218

    Article  MATH  Google Scholar 

  25. Martínez-Corral M, Javidi B, Martínez-Cuenca R, Saavedra G (2004) Integral imaging with improved depth of field by use of amplitude modulated microlens array. Appl Opt 43:5806–5813

    Article  Google Scholar 

  26. Martínez-Cuenca R, Saavedra G, Martínez-Corral M, Javidi B (2009) Progress in 3-D multiperspective display by integral imaging. Proc IEEE 97:1067–1077

    Article  Google Scholar 

  27. Martínez-Uso A, Latorre-Carmona P, Sotoca JM, Pla F, Javidi B (2016) Depth estimation in integral imaging based on a maximum voting strategy. IEEE J Display Technol 12(12):1715–1723

    Google Scholar 

  28. Okano F, Akai J, Mitani K, Okui M (2006) Real-time integral imaging based on extremely high resolution video system. Proc IEEE 94:490–501

    Article  Google Scholar 

  29. Okano F, Hoshino H, Arai J, Yuyama I (1997) Real-time pickup method for a three-dimensional image based on integral photography. Appl Opt 36(7):1598–1603

    Article  Google Scholar 

  30. Okoshi T (1980) Three-dimensional displays. Proc IEEE 68(5):548–564

    Article  Google Scholar 

  31. Perduz S, García MA, Puig D (2015) Focus-aided scene segmentation. Comput Vis Image Underst 133:66–75

    Article  Google Scholar 

  32. Shin DH, Lee BG, Lee JJ (2008) Occlusion removal method of partially occluded 3d objects using sub-image block matching in computational integral imaging. Opt Express 16(21):16294–16304

    Article  Google Scholar 

  33. Sinha S, Steedly D, Szeliski R, Agrawala M, Pollefeys M (2008) Interactive 3D architectural modeling from unordered photo collections. ACM Trans Graph 27:1–10

    Article  Google Scholar 

  34. Slabaugh G, Culbertson W, Malzbender T, Stevens M, Schafer R (2004) Methods for volumetric reconstruction of visual scenes. Int J Comput Vision 57(3):179–199

    Article  Google Scholar 

  35. Sokolov AP (1911) Autostereoscopy and integral photography by professor Lippmann’s method. Moscow State University Press, Moscow

    Google Scholar 

  36. Son JY, Son WH, Kim SK, Lee KH, Javidi B (2013) Three-dimensional imaging for creating real-world-like environments. Proc IEEE 101(1):190–205

    Article  Google Scholar 

  37. Stern A, Javidi B (2006) Three-dimensional image sensing, visualization and processing using integral imaging. Proc IEEE 94:591–607

    Article  Google Scholar 

  38. Stern A, Javidi B (2006) Three-dimensional imaging aperture integral imaging. Proc IEEE 94:591–607

    Article  Google Scholar 

  39. Svoboda T, Martinec D, Pajdla T (2005) A convenient multi-camera self-calibration for virtual environments. PRESENCE: Teleoperator Virtual Environ 14(4):407–422

    Article  Google Scholar 

  40. Wang TC, Efros AE, Ramamoorthi R (2016) Depth estimation with occlusion modeling using light-field cameras. IEEE Trans Pattern Anal Mach Intell 38(11):2170–2181

    Article  Google Scholar 

  41. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  42. Wheatstone C (1838) On some remarkable, and hitherto unobserved, phenomena of binocular vision. Philos Trans R Soc Lond 128:371–394

    Article  Google Scholar 

  43. Xiao X, Javidi B, Martinez-Corral M, Stern A (2013) Advances in three-dimensional integral imaging: sensing, display, and applications. Appl Opt 52(4):546–560

    Article  Google Scholar 

  44. Zhao Y, Xiao X, Cho M, Javidi B (2011) Tracking of multiples objects in unknown background using Bayesian estimation in 3d space. J Opt Soc Am A 28(9):1935–1940

    Article  Google Scholar 

  45. Zhu X, Cohen S, Schiller S, Milanfar P (2013) Estimating spatially varying defocus blur from a single image. IEEE Trans Image Process 22(12):4879–4891

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work was supported by the Spanish Ministry of Economy and Competitiveness (MINECO) under the Projects SEOSAT (ESP2013-48458-C4-3- P) and MTM2013-48371-C2-2-PDGI, by the Generalitat Valenciana through the Project PROMETEO-II-2014-062, and by the University Jaume I through the Project UJIP11B2014-09. B. Javidi would like to acknowledge support under NSF/IIS-1422179 and ONR under N00014-17-1-2561.

Author information

Authors and Affiliations

  1. Institute of New Imaging Technologies, Universitat Jaume I, Campus del Riu Sec, s/n, 12071, Castellón de la Plana, Spain

    José M. Sotoca, Pedro Latorre-Carmona, Hector Espinos-Morato & Filiberto Pla

  2. Electrical and Computer Engineering Department, University of Connecticut, Storrs, CT, 06269-4157, USA

    Bahram Javidi

Authors
  1. José M. Sotoca
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pedro Latorre-Carmona
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hector Espinos-Morato
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Filiberto Pla
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bahram Javidi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to José M. Sotoca.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sotoca, J.M., Latorre-Carmona, P., Espinos-Morato, H. et al. Depth estimation improvement in 3D integral imaging using an edge removal approach. Pattern Anal Applic 22, 33–45 (2019). https://doi.org/10.1007/s10044-018-0721-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-018-0721-4

Keywords

Search

Navigation