Abstract
This chapter discusses the topic of emerging light field displays from a signal processing perspective. Light field displays are defined as devices which deliver continuous parallax along with the focus and binocular visual cues acting together in rivalry-free manner. In order to ensure such functionality, one has to deal with the light field, conceptualized by the plenoptic function and its adequate parametrization, sampling and reconstruction. The light field basics and the corresponding display technologies are overviewed in order to address the fundamental problems of analyzing light field displays as signal processing channels, and of capturing and representing light field visual content for driving such displays. Spectral analysis of multidimensional sampling operators is utilized to profile the displays in question, and modern sparsification approaches are employed to develop methods for high-quality light field reconstruction and rendering.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
A. Gershun, “The light field,” Journal of Mathematics and Physics, vol. 18, no. 1-4, pp. 51-151, 1939.
E. H. Adelson and J. R. Bergen, “The plenoptic function and the elements of early vision,” Computational Models of Visual Processing. MIT Press, pp. 3-20, 1991.
M. Levoy and P. Hanrahan, “Light field rendering,” in Proc. ACM SIGGRAPH, 1996, pp. 31-42.
S.J.Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen, “The Lumigraph”, in Proc. ACM SIGGRAPH, 1996, pp. 43-54.
P. Moon and D.E. Spencer, The Photic Field. MIT Press, 1981.
R. Bregović, P. T. Kovács, T. Balogh, and A. Gotchev, “Display-specific light-field analysis,” in Proc. SPIE 9117, 911710, 2014.
C. K. Liang, Y. C. Shih, and H. H. Chen, “Light field analysis for modeling image formation,” IEEE Trans. Image Process. Vol. 20, no. 2, 446–460, 2011.
R. Bregović, P. T. Kovács, and A. Gotchev, “Optimization of light field display-camera configuration based on display properties in spectral domain,” Optics Express, vol. 24, no. 3, pp. 3067-3088, Feb. 2016.
R. Bolles, H. Baker, and D. Marimont, “Epipolar-plane image analysis: An approach to determine structure from motion,” Int. J. Comput. Vis., vol. 1, no. 1, pp. 7-55, 1987.
J.-X. Chai, X. Tong, S.-C. Chan, and H.-Y. Shum, “Plenoptic sampling,” in Proc. ACM SIGGRAPH 2000, pp. 307-318.
C. Zhang and T. Chen, “Spectral analysis for sampling image-based rendering data,” IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 1, pp 1038-1050, Nov. 2003.
J. Pearson, M. Brookes, and P. Dragotti, “Plenoptic layer-based modeling for image based rendering,” IEEE Trans. Image Process., vol. 22, no. 9, pp. 3405–3419, Sep. 2013.
C. Gilliam, P.L. Dragotti, and M. Brookes, “On the spectrum of the plenoptic function,” IEEE Trans. Image Proc., vol. 23, no. 2, pp. 502-516, Feb. 2014.
D. R. Proffitt and C. Caudek, “Depth perception and the perception of events,” in Handbook of Psychology. New York, NY, 2002.
A. Stern, Y. Yitzhaky, and B. Javidi, “Perceivable Light Fields: Matching the Requirements Between the Human Visual System and Autostereoscopic 3-D Displays,” Proc IEEE vol. 102, no. 10, pp. 1571-1587, 2014.
L. Goldmann and T. Ebrahimi, “Towards reliable and reproducible 3-D video quality assessment,” in Proc. SPIE Int. Soc. Opt. Eng., vol. 8043, 2011.
A. Boev, R. Bregović, and A. Gotchev, “Signal processing for stereoscopic and multi-view 3D displays,” in Handbook of Signal Processing Systems, 2nd edition, S. Bhattacharyya, E. Deprettere, R. Leupers, and J. Takala, eds., Springer, 2013.
M. S. Banks, D. M. Hoffman, J. Kim, and G. Wetzstein, “3D Displays,” Annual Review of Vision Science, vol. 2, pp. 397-435, 2016.
D. M. Hoffman, A. R. Girshick, K. Akeley, and M. S. Banks, “Vergence–accommodation conflicts hinder visual performance and cause visual fatigue,” Journal of Vision, vol. 8, no. 33, 2008.
F. L. Kooi and M, Lucassen, “Visual comfort of binocular and 3D displays,” in Proc. SPIE 4299, Human Vision and Electronic Imaging VI, 586, 2001.
P. A. Howarth, “Potential hazards of viewing 3-D stereoscopic television, cinema and computer games: a review,” Ophthalmic and Physiological Optics, vol. 31, pp 111–122, 2011.
M. Yamaguchi, “Light-field and holographic three-dimensional displays [Invited],” J. Opt. Soc. Am. A, vol. 33, no. 12, 2348-2364, 2016.
Y. Kajiki, H. Yoshikawa, and T. Honda, “Ocular accommodation by super multi-view stereogram and 45-view stereoscopic display,” in Proc. of Third International Display Workshops (IDW), 1996.
H. Hiura, T. Mishina, J. Arai, and Y. Iwadate, “Accommodation response measurements for integral 3D image,” in Proc. SPIE 9011, 90111H, 2014.
Y. Kim, et al., “Accommodative response of integral imaging in near distance,” J. Disp. Technol. Vol. 8, no. 2, pp. 70–78, 2012.
Y. Takaki, Y. Urano, S. Kashiwada, H. Ando, and K. Nakamura, “Super multi-view windshield display for long-distance image information presentation,” Opt. Express vol. 19, no. 2, pp. 704–716, 2011.
G. Lippmann, “Epreuves reversibles Photographies integrals,” Comptes Rendus Academie des Sciences, vol. 146, pp. 446–451, 1908.
J.-H. Jung, K. Hong and B. Lee, “Effect of viewing region satisfying super multi-view Condition in Integral Imaging,” SID Symposium Digest of Technical Papers, vol. 43, pp. 883–886, 2012.
H. Deng, Q.-H. Wang, C.-G. Luo, C.-L. Liu, and C. Li, “Accommodation and convergence in integral imaging 3D display,” J. SID, vol. 22, no. 3, pp. 158–162, 2014.
H. Navarro, R. Martínez-Cuenca, G. Saavedra, M. Martínez-Corral, and B. Javidi, “3D integral imaging display by smart pseudoscopic-to-orthoscopic conversion (SPOC),” Opt. Express, vol. 18, no. 25, pp. 25573–25583, 2010.
F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. vol. 36, pp. 1598–1603, 1997.
X. Xiao, B. Javidi, M. Martinez-Corral, and A. Stern, “Advances in three-dimensional integral imaging: sensing, display, and applications [Invited],” Appl. Opt. vol. 52, no. 4, pp. 546-560, 2013.
J. S. Jang, F. Jin, and B. Javidi, “Three-dimensional integral imaging with large depth of focus by use of real and virtual image fields,” Opt. Lett. Vol. 28, no. 16, pp. 1421–1423, 2003.
S. W. Min, B. Javidi, and B. Lee, “Enhanced three-dimensional integral imaging system by use of double display devices,” Appl. Opt. vol. 42, no. 20, pp. 4186–4195, 2003.
S.- Park, J. Yeom, Y. Jeong, N. Chen, J.-Y. Hong, and B. Lee, “Recent issues on integral imaging and its applications” J. Inf. Disp., vol. 15, no. 1, pp. 37–46, 2014.
C. van Berkel and J. A. Clarke, “Characterization and optimization of 3D-LCD module design”, in Proc. SPIE, 3012, pp.179-186, 1997.
Y. Takaki, K. Tanaka, and J. Nakamura, “Super multi-view display with a lower resolution flat-panel display”, Opt. Express, vol. 19, no. 5, pp. 4129–4139, 2011.
B. Javidi, F. Okano, and J. Y. Son, Three-Dimensional Imaging, Visualization, Display. New York, NY, USA: Springer-Verlag, 2009.
J. Geng, “Three-dimensional display technologies,” Adv. Opt. Photon., vol. 5, no. 4, pp. 456–535, 2013.
T. Honda, Y. Kajiki, S. Susami, T. Hamaguchi, T. Endo, T. Hatada, and T. Fujii, “A display system for natural viewing of 3-D images,” in Three-Dimensional Television, Video and Display Technologies. Berlin, Germany: Springer-Verlag, pp. 461–487, 2002.
Y. Takaki, Y. Urano, and H. Nishio, “Motion-parallax smoothness of short-, medium-, and long-distance 3D image presentation using multi-view displays,” Opt. Express, vol. 20, no. 24, pp. 27180-27197, 2012.
Y. Takaki, “Development of super multi-view displays,” ITE Transactions on Media Technology and Applications, vol. 2, no. 1, pp. 8–14, 2014.
Y. Kajiki, H. Yoshikawa and T. Honda: “Hologram-like video images by 45-view stereoscopic display”, in Proc. SPIE, 3012, pp.154-166, 1997.
T. Honda, D. Nagai and M. Shimomatsu: “Development of 3-D display system by a fan-like array of projection optics”, in Proc. SPIE, 4660, pp.191-199, 2001.
H. Nakanuma, H. Kamei, and Y. Takaki: “Natural 3D display with 128 directional images used for human-engineering evaluation”, in Proc. SPIE, 5664, pp.28-35, 2005.
K. Kikuta and Y. Takaki: “Development of SVGA resolution 128-directional display”, in Proc. SPIE, 6490, pp.64900U-1-8, 2007.
T. Kanebako and Y. Takaki: “Time-multiplexing display module for high-density directional display”, in Proc. SPIE, 6803, pp.68030P-1-8, 2008.
Y. Takaki and N. Nago: “Multi-projection of lenticular displays to construct a 256-view super multi-view display”, Opt. Express, vol. 18, no. 8, pp.8824-8835, 2010.
T. Balogh, “The HoloVizio system,” in Proc. SPIE 6055, 12 pages, 2006.
J. T. McCrickerd and N. George, “Holographic stereogram from sequential component photographs,” Applied Physics Letters, vol. 12, no. 1, pp. 10-12, 1968.
D. J. DeBitetto, “Holographic Panoramic Stereograms Synthesized from White Light Recordings,” Applied Optics, vol. 8, no. 8, pp. 1740-1741, 1969.
M. W. Halle, “Holographic stereograms as discrete imaging systems,” in Proc. SPIE, vol. 2176, pp. 73-84, 1994.
F. Yaraş, H. Kang, and L. Onural, “Real-time phase-only color holographic video display system using LED illumination,” Applied Optics, vol. 48, no. 34, pp. H48-H53, 2009.
D. Brotherton-Ratcliffe, F. Vergnes, A. Rodin, and M. Grichine Holographic Printer. U.S. Patent 1999b; No. US7800803B2.
Zebra Imaging Inc. Company (2012) http://www.zebraimaging.com/.
X. Li, C. P. Chen, H. Gao, Z. He, Y. Xiong, H. Li, W. Hu, Z. Ye, G. He, J. Lu, and Y. Su, “Video-rate holographic display using azo-dyedoped liquid crystal,” J. Display Technol., vol. 10, pp. 438–443, 2014.
S. Tay, M. Yamamoto, and N. Peyghambarian, “An updateable holographic 3-D display based on photorefractive polymers,” SID Symp. Dig. Tech. Pap. 39, pp. 356–357, 2008.
M. Lucente, Diffraction-specific fringe computation for electro-holography, Ph.D. dissertation, Cambridge, MA, USA, 1994.
T. Yatagai, “Three-dimensional displays using computer-generated holograms,” Optics Communications, vol. 12, no. 1, pp. 43-45, 1974.
J. Mäkinen, From light fields to wavefields: Hologram generation from multiperspective images, Master’s thesis, Tampere University of Technology, Finland, 2017.
B. E. A. Saleh and M. C. Teich, Fundamentals of photonics, 2nd ed. Hoboken, N.J: John Wiley & Sons, 2007.
Q. Y. J. Smithwick, J. Barabas, D. Smalley, and V. M. Bove, Jr., “Interactive Holographic Stereograms with Accommodation Cues,” in Proc. SPIE Practical Holography XXIV: Materials and Applications, 7619, 761903, 2010.
H. Zhang, Y. Zhao, L. Cao, and G. Jin, “Fully computed holographic stereogram based algorithm for computer-generated holograms with accurate depth cues,” Opt. Express vol. 23, no. 4, 3901-3913, 2015.
Ives FE. 1903. Parallax stereogram and process of making same. US Patent No. 725,567.
D. Lanman, M. Hirsch, Y. Kim Y, R. Raskar, “Content-adaptive parallax barriers: optimizing dual-layer 3D displays using low-rank light field factorization,” ACM Trans. Graph. vol. 29, no. 6, 163: 10 pages, 2010.
G. Wetzstein, D. Lanman, M. Hirsch, R. Raskar, “Tensor displays: compressive fight field synthesis using multilayer displays with directional backlighting,” ACM Trans. Graph. vol. 31, 80: 11 pages, 2012.
G. Wetzstein, D. Lanman, W. Heidrich, R. Raskar, ”Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. vol. 30, 95: 11 pages, 2011.
F-C. Huang, K. Chen, G. Wetzstein, “The light field stereoscope: immersive computer graphics via factored near-eye light field displays with focus cues,” ACM Trans. Graph., vol. 34, 60: 12 pages, 2015.
M. Hirsch, G. Wetzstein, R. Raskar, ”A Compressive Light Field Projection System,” ACM Trans. Graph., vol. 33, 4: 12 pages, 2014.
J.H. Lee, J. Park, D. Nam, S.Y. Choi, D.S. Park, and C.Y. Kim, “Optimal projector configuration design for 300-Mpixels multi-projection 3D display,” Opt. Express vol. 21, no. 22, 26820–26835, 2013.
E. Dubois, “The sampling and reconstruction of time-varying imagery with application in video systems,” in Proc. IEEE 73, 502–522, 1985.
E. Dubois, “Video sampling and interpolation,” in The Essential Guide to Video Processing, J. Bovik, ed., Academic Press, 2009.
P.Q. Nguyen and D. Stehlé, “Low-dimensional lattice basis reduction revisited,” ACM Trans. Algorithms, vol. 5, no. 4, 46 pages, 2009.
F. Aurenhammer, “Voronoi diagrams – A survey of a fundamental geometric data structure,” ACM Computing Surveys vol. 23, pp. 245-405, 1991.
E.B. Tadmor and R.E. Miller, Modeling Materials: Continuum, Atomistic and Multiscale Techniques, Cambridge University, 2011.
P. T. Kovács, K. Lackner, A. Barsi, A. Balázs, A. Boev, R. Bregović, and A. Gotchev, “Measurement of perceived spatial resolution in 3D light-field displays,” in Proc. IEEE Int. Conf. Image Processing, Paris, France, pp. 768–772, Oct. 2014.
P. T. Kovács, R. Bregović, A. Boev, A. Barsi, and A. Gotchev, “Quantifying spatial and angular resolution of 3D light-field displays,” IEEE Journal of Selected Topics in Signal Processing, vol. 11, no. 7, pp. 1213-1222, Oct. 2017.
Z. Lin and H.-Y. Shum, “A geometric analysis of light field rendering,” Int’l J. of Computer Vision, vol. 58, no. 2, pp. 121–138, 2004.
R. Ng, “Fourier Slice Photography,” in Proc. ACM SIGGRAPH, pp. 735–744, July 2005.
I. Tosic and K. Berkner, “Light Field Scale-Depth Space Transform for Dense Depth Estimation,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 441–448, June 2014.
K. Yücer, A. Sorkine-Hornung, O. Wang, and O. Sorkine-Hornung, “Efficient 3D Object Segmentation from Densely Sampled Light Fields with Applications to 3D Reconstruction,” ACM Trans. on Graphics, vol. 35, no. 3, 2016.
M. Tanimoto, “Overview of FTV (free-viewpoint television),” in Proc. IEEE Conf. Multimedia and Expo (ICME 2009), pp. 1552–1553, June 2009.
J. Jurik, T. Burnett, M. Klug, and P. Debevec, “Geometry-Corrected Light Field Rendering for Creating a Holographic Stereogram,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 9–13, 2012.
H. Shum, S. Chan, and S. Kang, Image-Based Rendering. Springer- Verlag, 2007.
S. Vagharshakyan, R. Bregovic, and A. Gotchev, "Light Field Reconstruction Using Shearlet Transform," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no.1, pp. 133-147, Jan. 2018.
C. Buehler, M. Bosse, L. McMillan, S. Gortler, and M. Cohen, “Unstructured lumigraph rendering,” in Proc. 28th Conf. on Computer Graphics and Interactive Techniques, pp. 425-432, 2001.
S. Fuhrmann, F. Langguth and M. Goesele “MVE – A Multi-View Reconstruction Environment,” in Proc EUROGRAPHICS Workshops on Graphics and Cultural Heritage, 2014.
S. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski “A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms,” in Proc. Conference on Computer Vision and Pattern Recognition (CVPR), 2006.
H. Hirschmuller, “Stereo Processing by Semiglobal Matching and Mutual Information,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 328–341, Feb. 2008.
S. N. Sinha, D. Scharstein and R. Szeliski, “Efficient High- Resolution Stereo Matching Using Local Plane Sweeps,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1582–1589, June 2014.
G. Zhang, J. Jia, T. Wong, and H. Bao, “Consistent Depth Maps Recovery from a Video Sequence,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 6, pp. 974–988, June 2009.
C. Fehn, “Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV,” in Proc. Stereoscopic Displays Appl, pp. 93-104, 2002.
J. Pearson, M. Brookes, and P. Dragotti, “Plenoptic Layer- Based Modeling for Image Based Rendering,” IEEE Trans. Image Processing, vol. 22, no. 9, pp. 3405–3419, Sept. 2013.
B. Olshausen and D. Field “Emergence of simple-cell receptive field properties by learning a sparse code for natural images”, Nature vol. 381, pp. 607-609, 1996.
D. Donoho, “Sparse Components Analysis and Optimal Atomic Decomposition”, Technical Report, Statistics, Stanford, 1998.
E. J. Candes, D. L. Donoho, Curvelets: A surprisingly effective nonadaptive representation for objects with edges. Stanford University, 1999.
E. J. Candes and D. L. Donoho, “New tight frames of curvelets and optimal representations of objects with piecewise c2 singularities,” Comm. Pure Appl. Math., vol. 57, no. 2, pp. 219–266, 2004.
G. Kutyniok, Shearlets: Multiscale analysis for multivariate data. Springer Science & Business Media, 2012.
M. Do and M. Vetterli, “The contourlet transform: an efficient directional multiresolution image representation,” IEEE Trans. Image Processing, vol. 14, no. 12, pp. 2091–2106, Dec 2005.
G.Easley, D.Labate, and W.-Q.Lim, “Optimally sparse image representations using shearlets,” in Proc. Fortieth Asilomar Conf. Signals, Systems and Computers (ACSSC ’06), pp. 974–978, Oct 2006.
G. Kutyniok and W.-Q. Lim, “Compactly supported shearlets are optimally sparse,” J. of Approximation Theory, vol. 163, no. 11, pp. 1564 – 1589, 2011.
J.-L. Starck, Y. Moudden, J. Bobin, M. Elad, and D. L. Donoho, “Morphological Component Analysis,” in Proc. SPIE 5914 Wavelets XI, 59140Q, May 2005.
J. Fadili, J.-L. Starck, M. Elad, and D. Donoho, “Mcalab: Reproducible Research in Signal and Image Decomposition and Inpainting,” IEEE Computing in Science & Engineering, vol. 12, no. 1, pp. 44–63, 2010.
L. Shi, H. Hassanieh, A. Davis, D. Katabi, and F. Durand, “Light Field Reconstruction Using Sparsity in the Continuous Fourier Domain,” ACM Trans. on Graphics, vol. 34, no. 1, 2014.
K. Marwah, G. Wetzstein, Y. Bando, and R. Raskar, “Compressive light field photography using overcomplete dictionaries and optimized projections,” ACM Transactions on Graphics, vol. 32, no. 4, pp. 1-11, 2013.
Z. Li, Image patch modeling in a light field. PhD thesis, EECS Department, University of California, Berkeley, May 2014.
D. C. Schedl, C. Birklbauer, and O. Bimber, “Directional Super-Resolution by Means of Coded Sampling and Guided Upsampling,” in Proc. IEEE Conf. Computational Photography (ICCP), pp. 1–10, 2015.
O. Johannsen, A. Sulc, and B. Goldluecke, "What Sparse Light Field Coding Reveals about Scene Structure," in Proc. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, pp. 3262-3270, 2016.
N.K. Kalantari, T.-C. Wangand and R. Ramamoorthi, “Learning-Based View Synthesis for Light Field Cameras,” ACM Trans. on Graphics, vol. 35, no. 6, 2016.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer International Publishing AG, part of Springer Nature
About this chapter
Cite this chapter
Bregovic, R., Sahin, E., Vagharshakyan, S., Gotchev, A. (2019). Signal Processing Methods for Light Field Displays. In: Bhattacharyya, S., Deprettere, E., Leupers, R., Takala, J. (eds) Handbook of Signal Processing Systems. Springer, Cham. https://doi.org/10.1007/978-3-319-91734-4_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-91734-4_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-91733-7
Online ISBN: 978-3-319-91734-4
eBook Packages: EngineeringEngineering (R0)