Skip to main content
SpringerLink
Log in

Generic Content Creation for 3D Displays

  • Chapter
  • First Online:
3D-TV System with Depth-Image-Based Rendering

Abstract

Future 3D productions in the fields of digital signage, commercials, and 3D Television will cope with the problem that they have to address a wide range of different 3D displays, ranging from glasses-based standard stereo displays to auto-stereoscopic multi-view displays or even light-field displays. The challenge will be to serve all these display types with sufficient quality and appealing content. Against this background this chapter discusses flexible solutions for 3D capture, generic 3D representation formats using depth maps, robust methods for reliable depth estimation, required preprocessing of captured multi-view footage, postprocessing of estimated depth maps, and, finally, depth-image-based rendering (DIBR) for creating missing virtual views at the display side.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Feldmann I, Schreer O, Kauff P, Schäfer R, Fei Z, Belt HJW, Divorra Escoda Ò (2009) Immersive multi-user 3D video communication. In: Proceedings of international broadcast conference (IBC 2009), Amsterdam, NL, Sept 2009

    Google Scholar 

  2. Divorra Escoda O, Civit J, Zuo F, Belt H, Feldmann I, Schreer O, Yellin E, Ijsselsteijn W, van Eijk R, Espinola D, Hagendorf P, Waizennegger W, Braspenning R (2010) Towards 3D-aware telepresence: working on technologies behind the scene. In: Proceedings of ACM conference on computer supported cooperative work (CSCW), new frontiers in telepresence, Savannah, Georgia, USA, 06–10 Feb 2010

    Google Scholar 

  3. Feldmann I, Atzpadin N, Schreer O, Pujol-Acolado J-C, Landabaso J-L, Divorra Escoda O (2009) multi-view depth estimation based on visual-hull enhanced hybrid recursive matching for 3D video conference systems. In: Proceedings of 16th international conference on image processing (ICIP 2009), Kairo, Egypt, Nov 2009

    Google Scholar 

  4. Waizenegger W, Feldmann I, Schreer O (2011) Real-time patch sweeping for high-quality depth estimation in 3D videoconferencing applications. In: SPIE 2011 conference on real-time image and video processing, San Francisco, CA, USA, 23–27 Jan 2011, Invited Paper

    Google Scholar 

  5. Pastoor S (1991) 3D-Television: a survey of recent research results on subjective requirements. Signal Process Image Commun 4(1):21–32

    Article  Google Scholar 

  6. IJsselsteijn WA, de Ridder H, Vliegen J (2000) Effects of stereoscopic filming parameters and display duration on the subjective assessment of eye strain. In: Proceedings of SPIE stereoscopic displays and virtual reality systems, San Jose, Apr 2000

    Google Scholar 

  7. Mendiburu B (2008) 3D movie making—stereoscopic digital cinema from script to screen. Elsevier, ISBN: 978-0-240-81137-6

    Google Scholar 

  8. Yeh Y–Y, Silverstein LD (1990) Limits of fusion and depth judgement in stereoscopic color pictures. Hum Factors 32(1):45–60

    Google Scholar 

  9. Holliman N (2004) Mapping perceived depth to regions of interest in stereoscopic images. In: Stereoscopic displays and applications XV, San Jose, California, Jan 2004

    Google Scholar 

  10. Jones G, Lee D, Holliman N, Ezra D (2001) Controlling perceived depth in stereoscopic images. In: Proceedings SPIE stereoscopic displays and virtual reality systems VIII, San Jose, CA, USA, Jan 2001

    Google Scholar 

  11. Woods A, Docherty T, Koch R (1993) Image distortions in stereoscopic video systems. Proc. SPIE 1915:36–48

    Article  Google Scholar 

  12. Faubert J (2000) Motion parallax, stereoscopy, and the perception of depth: practical and theoretical issues. In: Proceedings of SPIE three-dimensional video and display: devices and systems, Boston, MA, USA, pp 168–191, Nov 2000

    Google Scholar 

  13. Zilly F, Kluger J, Kauff P (2011) Production rules of 3D stereo acquisistion. In: Proceedings of the IEEE (PIEEE), special issue on 3D media and displays, vol 99, issue 4, pp 590–606, Apr 2011

    Google Scholar 

  14. Johnson RB, Jacobsen GA (2005) Advances in lenticular lens arrays for visual display. In: Current developments in lens design and optical engineering VI, proceedings of SPIE, vol 5874, Paper 5874-06, San Diego, Aug 2005

    Google Scholar 

  15. Börner R (1993) Auto-stereoscopic 3D imaging by front and rear projection and on flat panel displays. Displays 14(1):39–46

    Article  Google Scholar 

  16. Omura K, Shiwa S, Kishino F (1995) Development of lenticular stereoscopic display systems: multiple images for multiple viewers. In: Proceedings of SID 95 digest, pp 761–763

    Google Scholar 

  17. Hamagishi G et al (1995) New stereoscopic LC displays without special glasses. Proc Asia Disp 95:921–927

    Google Scholar 

  18. Dodgson NA, Moore JR, Lang SR, Martin G, Canepa P (2000) A time sequential multi-projector autostereoscopic display. J Soc Inform Display 8(2):169–176

    Article  Google Scholar 

  19. Lowe D (2004) Distinctive image features from scale invariant keypoints. IJCV 60(2):91–110

    Article  Google Scholar 

  20. Bay H, Ess A, Tuytelaars T, Van Gool L (2008) SURF: speeded up robust features. Comput Vis Image Underst (CVIU) 110(3):346–359

    Article  Google Scholar 

  21. Zilly F, Riechert C, Eisert P, Kauff P (2011) Semantic kernels binarized—a feature descriptor for fast and robust matching. In: Conference on visual media production (CVMP), London, UK, Nov 2011

    Google Scholar 

  22. Hartley R, Zisserman A (2000) Multiple view geometry in computer vision. Cambrigde University Press, Cambrigde

    Google Scholar 

  23. Fischler M, Bolles R (1980) Random sample consensus: a paradigm for model fitting applications to image analysis and automated cartography. In: Proceedings of image understanding workshop, April 1980, pp 71–88

    Google Scholar 

  24. Fusiello A, Trucco E, Verri A (2000) A compact algorithm for rectification of stereo pairs. Mach Vis Appl 12(1):16–22

    Article  Google Scholar 

  25. Mallon J, Whelan PF (2005) Projective rectification from the fundamental matrix. Image Vis Comput 23(7):643–650

    Article  Google Scholar 

  26. Wu H-H, Yu Y-H (2005) Projective rectification with reduced geometric distortion for stereo vision and stereoscopic video. J Intell Rob Syst 42:71–94

    Article  Google Scholar 

  27. Scharstein D, Szeliski R (2001) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1/2/3):7–42, Apr–June 2002. Microsoft Research Technical Report MSR-TR-2001-81, Nov 2001

    Google Scholar 

  28. Brown MZ, Burschka D, Hager GD (2003) Advances in computational stereo. IEEE Trans Pattern Anal Mach Intell (PAMI) 25(8):993–1008

    Google Scholar 

  29. Aschwanden P, Guggenbuhl W (1993) Experimental results from a comparative study on correlation-type registration algorithms. In: Forstner W, Ruwiedel St (eds) Robust computer vision. Wickmann, Karlsruhe, pp 268–289

    Google Scholar 

  30. Wegner K, Stankiewicz O (2009) Similarity measures for depth estimation. In: 3DTV conference, the true vision capture transmission and display of 3D video

    Google Scholar 

  31. Birchfield S, Tomasi C (1996) Depth discontinuities by pixel-to-pixel stereo. In: Technical report STAN-CS-TR-96-1573, Stanford University, Stanford

    Google Scholar 

  32. Belhumeur PN (1996) A Bayesian approach to binocular stereopsis. IJCV 19(3):237–260

    Article  Google Scholar 

  33. Cox IJ, Hingorani SL, Rao SB, Maggs BM (1996) A maximum likelihood stereo algorithm. CVIU 63(3):542–567

    Google Scholar 

  34. Boykov Y, Veksler O, Zabih R (2001) Fast approximate energy minimization via graph cuts. IEEE Trans Pattern Anal Mach Intell 23(11):1222–1239

    Google Scholar 

  35. Kolmogorov V, Zabih R (2001) Computing visual correspondence with occlusions using graph cuts. Proc Int Conf Comput Vis 2:508–515

    Google Scholar 

  36. Kolmogorov V, Zabih R (2005) Graph cut algorithms for binocular stereo with occlusions. In: Mathematical models in computer vision: the handbook. Springer, New York

    Google Scholar 

  37. Kolmogorov V, Zabih R (2004) What energy functions can be minimized via graph cuts? IEEE Trans Pattern Anal Mach Intell 26(2):147–159

    Article  Google Scholar 

  38. Bleyer M, Gelautz M (2007) Graph-cut-based stereo matching using image segmentation with symmetrical treatment of occlusions. Signal Process Image Commun 22(2):127–143

    Article  Google Scholar 

  39. Sun J, Shum HY, Zheng NN (2002) Stereo matching using belief propagation. ECCV

    Google Scholar 

  40. Yang Q, Wang L, Yang R, Wang S, Liao M, Nister D (2006) Real-time global stereo matching using hierachical belief propagation. In: Proceedings of British machine computer vision, 2006

    Google Scholar 

  41. Felzenswalb PF, Huttenlocher DP (2006) Efficient belief propagation for early vision. Int J Comput Vis 70(1) (October)

    Google Scholar 

  42. Brown MZ, Burschka D, Hager GD (2003) Advances in computational stereo. IEEE Trans Pattern Anal Mach Intell 25(8):993–1008

    Article  Google Scholar 

  43. Kopf J, Cohen M, Lischinski D, Uyttendaele M (2007) Joint bilateral upsampling. In: Proceedings of the SIGGRAPH conference on ACM Transactions on Graphics, vol 26, no 3

    Google Scholar 

  44. Riemens AK, Gangwal OP, Barenbrug B, Berretty R-PM (2009) Joint multi-step joint bilateral depth upsampling. In: Proceedings of SPIE visual communications and image processing, vol 7257, article M, Jan 2009

    Google Scholar 

  45. Atzpadin N, Kauff P, Schreer O (2004) Stereo analysis by hybrid recursive matching for real-time immersive video stereo analysis by hybrid recursive matching for real-time immersive video conferencing. In: IEEE Transactions on circuits and systems for video technology, special issue on immersive telecommunications, vol 14, No. 3, pp 321–334, Jan 2004

    Google Scholar 

  46. Muscade (MUltimedia SCAlable 3D for Europe), European FP7 research project. http://www.muscade.eu/

  47. Ziegler M, Falkenhagen L, ter Horst R, Kalivasd D (1998) Evolution of stereoscopic and three-dimensional video. Signal Process Image Commun 14(1–2):173–1946

    Article  Google Scholar 

  48. Redert A, Op de Beeck M, Fehn C, IJsselsteijn W, Pollefeys M, Van Gool L, Ofek E, Sexton I, Surman P (2002) ATTEST—advanced three-dimensional television systems technologies. In: Proceedings of first international symposium on 3D data processing, visualization, and transmission, Padova, Italy, pp 313–319, June 2002

    Google Scholar 

  49. Mohr R, Buschmann R, Falkenhagen L, Van Gool L, Koch R (1998) Cumuli, panorama, and vanguard project overview. In: 3D structure from multiple images of large-scale environments, lecture notes in computer science, vol 1506/1998, pp 1–13. doi:10.1007/3-540-49437-5_1

    Google Scholar 

  50. Fehn C (2004) Depth-image based rendering (DIBR), compression and transmission for a new approach on 3D-TV. In: Proceedings of SPIE stereoscopic display and virtual reality systems XI, San Jose, CA, USA, pp 93–104, Jan 2004

    Google Scholar 

  51. Köppel M, Ndjiki-Nya P, Doshkov D, Lakshman H, Merkle P, Mueller K, Wiegand T (2010) Temporally consistent handling of disocclusions with texture synthesis for depth-image-based rendering. In: Proceedings of IEEE ICIP, Hong Kong

    Google Scholar 

  52. Ndjiki-Nya P, Köppel M, Doshkov D, Lakshman H, Merkle P, Mueller K, Wiegand T (2010) Depth-image based rendering with advanced texture synthesis. In: Proceedings of IEEE international conference on multimedia & expo, Singapore

    Google Scholar 

  53. 3D4YOU, European FP7 research project. http://www.3d4you.eu/

  54. Zilly F, Müller M, Kauff P (2010) The stereoscopic analyzer—an image-based assistance tool for stereo shooting and 3D production. In: Proceedings of ICIP 2010, special session on image processing for 3D cinema production, Hong Kong, 26–29 Sept 2010

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Image Processing Department, Fraunhofer Institute for Telecommunications—Heinrich Hertz Institute, Einsteinufer 37, 10587, Berlin, Germany

    Frederik Zilly, Marcus Müller & Peter Kauff

Authors
  1. Frederik Zilly
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marcus Müller
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peter Kauff
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Frederik Zilly .

Editor information

Editors and Affiliations

  1. , School of Electrical & Electronic, Nanyang Technological University, Nanyang Avenue 50, Singapore, 639798, Singapore

    Ce Zhu

  2. Electronic Engineering, Department of Information Science &, Zheda Road 38, Hangzhou, 310027, China, People's Republic

    Yin Zhao

  3. , Department of Information Science &, Zhejiang University, Zheda Road 38, Hangzhou, 310027, China, People's Republic

    Lu Yu

  4. Graduate School of Engineering, Department of Electrical Engineering and, Nagoya University, Nagoya, 464-8603, Japan

    Masayuki Tanimoto

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer Science+Business Media New York

About this chapter

Cite this chapter

Zilly, F., Müller, M., Kauff, P. (2013). Generic Content Creation for 3D Displays. In: Zhu, C., Zhao, Y., Yu, L., Tanimoto, M. (eds) 3D-TV System with Depth-Image-Based Rendering. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-9964-1_2

Download citation

  • DOI: https://doi.org/10.1007/978-1-4419-9964-1_2

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4419-9963-4

  • Online ISBN: 978-1-4419-9964-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation