Skip to main content
SpringerLink
Log in
Book cover

3D Visual Content Creation, Coding and Delivery pp 41–95Cite as

3D Content Acquisition and Coding

  • Chapter
  • First Online:

Part of the book series: Signals and Communication Technology ((SCT))

Abstract

This chapter starts by addressing the impact of the inaccurate camera system alignment on the spatial reconstruction accuracy and stereo perception. An experimental study is described, using a stereoscopic camera setup and its deterministic relations derived by trigonometry, spatial model, and basic stereoscopic formulas. The significance of errors that can occur for possible cameras system setup is analyzed in order to find the appropriate settings and physical constraints of the camera system, which minimize the error. Then, the chapter presents an overview of the compression tools used in current stereoscopic and multiview video encoders. It includes the stereoscopic frame-compatible formats using spatial multiplex in the side-by-side and top-and-bottom fashion; the video plus depth representation, the layered coding approach, and multiview encoding. Furthermore, an extension of multiview video compression for the arbitrary camera arrangements is presented. The current status of MPEG exploration experiments of next-generation video codec technologies is also outlined in the last section. First, the UltraHD compression performance beyond HEVC is presented and second, the recent developments in HDR/WCG format conversion and coding are presented. Finally, the testing procedures and 3D projection formats for 360° video are addressed.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Sin-Yi, J., Chang, N.Y.-C., Chin-Chia, W., Cheng-Hei, W., Kai-Tai, S.: Error analysis and experiments of 3D reconstruction using a RGB-D sensor. In: IEEE International Conference CASE. Taipei (2014)

    Google Scholar 

  2. Belhaoua, A., Kohler, S., Hirsh, E.: Estimation of 3D reconstruction errors in a stereovision system. In: Proceedings Modeling Aspects in Optical Metrology. Mnich, Germany (2009)

    Google Scholar 

  3. Kyt, M., Nuutinen, M., Oittinen, P.: Method for measuring stereo camera depth accuracy based on stereoscopic vision. In: Proceedings of SPIE 7864. San Francisco, California, USA, (2011)

    Google Scholar 

  4. Knight, J., Reid, I.: Active visual alignment of a mobile stereo camera platform. In: IEEE International Conference on Robotics and Automation Proceedings (ICRA). San Francisco, CA (2000)

    Google Scholar 

  5. Resko, B., Baranyi, P.: Stereo camera alignment based on disparity selective cells in the visual cortex. In: IEEE 3rd International Conference on Computational Cybernetics (ICCC) (2005)

    Google Scholar 

  6. Chang, C., Chatterjee, S.: Quantization error analysis in stereovision. In: 26th Conference on Signals, Systems & Computers. Pacific Grove, CA (1992)

    Google Scholar 

  7. Fooladgar, F., Samavi, S., Soroushmehr, S.M.R.: Geometrical analysis of altitude estimation error caused by pixel quantization in stereo vision. In: 20th Iranian Conference on Electrical Engineering (ICEE). Tehran, Iran (2012)

    Google Scholar 

  8. Belhaoua, A., Kohler, S., Hirsh, E.: Error evaluation in a stereovision-based 3D reconstruction system. EURASIP J Image Video Process 2010, 1–12 (2010)

    Article  Google Scholar 

  9. Kamencay, P., Breznan, M., Jarina, R., Lukac, P., Zachariasova, M.: Improved depth map estimation from stereo images based on hybrid method. Radioengineering 21(1), 70–78 (2012)

    Google Scholar 

  10. Chang, W., Cho, K., Ryu, W., Lee, S.-Y.: Error cost function for mirror-based three-dimensional reconstruction. Electron. Lett. 50(16), 1134–1136 (2014)

    Article  Google Scholar 

  11. Fooladgar, F., Samavi, S., Soroushmehr, S.M.R., Shirani, S.: Geometrical Analysis of Localization Error in Stereo Vision Systems. IEEE Sens. J. 13(11), 4236–4246 (2013)

    Article  Google Scholar 

  12. Zhao, W., Nandhakumar, N.: Effects of camera alignment errors on stereoscopic depth estimates. Pattern Recogn. 29(12), 2115–2126 (1996)

    Article  Google Scholar 

  13. Ding, X., Xu, L., Wang, H., Wang, X., Lv, G.: Stereo depth estimation under different camera calibration and alignment errors. Appl. Opt. 50(10), 1289–1301 (2011)

    Article  Google Scholar 

  14. Bolecek, L., Ricny, V.: Influence of stereoscopic camera system alignment error on the accuracy of 3D reconstruction. Radioengineering 24(2), 610–620 (2015)

    Article  Google Scholar 

  15. Craig,J.: Introduction to Robotics: Mechanics and Control, 3rd edn. Pearson, Prentice Hall (2004)

    Google Scholar 

  16. Wheatstone, C.: Contributions to the physiology of vision I: on some remarkable and hitherto unobserved phenomena of vision. Phil. Trans. R. Soc. (Biol.) 18(13), 371–375 (1838)

    Article  Google Scholar 

  17. Angueira, P., Vega, D.L.L., Morgade, J., Velez, M.M.: Transmission of 3D Video over Broadcasting. In: Zhu, C., Zhao, Y., Yu, L., Tanimoto, M. (eds.) 3D-TV system with depth-image-based rendering, pp. 299–344. Springer, Heidelberg (2012)

    Google Scholar 

  18. Lebreton, P., Barkowsky, M., Raake, A., Callet, P.L.: 3D Video In: Möller, S., Ra-Ake, A. (eds.) Quality of Experience Advanced Concepts, Applications and Methods, pp. 299–313. Springer, Heidelberg (2014)

    Google Scholar 

  19. Liu, Y., Yang, J., Chu, R.: Objective evaluation criteria for shooting quality of stereo cameras over short distance. Radioengineering 24(1), 305–313 (2015)

    Article  Google Scholar 

  20. Merkle, P., Muller, K., Wiegand, T.: 3D Video: acquisition, coding, and display. IEEE Trans. Consumer Electro. 56(2), 946–950 (2010)

    Article  Google Scholar 

  21. Slanina, M., Kratochvil, T., Ricny, V., Bolecek, L., Kaller, O., Polak, L.: Testing QoE in different 3D HDTV technologies. Radioengineering 21(1), 445–454 (2012)

    Google Scholar 

  22. Polak, L., Kufa, J., Zach, O., Kaller, O., Bolecek, L., Slanina, M., Kratochvil, T.: Study of advanced compression tools for stereoscopic video by objective metrics. In: 26th international conference on Radioelektronika. Kosice, Slovakia (2016)

    Google Scholar 

  23. Vetro, A., Tourapis, A.M., Müller, K., Chen, T.: 3D-TV content storage and transmission. In: IEEE Trans Broadcast Spec Issue 3D-TV Horizon: Contents System Visual Percept, 57(2), 384–394 (2011)

    Google Scholar 

  24. (2012) Digital Video Broadcasting (DVB); Frame Compatible Plano-stereoscopic 3DTV. ETSI TS 101 547, v1.1.1

    Google Scholar 

  25. (2013) Features of Three-Dimensional Television Video Systems for Broadcasting. ITU-R BT 2160–4, v1.1.1

    Google Scholar 

  26. Projector guide, Simple Guide in Process of choosing a Projector. Available online at: http://projector-guide.com/3d-dlp/side-by-side-three-d

  27. Sound&Vision.: 3D Broadcast Formats. Available online at:http://www.soundandvision.com/content/3d-broadcast-formats#L5R1U1cc60v48GWT.97

  28. Minoli, D.: 3DTV content capture, encoding and transmission building the transport infrastructure for commercial services. T&F Group, Boca Raton (2010)

    Google Scholar 

  29. Livolsi, B.: What Does “3D Ready” Mean? Dispelling the Myths about 3D Projection. Available online at: http://www.projectorcentral.com/what_does_3d_ready_mean.htm. (2010)

  30. Merkle, P., Smolic, A., Muller, K., Wiegand, T.: Multi-view video plus depth representation and coding. In: 14th International Conference on ICIP. San Antonio, Texas (U.S.A.) (2007)

    Google Scholar 

  31. Müller, K., Merkle, P., Wiegand, T.: 3D video representation using depth maps. Proc. IEEE 99(4), 643–656 (2011)

    Article  Google Scholar 

  32. Vetro, A.: Frame compatible formats for 3D video distribution. In: 17th international conference on ICIP. Hong Kong, People’s Republic of China (2010)

    Google Scholar 

  33. Smolic, A., et al.: Coding algorithms for 3DTV—a survey. IEEE Trans. Circuits Syst. Video Technol. 17(11), 1606–1621 (2007)

    Article  MathSciNet  Google Scholar 

  34. Bing, B.: Next Generation Video Coding and Streaming. Wiley, New York (2015)

    Book  Google Scholar 

  35. Su, G.-M., Lai, Y.-C., Kwasinski, A., Wang, H.: 3D Visual Communications. Willey, UK (2013)

    Google Scholar 

  36. Aflaki, P., Hannuksela, M.M., Hakkinen, J,, Lindroos, P., Gabbouj. M.: Subjective study on compressed asymmetric stereoscopic video. In: 17th International Conference on ICIP. Hong Kong, People’s Republic of China (2010)

    Google Scholar 

  37. (2014) The International Telecommunication Union (ITU-T); Advanced video coding for generic audiovisual services. ITU-T Rec. H.264

    Google Scholar 

  38. VideoLAN Organization.: x264 free software library. Available online at:http://www.videolan.org/developers/x264.html

  39. Fraunhofer, H.H.I.: H.264/AVC Software Coordination. Available online at: http://iphome.hhi.de/suehring/tml

  40. Vetro, A., Wiegand, T., Sullivan, G.J.: Overview of the stereo and multiview video coding extensions of the H.264/MPEG-4 AVC Standard. In: IEEE Proceedings, 99(4), 626–642 (2011)

    Google Scholar 

  41. VideoHelp – Forum.: Guides, Software. FRIM 3D-MVC Encoder/Decoder 1.26. Available online at: http://www.videohelp.com/software/FRIM

  42. Muprhy, C.: Multiview Video Coding: H.264 Annex H (JMVC). Available online at: https://github.com/cmurphy/JMVC

  43. Sullivan, G.J., Ohm, J.R., Han, W.J., Wiegand, T.: Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1649–1668 (2012)

    Article  Google Scholar 

  44. Fraunhofer, H.H.I.: High Efficiency Video Coding (HEVC). Available online at: https://hevc.hhi.fraunhofer.de

  45. Sullivan, G.J., et al.: Standardized extensions of high efficiency video coding. IEEE J. Sel. Topics Signal Process 7(6), 1001–1016 (2013)

    Article  Google Scholar 

  46. Fraunhofer, H.H.I.: Multiview High Efficiency Video Coding (MV-HEVC). Available online at: https://hevc.hhi.fraunhofer.de/mvhevc

  47. FFmpeg.: Cross-Platform Solution to Record, Convert and Stream Audio and Video. Available online:http://ffmpeg.org/download.html

  48. Cheng, E., Burton, P., Burton, J., Joseski, A., Burnett, I.: RMIT3DV: Pre-Announcement of a Creative Commons Uncompressed HD 3D Video Database. In: 4th international workshop on QoMEX. Melbourne, Australia (2012)

    Google Scholar 

  49. Domanski, M., Grajek, T., Klimaszewski, K., et al.: Poznan Multiview Video Test Sequences and Camera Parameters. ISO/IEC JTC1/SC29/WG11 MPEG 2009/M17050. Xian, China (2009)

    Google Scholar 

  50. (2008) Subjective video quality assessment methods for multimedia applications. ITU-T Rec P 910

    Google Scholar 

  51. Zach, O., Slanina, M.: A matlab-based tool for video quality evaluation without reference. Radioengineering 23(1), 405–411 (2014)

    Google Scholar 

  52. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process 13(4), 600–612 (2004)

    Article  Google Scholar 

  53. Pinson, M.H., Wolf, S.: A new standardized method for objectively measuring video quality. IEEE Trans on Broadcast 50(3), 312–322 (2004)

    Article  Google Scholar 

  54. Yasakethu, S.L.P., et al.: Quality analysis for 3D video using 2D video quality models. IEEE Trans. Consum. Electro 54(4), 1969–1976 (2008)

    Article  Google Scholar 

  55. Saygli, G., Goktug, C., Tekalp, A.M.: Evaluation of asymmetric stereo video coding and rate scaling for adaptive 3D video streaming. IEEE Trans. Broadcast. 57(2), 593–601 (2011)

    Article  Google Scholar 

  56. Sullivan, G.J., Boyce, J.M., Chen, Y., Ohm, J.-R., Segall, C.A., Vetro, A.: Standardized extensions of high efficiency video coding (HEVC). IEEE J. Selec. Topics Signal Process. 7(6), 1001–1016 (2013)

    Article  Google Scholar 

  57. Müller, K., Merkle, P., Wiegand, T.: 3D video representation using depth maps. Proc. IEEE 99(4), 643–656 (2011)

    Article  Google Scholar 

  58. Lu, Yu., Wang, Qing, Ang, Lu, Sun, Yule: Response to call for evidence on free-viewpoint television: Zhejiang University. ISO/IEC JTC1/SC29/WG11, MPEG2016/m37608. San Diego, US (2016)

    Google Scholar 

  59. 3D HEVC reference codec available online https://hevc.hhi.fraunhofer.de/svn/svn_3DVCSoftware/tags/HTM-13.0

  60. Domański, Marek, Dziembowski, Adrian, Grzelka, Adam, Kowalski, Łukasz, Mieloch, Dawid, Samelak, Jarosław, Stankiewicz, Olgierd, Stankowski, Jakub, Wegner, Krzysztof: [FTV AHG] technical description of Poznan University of technology proposal for call for evidence on free-viewpoint television. ISO/IEC JTC1/SC29/WG11, MPEG2016/m37893. San Diego, US (2016)

    Google Scholar 

  61. Müller, K., Vetro, A.: Common Test Conditions of 3DV Core Experiments Joint Collaborative Team on 3D Video Coding Extension Development of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 7th Meeting: Doc. JCT3 V-G1100, San José, US, Jan. 2014

    Google Scholar 

  62. Domański, M., Grajek, T., Klimaszewski, K., Kurc, M., Stankiewicz, O., Stankowski, J., Wegner, K.: Poznań multiview video test sequences and camera parameters. ISO/IEC JTC1/SC29/WG11 MPEG Doc. M17050, Xian, China, Oct. 2009

    Google Scholar 

  63. Rusanovskyy, D., Aflaki, P., Hannuksela, M.M.: “Undo Dancer 3DV sequence for purposes of 3DV standardization. ISO/IEC JTC1/SC29/WG11 MPEG Doc. M20028, Geneva, Switzerland, Mar. 2011

    Google Scholar 

  64. Tanimoto, M., Fujii, T., Fukushima, N.: 1D parallel test sequences for MPEG-FTV. ISO/IECJTC1/SC29/WG11 MPEG Doc. M15378, Archamps, France, Apr. 2008

    Google Scholar 

  65. Ho, Y.S., Lee, E.K., Lee, C.: Multiview video test sequence and camera parameters. ISO/IECJTC1/SC29/WG11 MPEG Doc. M15419, Archamps, France, Apr. 2008

    Google Scholar 

  66. Domański, M., Dziembowski, A. Kuehn, A., Kurc, M., Łuczak, A., Mieloch, D., Siast, J., Stankiewicz, O., Wegner, K.: Poznan Blocks—a multiview video test sequence and cam-era parameters for Free Viewpoint Television. ISO/IEC JTC1/SC29/WG11 Doc. M32243, San Jose, USA, Jan. 2014

    Google Scholar 

  67. Big Buck Bunny test sequence available online http://www.bigbuckbunny.org/

  68. Zitnick, C.L., Kang, S.B., Uyttendaele, M., Winder, S., Szeliski, R.: High-quality video view interpolation using a layered representation. ACM Trans. Graph. 23(3), 600–608 (2004)

    Article  Google Scholar 

  69. Bjøntegaard, G.: calculation of average psnr differences between RD-curves. ITU-T SG16, Doc. VCEG-M33, Austin, USA, Apr. 2001

    Google Scholar 

  70. Alves, G., Pereira, F., daSilva, E.A.B.: Light field imaging coding: Performance assessment methodology and standards benchmarking. In” Proceedings IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 2016

    Google Scholar 

  71. Milovanovic, D., Kukolj, D.: Recent advances in UHD video coding technology: High Dynamic Range and Wide Color Gamut. In: Assuncao, P.A., Vanam, R. (eds.) IEEE COMSOC Multimedia Communications Technical Committee, MMTC Communications—Frontiers, Special issue on Ultra-high definition video communications, vol. 11(1), pp. 50–55, Jan. 2016

    Google Scholar 

  72. Milovanović, D., Kukolj, D., Bojković, Z.: Recent advances on 3D video coding technology: HEVC standardization framework, Chapter 4 in Connected media in the future Internet era (Kondoz, A., Dagiuklas, T. (eds.)), Springer-Verlag, pp. 77–106 (2016)

    Google Scholar 

  73. Ström, J., Samuelsson, J.: Progress report from MPEG. SMPTE Motion Imaging J. 125(7), 80–84 (2016)

    Article  Google Scholar 

  74. Samelak, J., Stankowski, J., Domański, M.: Adaptation of the 3D-HEVC coding tools to arbitrary locations of cameras”, International Conference on Signals and Electronic Systems, ICSES 2016, Kraków, Poland, September 5–7 2016, pp. 107–112

    Google Scholar 

  75. Zakharchenko, V., Choi, K.P., Park, J.H.: Quality metric for spherical panoramic video. In: Proceedings 9970 Optics and Photonics, SPIE Optical Engineering + Applications, San Diego, 2016. pp. C1–9

    Google Scholar 

  76. Grewl, P.K., Viswanath, K.S., Golnaraghi, F.: Minimization of position uncertainty using 3-D stereo imaging technique for the real-time positioning of a handheld breast tissue anomaly detection probe. In: Fourth International Conference on ICCCNT. Tiruchengode, India, (2013)

    Google Scholar 

Download references

Acknowledgements

This book chapter was partially supported by COST Action IC1105—3D-ConTourNet.

Sections 3.2 and 3.3 were supported by the Ministry of Education, Youth and Sports (MEYS) of the Czech Republic no. LD15020 (QOCIES) and by the BUT project no. FEKT-S-17-4426. The research described in these sections was financed by Czech Ministry of Education in frame of National Sustainability Program under grant LO1401. For research, the infrastructure of the SIX Center was used. Section 3.4 was supported by National Science Centre, Poland according to the decision DEC-2012/05/B/ST7/01279.

Section 3.5 was supported by the Ministry of Education, Science and Technological Development of the Republic of Serbia under Grant TR-32034, and Secretary of Science of APV under the Grant 142-451-2484/2017-01/01.

Author information

Authors and Affiliations

  1. Faculty of Technical Sciences, University of Novi Sad, Novi Sad, Serbia

    Dragan Kukolj

  2. Department of Radio Electronics, SIX Research Center, Brno University of Technology (BUT), Brno, Czech Republic

    Libor Bolecek, Ladislav Polak, Tomas Kratochvil, Ondrej Zach, Jan Kufa & Martin Slanina

  3. Chair of Multimedia Telecommunications and Microelectronics, Poznań University of Technology, Poznań, Poland

    Tomasz Grajek, Jarosłw Samelak & Marek Domański

  4. University of Belgrade, Beograd, Serbia

    Dragorad A. Milovanovic

Authors
  1. Dragan Kukolj
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Libor Bolecek
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ladislav Polak
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tomas Kratochvil
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ondrej Zach
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jan Kufa
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Martin Slanina
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Tomasz Grajek
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Jarosłw Samelak
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Marek Domański
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Dragorad A. Milovanovic
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dragan Kukolj .

Editor information

Editors and Affiliations

  1. Instituto de Telecomunicações and Politécnico de Leiria, Leiria, Portugal

    Pedro Amado Assunção

  2. Department of Signal Processing, Tampere University of Technology, Tampere, Finland

    Atanas Gotchev

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer International Publishing AG, part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Kukolj, D. et al. (2019). 3D Content Acquisition and Coding. In: Assunção, P., Gotchev, A. (eds) 3D Visual Content Creation, Coding and Delivery. Signals and Communication Technology. Springer, Cham. https://doi.org/10.1007/978-3-319-77842-6_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-77842-6_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-77841-9

  • Online ISBN: 978-3-319-77842-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation