Skip to main content
SpringerLink
Log in

Depth Map Coding for 3DTV Applications

  • Chapter
  • First Online:
Connected Media in the Future Internet Era

Abstract

The communication of multi-view videos requires the transmission and storage of huge amounts of data. To reduce this storage and bandwidth requirements, a reduced set of videos together with the depth information can be used. The 3D geometry information in the depth maps is used in conjunction with the texture information to generate any intermediate view between two received video streams. Unlike texture information, depth data is characterized by large homogeneous areas and sharp edges, where the latter define object boundaries. These different features imply that encoding of depth maps with the standard texture encoder might not be optimal and thus different methods can be used to improve the coding efficiency of depth maps. This chapter presents coding algorithms used for the compression of depth maps in 3DTV applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. (2010) Report on experimental framework for 3D video coding. ISO/IEC JTC1/SC29/WG11, N11631

    Google Scholar 

  2. ITU-T and ISO/IEC JTC 1/SC 29 (MPEG) (2013) High efficiency video coding, Recommendation ITU-T H.265 and ISO/IEC 23008-2

    Google Scholar 

  3. Sullivan G, Ohm J, Han W, Wiegand T (2012) Overview of the high efficiency video coding (HEVC) standard. IEEE Trans Circ Syst Video Technol 22(12):1649–1668

    Article  Google Scholar 

  4. ITU-T and ISO/IEC JTC1 (2010) Advanced video coding for generic audiovisual services, ITU-T Recommendation H.264 and ISO/IEC 14496-10 (MPEG-4 AVC)

    Google Scholar 

  5. Müller K, Schwarz H, Marpe D, Bartnik C, Bosse S, Brust H, Hinz T, Lakshman H, Merkle P, Rhee F, Tech G, Winken M, Wiegand T (2013) 3D high-efficiency video coding for multi-view video and depth data. IEEE Trans Image Process 22(9):3366–3378

    Article  MathSciNet  Google Scholar 

  6. Schwarz H, Bartnik C, Bosse S, Brust H, Hinz T, Lakshman H, Merkle P, Müller K, Rhee H, Tech G, Winken M, Marpe D, Wiegand T (2012) Extension of high efficiency video coding (HEVC) for multiview video and depth data. In: Proceedings of the 19th IEEE international conference on image processing. pp 205–208

    Google Scholar 

  7. Sullivan G, Boyce J, Chen Y, Ohm J, Segall C, Vetro A (2013) Standardized extensions of high efficiency video coding (HEVC). IEEE J Selected Topics Signal Process 7(6):1001–1016

    Article  Google Scholar 

  8. Tanimoto M, Tehrani M, Fujii T, Yendo T (2011) Free-viewpoint TV—a review of the ultimate 3DTV and its related technologies. IEEE signal processing magazine. pp 67–76

    Google Scholar 

  9. Do L, Zinger S, Morvan Y, With P (2009) Quality improving techniques in DIBR for free-viewpoint video. In: Proceedings of 3DTV conference: the true vision—capture, transmission and display of 3D video

    Google Scholar 

  10. Müller K, Smolic A, Dix K, Kauff P, Wiegand T (2010) Reliability-based generation and view synthesis in layered depth video. In Proceedings of the IEEE 10th workshop on multimedia signal processing. pp 34–39

    Google Scholar 

  11. Li W, Zhou J, Li B, Sezan I (2009) Virtual view specification and synthesis for free viewpoint television. IEEE Trans Circ Syst Video Technol 19(4):533–546

    Article  Google Scholar 

  12. Shoemake K (1985) Animation rotation with quaternion curves. In: Proceedings of 12th annual conference on computer graphics and interactive techniques, vol 19, no. 3, pp 245–254

    Google Scholar 

  13. Lee C, Ho Y (2009) View synthesis using depth map for 3D video. In: Proceedings of the Asia-Pacific Signal Information Processing Association annual summit and conference. pp 350–357

    Google Scholar 

  14. Oh K, Yea S, Vetro A, Ho Y (2010) Virtual view synthesis method and self-evaluation metrics for free viewpoint television and 3D video. Int J Imaging Syst Technol 20(4):378–390

    Article  Google Scholar 

  15. Yang X, Lui J, Sun J, Li X, Liu W, Gao Y (2011) DIBR based view synthesis for free-viewpoint television. In: Proceedings of 3DTV conference: the true vision—capture, transmission and display of 3D video

    Google Scholar 

  16. Mori Y, Fukushima N, Fujii N, Tanimoto M (2008) View generation with 3D warping using depth information for FTV. In: Proceedings of 3DTV conference: the true vision—capture, transmission and display of 3D video

    Google Scholar 

  17. Zarb T, Debono, C (2014) Depth-based image processing for 3D video rendering applications. In: Proceedings of the 21st international conference on systems, signals and image processing. pp 215–218

    Google Scholar 

  18. Tran A, Harada, K (2013) View synthesis with depth information based on graph cuts for FTV. In: Proceedings of the 19th Korea-Japan joint workshop on frontiers of computer vision. pp 289–294

    Google Scholar 

  19. Xu J, Yan F, Cao X (2014) Stereoacuity-guided depth image based rendering. In: Proceedings of the IEEE international conference on multimedia and expo

    Google Scholar 

  20. Lei J, Zhang C, Fang Y, Gu Z, Ling N, Hou C (2015) Depth sensation enhancement for multiple virtual view rendering. IEEE Trans Multimedia 17(4):457–469

    Article  Google Scholar 

  21. JCT3V-J1005 (2014) 3D-HEVC test model 10, joint collaborative team on 3D video coding extension development of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11

    Google Scholar 

  22. JCT3V-G1100 (2014) Common test conditions of 3DV core experiments. Joint collaborative team on 3D video coding extension development of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11

    Google Scholar 

  23. Jager F (2012) Simplified depth map intra coding with an optional depth lookup table. In: Proceedings of the international conference on 3D imaging. pp 1–4

    Google Scholar 

  24. Merkle P, Bartnik C, Müller K, Marpe D, Wiegand T (2012) 3D video: depth coding based on inter-component prediction of block partitions. In: Proceedings of the picture coding symposium. pp 149–152

    Google Scholar 

  25. Song Y, Ho Y-S (2013) Simplified inter-component depth modeling in 3D-HEVC. In: Proceedings of the IEEE 11th image, video and multidimensional signal processing workshop

    Google Scholar 

  26. Zhang M, Zhao C, Xu J, Bai H (2013) A Fast depth-map wedgelet partitioning scheme for intra prediction in 3D video coding. In: Proceedings of the IEEE international symposium on circuits and systems. pp 2852–2855

    Google Scholar 

  27. Merkle P, Muller K, Marpe D, Wiegand T (2015) Depth intra coding for 3D video based on geometric primitives. IEEE Trans Circ Syst Video Technol

    Google Scholar 

  28. Zamarin M, Salmistraro M, Forchhammer S, Ortega A (2013) Edge-preserving intra depth coding based on context-coding and H.264/AVC. Multimedia and expo (ICME), 2013 IEEE international conference on, pp 1–6

    Google Scholar 

  29. Oh B, Wey H, Park D (2012) Plane segmentation based intra prediction for depth map coding. In: Proceedings of the picture coding symposium. pp 41–44

    Google Scholar 

  30. Shen G, Kim W-S, Ortega A, Lee J, Wey H (2010) Edge-aware intra prediction for depth-map coding. In: Proceedings of the 2010 IEEE international conference on image processing. pp 3393–3396

    Google Scholar 

  31. Krishnamurthy R, Chai B, Tao H, Sethuraman S (2001) Compression and transmission of depth maps for image-based rendering. In: Proceedings of the 2001 IEEE international conference on image processing. pp 828–831

    Google Scholar 

  32. Merkle P, Morvan Y, Smolic A, Farin D, Müller K, With P, Wiegand T (2009) The effects of multiview depth video compression on multiview rendering. Image Commun 24(1–2):73–88

    Google Scholar 

  33. Zia W, Diepold K, Sarkis M (2010) Fast depth map compression and meshing with compressed tritree. In: Proceedings of the 9th Asian conference on computer vision—volume part II. pp 44–55

    Google Scholar 

  34. Shen G, Kim W-S, Narang S, Ortega A, Lee J, Wey H (2010) Edge-adaptive transforms for efficient depth map coding. In: Proceedings of the picture coding symposium

    Google Scholar 

  35. Hu W, Cheung G, Ortega A, Au O (2015) Multiresolution graph Fourier transform for compression of piecewise smooth images. IEEE Trans Image Process 24(1):419–433

    Article  MathSciNet  Google Scholar 

  36. Graziosi D, Rodrigues N, Pagliari C, Silva E, Faria S, Perez M, Carvalho M (2010) Multiscale recurrent pattern matching approach for depth map coding. In: Proceedings of the picture coding symposium. pp 294–297

    Google Scholar 

  37. Graziosi D, Rodrigues N, Pagliari C, Faria S, Silva E, Carvalho M (2010) Compressing depth maps using multiscale recurrent pattern image coding. Electron Lett 46(5):340–341

    Article  Google Scholar 

  38. Francisco N, Rodrigues N, Silva E, Carvalho M, Faria S, Silva V, Reis M (2008) Multiscale recurrent pattern image coding with a flexible partition scheme. In: Proceedings of the 15th IEEE international conference on image processing

    Google Scholar 

  39. Lucas L, Rodrigues N, Pagliari C, Silva E, Faria S (2012) Efficient depth map coding using linear residue approximation and a flexible prediction framework. In: Proceedings of the 19th IEEE international conference on image processing

    Google Scholar 

  40. Lucas L, Rodrigues N, Pagliari C, Silva E, Faria S (2013) Predictive depth map coding for efficient virtual view synthesis. In: Proceedings of the 20th IEEE international conference on image processing

    Google Scholar 

  41. Lucas L, Wegner K, Rodrigues N, Pagliari C, Silva E, Faria S (2015) Intra predictive depth map coding using flexible block partitioning. IEEE Trans Image Process 24(11):4055–4068

    Article  MathSciNet  Google Scholar 

  42. Müller K, Merkle P, Tech G, Wiegand T (2012) 3D video coding with depth modeling modes and view synthesis optimization. In: Proceedings of the signal and information processing association annual summit and conference

    Google Scholar 

  43. Park C-S (2015) Edge-based intramode selection for depth-map coding in 3d-HEVC. IEEE Trans Image Process 24(1):155–162

    Article  MathSciNet  Google Scholar 

  44. Chen Y, Tech G, Wegner K, Yea S (2014) Test model 9 of 3D-HEVC and MV-HEVC, document JCT3V-I1003, ITU-T SG 16 WP3 and ISO/IEC JTC 1/Sc 29/WG 11

    Google Scholar 

  45. Gu Z, Zheng J, Ling N, Zhang P (2013) Fast depth modeling mode selection for 3D HEVC depth intra coding. In: Proceedings of the IEEE international conference on multimedia and expo workshops

    Google Scholar 

  46. Park C-S (2015) Efficient intra-mode decision algorithm skipping unnecessary depth-modelling modes in 3D-HEVC. Electronics Lett 51(10):756–758

    Article  Google Scholar 

  47. Ricci K, Debono C (2015) A fast inter-component depth modeling technique for 3D high efficiency video coding. In: Proceedings of 3DTV-conference 2015—immersive and interactive 3D media experience over networks

    Google Scholar 

  48. Witten I, Neal R, Cleary J (1987) Arithmetic coding for data compression. Commun ACM 30(6):520–540

    Article  Google Scholar 

  49. ISO/IEC JTC1/SC29/WG11 MPEG2013/M31520, Wegner K, Stankiewicz O, Tanimoto M, Domanski M (2013) Enhanced view synthesis reference software (VSRS) for free-viewpoint television

    Google Scholar 

  50. Bjøntegaard G (2001) Calculation of average PSNR differences between RD-curves. ITU-T SG 16 Q.6 VCEG, Doc. VCEG-M33

    Google Scholar 

  51. Wildeboer M, Yendo T, Tehrani M, Fujii T, Tanimoto M (2010) Color based depth up-sampling for depth compression. In: Proceedings of the picture coding symposium. pp 170–173

    Google Scholar 

  52. Yang Y, Zheng J (2013) Edge-guided depth map resampling for HEVC 3D video coding. In: Proceedings of the international conference on virtual reality and visualization. pp 132–137

    Google Scholar 

  53. Schwarz S, Olsson R, Sjöström M, Tourancheau S (2012) Adaptive depth filtering for HEVC 3D video coding. In: Proceedings of the picture coding symposium. pp 49–52

    Google Scholar 

  54. Gonzalez R, Woods R (2007) Digital image processing, 3rd edn. Prentice Hall

    Google Scholar 

  55. Zammit L (2015) Depth map processing for down-sampled MV-HEVC encoded depth video. M.Sc. Dissertation, University of Malta

    Google Scholar 

  56. Graziosi D, Dong T, Vetro A (2012) Depth map up-sampling based on edge layers. In: Proceedings of the Asia-Pacific signal information processing association annual summit and conference

    Google Scholar 

  57. Huiping D, Li Y, Jinbo Q, Juntao Z (2012) A joint texture/depth edge-directed up-sampling algorithm for depth map coding. In: Proceedings of the IEEE international conference on multimedia and expo. pp 646–650

    Google Scholar 

  58. Oh K-J, Yea S, Vetro A, Ho Y-S (2009) Depth reconstruction filter and down/up sampling for depth coding in 3-D video. IEEE Signal Process Lett 16:747–750

    Article  Google Scholar 

  59. Lee S, Lee S, Wey H, Lee J (2012) 3D-AVC—CE6 related results on Samsung’s in-loop depth resampling. ISO/IEC JTC1/SC29/WG11 Document M23661, San Jose, USA

    Google Scholar 

  60. Graziosi D, Rodrigues N, Silva E, Carvalho M, Faria S, Tian D, Vetro A (2013) Analysis of depth map resampling filters for depth-based 3D video coding. Conference on telecommunications—ConfTele

    Google Scholar 

  61. Guarda A, Santos J, Graziosi D, Rodrigues N, Faria S (2014) A novel trilateral filter technique for depth map processing in 3D video coding. In: Proceedings of the 3DTV conference—3DTV-CON, Budapest, Hungary

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank the Interactive Visual Media group at Microsoft Research for providing the Ballet and Breakdancers test sequences, Nagoya University for providing the Balloons test sequence, and National Institute of Information and Communications Technology (NICT) for providing the Sharks test sequence for research purposes.

Author information

Authors and Affiliations

  1. Department of Communications and Computer Engineering, University of Malta, Msida, Malta

    Carl James Debono

  2. Instituto de Telecomunicações, Leiria, Portugal

    Sérgio Faria, Luís Lucas & Nuno Rodrigues

  3. Universidade Federal do Rio de Janeiro, Rio de Janeiro, Brazil

    Sérgio Faria, Luís Lucas & Nuno Rodrigues

Authors
  1. Carl James Debono
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sérgio Faria
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luís Lucas
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nuno Rodrigues
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carl James Debono .

Editor information

Editors and Affiliations

  1. Institute for Digital Technologies, Loughborough University London, London, United Kingdom

    Ahmet Kondoz

  2. Division of Computer Science and Informatics, London South Bank University, London, United Kingdom

    Tasos Dagiuklas

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer Science+Business Media New York

About this chapter

Cite this chapter

Debono, C.J., Faria, S., Lucas, L., Rodrigues, N. (2017). Depth Map Coding for 3DTV Applications. In: Kondoz, A., Dagiuklas, T. (eds) Connected Media in the Future Internet Era. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-4026-4_6

Download citation

  • DOI: https://doi.org/10.1007/978-1-4939-4026-4_6

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4939-4024-0

  • Online ISBN: 978-1-4939-4026-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation