Skip to main content
SpringerLink
Log in

3D hybrid just noticeable distortion modeling for depth image-based rendering

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The 3D Just Noticeable Distortion (JND) threshold in essence depends on Human Visual Sensitivity (HVS). This paper carves out a Hybrid Just Noticeable Distortion (HJND) model to measure JND threshold in the framework of Depth Image-Based Rendering (DIBR) for 3D video. The critical differences between 2D and 3D visual perception, depth saliency and geometric distortion, are combined into the HJND model because their significant influence on HVS. To save bit, the HJND model is introduced into the Multi-view Video plus Depth (MVD) encoding framework as a residual filter. After the residue is filtered by HJND and the reference model named Joint Just Noticeable Distortion (JJND), bit saving is achieved up to 28.79% and 23.53%, respectively, and the 3D impaired videos filtered by HJND and JJND have the similar subjective quality. The experiments demonstrate that HJND describes HVS for 3D video more accurately than the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Anmin L, Weisi L, Paul M, Chenwei D, Fan Z (2010) Just noticeable difference for images with decomposition model for separating edge and textured regions. IEEE Trans Circ Syst Video Technol 20(11):1648–1652

    Article  Google Scholar 

  2. Chen H, Ruimin H, Jinhui H, Wang Z (2010) Temporal color just noticeable distortion model and its application for video coding. In: 2010 IEEE international conference on multimedia and expo (ICME), July 2010, pp 713–718

  3. Chun-Hsien C, Chi-Wei C (1996) A perceptually optimized 3d subband codec for video communication over wireless channels. IEEE Trans Circ Syst Video Technol 6(2):143–156

  4. Chun-Hsien C, Yun-Chin Li (1995) A perceptually tuned subband image coder based on the measure of just-noticeable-distortion profile. IEEE Trans Circ Syst Video Technol 5(6):467–476

  5. De Silva D, Fernando W, Worrall S, Yasakethu S, Kondoz A (2010) Just noticeable difference in depth model for stereoscopic 3d displays. In: 2010 IEEE international conference on multimedia and expo (ICME), July 2010, pp 1219–1224

  6. Gang X, Zhang Z (1996) Epipolar geometry in stereo, motion and object recognition: a unified approach, vol 6. Springer

  7. Gao Y, et al (2011) Perceptual multiview video coding using synthesized just noticeable distortion maps. In: 2011 IEEE international symposium on circuits and systems (ISCAS). IEEE

  8. Huynh-Thu Q, Barkowsky M, Le Callet P (2011) The importance of visual attention in improving the 3dtv viewing experience: overview and new perspectives. IEEE Trans Broadcast 57(2):421–431

  9. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259

    Article  Google Scholar 

  10. Jezzard P, Balaban RS (1995) Correction for geometric distortion in echo planar images from B0 field variations. Magn Reson Med 34.1:65–73

    Article  Google Scholar 

  11. Li X, Wang Y, Zhao D, Jiang T, Zhang N (2011) Joint just noticeable difference model based on depth perception for stereoscopic images. In: 2011 IEEE visual communications and image processing (VCIP), Nov 2011, pp 1–4

  12. Luo Z, Song L, Zheng S et al (2013) H. 264/AVC perceptual optimization coding based on JND directed coefficient suppression. IEEE Trans Circ Syst Video Technol 23(6):935–948

    Article  Google Scholar 

  13. Merkle P, Morvan Y, Smolic A, Farin D, Muller K, de With PHN, Wiegand T (2009) The effects of multi-view depth video compression on multiview rendering. Special issue on advances in three-dimensional television and video. Signal Process Image Commun 24(1–2):73–88

    Article  Google Scholar 

  14. Merkle P, Smolic A, Muller K, Wiegand T (2007) Efficient prediction structures for multi-view video coding. IEEE Trans Circ Syst Video Technol 17(11):1461–1473

    Article  Google Scholar 

  15. Methodology for the subjective assessment of the quality of television pictures (2002) ITU Document ITU-R BT.500-11, Geneva, Switzerland

  16. Muller K (2008) HHI test material for 3d video. ISO/IECJTC1/SC29/WG11. M15413

  17. Nicolas A, Cruz DS, Ebrahimi T (2002) MESH: measuring errors between surfaces using the Hausdorff distance. ICME (1)

  18. Sequeira V et al (2001) Hybrid 3D reconstruction and image-based rendering techniques for reality modeling. In: Proceedings international conference on videometrics and optical methods for 3D shape measurement, vol 4309. SPIE, pp 126–136

  19. Shum HY, Kang SB (2000) A Review of image-based rendering techniques. In: Proceedings of the international conference on Visual communication and image processing (VCIP 00), vol 4067. SPIE, pp 2–13

  20. Smolic A, Muller K, Dix K, Merkle P, Kauff P, Wiegand T (2008) Intermediate view interpolation based on multiview video plus depth for advanced 3d video systems. In: 15th IEEE international conference on image processing, 2008. ICIP 2008, pp 2448–2451

  21. Stankiewicz O, Wegner K (2009) Poznan multi-view video test sequences and camera parameters. ISO/IECJTC1/SC29/WG11 M15413

  22. Tanimoto M, Fujii T, Suzuki K (2008) Improvement of depth map estimation and view synthesis. ISOIECJTC1SC29WG11, M15090. Antalya, Turkey

  23. Quan H-T, Ghanbari M (2008) Scope of validity of PSNR in image/video quality assessment. Electron Lett 44.13:800–801

    Google Scholar 

  24. Qiaoyan Z, Gangyi J, Fen C, Zongju P, Feng S, Mei Y (2013) A just noticeable distortion based rate control algorithm for multiview video coding. JSW 8(10):2541–2548

    Google Scholar 

  25. Wang Z, Al B (2006) Modern image quality assessment (synthesis lectures on image, video, and multimedia processing), 1st edn. Morgan Claypool Publishers

  26. Yang XK, Ling WS, Lu ZK, Ong EP, Yao SS (2005) Just noticeable distortion model and its applications in video coding. Signal Processing Image Commun 20(7):662–680

    Article  Google Scholar 

  27. Yaqing N, Souidene W, Beghdadi A (2011) A visual sensitivity model based stereo image watermarking scheme. In: 2011 3rd European workshop on visual information processing (EUVIP), pp 211–215

  28. Zhang Y, Jiang G, Mei Y, Chen K, Dai Q (2010) Stereoscopic visual attention-based regional bit allocation optimization for multiview video coding. EURASIP J Adv Signal Process 2010:60:1–60:24

    Google Scholar 

  29. Zhao Y, Chen Z, Ce Z, Tan Y-P, Lu Y (2011) Binocular just-noticeable-difference model for stereoscopic images. IEEE Signal Process Lett 18(1):19–22

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by Major Program of National Natural Science Foundation of China (NSFC) under Grant 61231015, the National NSFC under Grants 61271256, 61172173, 61172174, and 61303114, and by the Major Science and Technology Innovation Plan of Hubei Province under Grant 2013AAA020.

Author information

Authors and Affiliations

  1. National Engineering Research Center for Multimedia Software and the School of Computer, Wuhan University, Wuhan, 430079, China

    Rui Zhong, Ruimin Hu & Zhongyuan Wang

  2. School of Electrical and Electronic Engineering, Nanyang Technological University, 50 Nanyang Avenue, Singapore, 639798, Singapore

    Shizheng Wang

Authors
  1. Rui Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruimin Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhongyuan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shizheng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ruimin Hu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhong, R., Hu, R., Wang, Z. et al. 3D hybrid just noticeable distortion modeling for depth image-based rendering. Multimed Tools Appl 74, 10457–10478 (2015). https://doi.org/10.1007/s11042-014-2176-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-014-2176-y

Keywords

Search

Navigation