Skip to main content
SpringerLink
Log in

A new depth image quality metric using a pair of color and depth images

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Typical depth quality metrics require the ground truth depth image or stereoscopic color image pair, which are not always available in many practical applications. In this paper, we propose a new depth image quality metric which demands only a single pair of color and depth images. Our observations reveal that the depth distortion is strongly related to the local image characteristics, which in turn leads us to formulate a new distortion assessment method for the edge and non-edge pixels in the depth image. The local depth distortion is adaptively weighted using the Gabor filtered color image and added up to the global depth image quality metric. The experimental results show that the proposed metric closely approximates the depth quality metrics that use the ground truth depth or stereo color image pair.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Chai B-B, Sethuraman S, Sawhney HS, Hatrack P (2004) Depth map compression for real-time view-based rendering. Pattern Recogn Lett 25(7):755–766

    Article  Google Scholar 

  2. Daugman J (1985) Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. Opt Soc Amer J A: Opt Image Sci 2:1160–1169

    Article  Google Scholar 

  3. D’Angelo A, Barni M, Menegaz G (2007) Perceptual quality evaluation of geometric distortions in images. In: Proceedings SPIE, Human Vision and Electronic Imaging XII, pp 1–12

  4. D’Angelo A, Zhaoping L, Barni M (2010) A full-reference quality metric for geometrically distorted images. IEEE Trans Image Process 19(4):867–881

    Article  MathSciNet  Google Scholar 

  5. Fehn C (2004) Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3d-TV. In: Proceedings of SPIE, pp 93–104

  6. Fehn C, De la Barré R, Pastoor S (2006) Interactive 3-DTV-concepts and key technologies. Proc IEEE 94(3):524–538

    Google Scholar 

  7. Gallant J, Braun J, Essen DV (1993) Selectivity for polar, hyperbolic, and cartesian gratings in macaque visual cortex. Science 259(5091):100–103

    Article  Google Scholar 

  8. Geiger A, Lenz P, Urtasun R (2012) Are We ready for Autonomous Driving? The KITTI Vision Benchmark Suite. In: Conference on Computer Vision and Pattern Recognition

  9. Grigorescu C, Petkov N, Kruizinga P (2002) Comparison of texture features based on Gabor filters. IEEE Trans Image Process 11(10):1160–1167

    Article  MathSciNet  Google Scholar 

  10. Grigorescu C, Petkov N, Westenberg MA (2003) Contour detection based on nonclassical receptive field inhibition. IEEE Trans Image Process 12(7):729–739

    Article  Google Scholar 

  11. Guney F, Geiger A (2015) Displets: Resolving Stereo Ambiguities using Object Knowledge. In: Conference on Computer Vision and Pattern Recognition

  12. Ha K, Bae S. -H., Kim M (2013) An objective no-reference perceptual quality assessment metric based on temporal complexity and disparity for stereoscopic video. IEIE Trans Smart Signal Comput 2(5):255–265

    Google Scholar 

  13. Han J, Shao L, Xu D, Shotton J (2013) Enhanced computer vision with microsoft kinect sensor: a review. IEEE Trans Cybern 43(5):1318–1334

    Article  Google Scholar 

  14. Hartley RI (1999) Theory and practice of projective rectification. Int J Comput Vis 35(2):115–127

    Article  Google Scholar 

  15. Jain M, Gupta M, Jain NK (2014) The design of the IIR differintegrator and its application in edge detection. J Inf Process Syst 10(2):223–239

    Article  Google Scholar 

  16. Janoch A, Karayev S, Jia Y, Barron JT, Fritz M, Saenko K, Darrell T (2011) Acategory-level 3-d object dataset: putting the kinect to work. In: Proceedings of ICCV Workshop on Consumer Depth Cameras for Computer Vision, pp 1168–1174

  17. Jung S-W (2014) Image contrast enhancement using color and depth histograms. IEEE Signal Process Lett 21(4):382–385

    Article  Google Scholar 

  18. Khongkraphan K (2014) An efficient color edge detection using the Mahalanobis distance. J Inf Process Syst 10(4):589–601

    Article  Google Scholar 

  19. Klaus A, Sormann M, Karner K (2006) Segment-based stereo matching using brief propagation and a self-adapting dissimilarity measure. In: Proceedings of IEEE Conference on Pattern Recongnition, pp 15–18

  20. Le T-H, Lee S, Jung S -W, Won CS (2015) Reduced reference quality metric for depth images. In: Proceedings Advanced Multimedia and Ubiquitous Engineering, pp 117–122

  21. Scharstein D, Szeliski R (2002) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int J Comput Vis 47(1-3):7–42

    Article  MATH  Google Scholar 

  22. Silberman N, Hoiem D, Kohli P, Fergus R (2012) Indoor segmentation and support inference from rgbd images. In: Proceedings of European Conference on Computer Vision, pp 746–760

  23. Smith SM, Brady JM (1997) SUSAN-A new approach to low level image processing. Int J Comput Vis 23(1):45–78

    Article  Google Scholar 

  24. Smolic A, Mueller K, Stefanoski N, Ostermann J, Gotchev A, Akar GB, Triantafyllidis G, Koz A (2007) Coding algorithms for 3DTV-a survey. IEEE Trans Circ Syst Video Technol 17(11):1606–1621

    Article  Google Scholar 

  25. Smolic A, Mueller K, Merkle P, Kauff P, Wiegand T (2009) An overview of available and emerging 3D video formats and depth enhanced stero as efficient generic solution. In: Proceedings of 27th Conference on Picture Coding Symposium, pp 389–392

  26. Strecha C, Fransens R, Gool L.V (2006) Combined depth and outlier estimation in multi-view stereo. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 2394–2401

  27. Um G. -M., Kim KY, Ahn C, Lee K.H (2005) Three-dimensional scene reconstruction using multiview images and depth camera, pp 271–280

  28. Vipparthi S, Nagar S (2014) Color directional local quinary patterns for content based indexing and retrieval. Human-centric Comput Inf Sci 4(6):1–13

    Google Scholar 

  29. Wang Z-F, Zheng Z-G (2008) A region based stereoo matching algorithm using cooperative optimization. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recongnition, pp 1–8

  30. Xiao J, Owens A, Torralba A (2013) SUN3d: A database of big spaces reconstructed using SfM and object labels. In: Proceedings International Conference on Computer Vision, pp 1–8

  31. Yang Q, Yang R, Davis J, Nister D (2007) Spatial-depth super resolution for range images. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 1–8

  32. Zhou B, Lapedriza A, Xiao J, Torralba A, Oliva A (2014) Learning deep features for scene recognition using places database. In: Proceedings of Advances in Neural Information Processing Systems, pp 1–9

Download references

Acknowledgments

Dr. Thanh-Ha Le’s work was supported by the basic research projects in natural science in 2012 of the National Foundation for Science & Technology Development (Nafosted), Vietnam (102.01-2012.36, Coding and communication of multiview video plus depth for 3D Television Systems). Prof. Seung-Won Jung’s research was supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Science, ICT & Future Planning (NRF-2014R1A1A2057970).

Author information

Authors and Affiliations

  1. University and Engineering and Technology, Vietnam National University, Hanoi, Vietnam

    Thanh-Ha Le

  2. Department of Multimedia Engineering, Dongguk University, Pildong-ro 1gil, Jung-gu, Seoul, 100-715, Korea

    Seung-Won Jung

  3. Department of Electronics and Electrical Engineering, Dongguk University, Pildong-ro 1gil, Jung-gu, Seoul, 100-715, Korea

    Chee Sun Won

Authors
  1. Thanh-Ha Le
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seung-Won Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chee Sun Won
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Seung-Won Jung.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Le, TH., Jung, SW. & Won, C.S. A new depth image quality metric using a pair of color and depth images. Multimed Tools Appl 76, 11285–11303 (2017). https://doi.org/10.1007/s11042-016-3392-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3392-4

Keywords

Search

Navigation