Multimedia Tools and Applications

, Volume 74, Issue 17, pp 6937–6950 | Cite as

A no reference depth perception assessment metric for 3D video

  • G. Nur Yilmaz


Recent technological breakthroughs in 3-Dimensional (3D) video capture, display, coding, transmission, rendering, etc. have led the advances of 3D multimedia applications into the consumer market. However, the effect of these technologies on 3D video Quality of Experience (QoE) has not been thoroughly investigated to speed up the wide-spread proliferation of the 3D video applications in this market. Quality and depth perception assessment of 3D video from the view of end users reflects the most important aspect of 3D video QoE. Therefore, evaluating quality and depth perception of 3D video should be given the utmost attention. Currently, the depth perception assessment of 3D video can only be achieved using time consuming and rigorous subjective assessments due to the lack of reliable and efficient objective metrics. Assessing the depth perception using Full-Reference (FR)/Reduced Reference (RR) objective metrics is not efficient for on the fly 3D video applications due to the requirement of original video/extracted information at the receiver side. Thus, a No Reference (NR) metric, which does not need any original video related information at the receiver side to predict the depth perception, is proposed in this paper. Three important cues (i.e., binocular parallax, lateral motion, and aerial perspective) for Human Visual System (HVS) to perceive the depth of a 3D video are utilized to develop the NR metric. Experimental results devised using the proposed metric prove the effectiveness of it to predict the depth perception.


3D video Aerial perspective Binocular parallax Depth perception Gaussian Mixture Model (GMM) Expectation Maximization (EM) QoE Lateral motion 


  1. 1.
    Albonico A, Valenzise G, Naccari M, Tagliasacchi M, Tubaro S (2009) A Reduced-reference video structural similarity metric based on no-reference estimation of channel-induced distortion. IEEE International Conference on Acoustics, Speech, and Signal Processing, Taipei, Taiwan, 19–24 Apr. 2009Google Scholar
  2. 2.
    Argyropoulos S, Raake A, Garcia MN, List P (2011) No reference bit stream model for video quality assessment of H.264/AVC video based on packet loss visibility. In: IEEE Int. Conf. on Acoustics, Speech, and Signal Processing (ICASSP)Google Scholar
  3. 3.
    Argyropoulos S, Raake A, Garcia MN, List P (2011) No-reference video quality assessment for SD and HD H.264/AVC sequences based on continuous estimates of packet loss visibility. IEEE Third International Workshop on Quality of Multimedia Experience (QoMEX), Mechelen, 7–9 September, 2011 Google Scholar
  4. 4.
    Brandao T, Queluz MP (2008) No-reference image quality assessment based on DCT domain statistics. Signal Process 88(4):822–833MATHCrossRefGoogle Scholar
  5. 5.
    Brandao T, Queluz MP (2010) No-reference quality assessment of H.264/AVC encoded video. IEEE Trans Circ Syst Video Technol 20(11):1437–1447CrossRefGoogle Scholar
  6. 6.
    Callet PL, Viard-Gaudin C, Pechard S, Caillaultn E (2006) No reference and reduced reference video quality metrics for end to end QoS monitoring. IEICE Trans Commun E85_B(2):289–296CrossRefGoogle Scholar
  7. 7.
    Cheong LF, Xiang T (2001) Characterizing depth distortion under different genetic motions. Int J Comput Vis 44(3):199–217MATHCrossRefGoogle Scholar
  8. 8.
    Devore JL (2003) Probability and statistics for engineering and the sciences, 6th edn. Duxbury Press, Pacific Grove, CA Google Scholar
  9. 9.
    Do CB, Batzoglou S (2008) What is the Expectation Maximization algorithm? Nat Biotechnol 26(8):897–899CrossRefGoogle Scholar
  10. 10.
    Donghyun K, Seungchul R, Kwanghoon S (2012) Depth perception and motion cue based 3D video quality assessment. IEEE International Symposium on Broadband Multimedia Systems and Broadcasting(BMSB), 27–29 June 2012Google Scholar
  11. 11.
    Eden A (2007) No-reference estimation of the coding PSNR for H.264-coded sequences. IEEE Trans Consumer Electronics 53(2):667–674CrossRefGoogle Scholar
  12. 12.
    Farias MCQ, Mitra SK (2005) No reference video quality metric based on artifact measurements. IEEE International Conference on Image Processing, 11–14 September 2005Google Scholar
  13. 13.
    Gunawan IP, Ghanbari M (2008) Reduced-reference video quality assessment using discriminative local harmonic strength with motion consideration. IEEE Trans Circ Syst Video Technol 18(1):71–83CrossRefGoogle Scholar
  14. 14.
    Hall P, Wand MP (1988) On the minimization of absolute distance in kernel density estimation. Stat Probab Lett 6:311–314MATHMathSciNetCrossRefGoogle Scholar
  15. 15.
    Hansard M, Horaud R (2008) Cyclopean geometry of binocular vision. J Opt Soc Am 25:2357–2369CrossRefGoogle Scholar
  16. 16.
    Harris JM, German KJ (2008) Comparing motion induction in lateral motion and motion in depth. Elsevier Vis Res 48:695–702CrossRefGoogle Scholar
  17. 17.
    Hewage CTER, Martini MG (2010) Reduced-reference quality evaluation for compressed depth maps associated with colour plus depth 3D video. 17th IEEE International Conference on Image Processing, Hong Kong, 26–29 Sep. 2010Google Scholar
  18. 18.
    Hewage CTER, Worrall ST, Dogan S, Villette S, Kondoz AM (2009) Quality evaluation of color plus depth map-based stereoscopic video. IEEE J Sel Top Signal Proc 3(2):304–318CrossRefGoogle Scholar
  19. 19.
    Hsien K (2005) K Means clustering, nearest cluster and Gaussian mixture. available at:, Jun. 2005
  20. 20.
    Huynh-Thu Q, Ghanbari M (2008) Scope of validity of PSNR in image/video quality assessment. IET Electron Lett 44(13):800–801CrossRefGoogle Scholar
  21. 21.
    Huynh-Thu Q, Le Callet P, Barkowsky M (2010) Video quality assessment: from 2D to 3D challenges and future trends. 17th IEEE International Conference on Image Processing, Hong Kong, 26–29 Sep. 2010Google Scholar
  22. 22.
    ITU-R BT.500–11 (2002) Methodology for the subjective assessment of the quality of television picturesGoogle Scholar
  23. 23.
    Jones V (2009) Mean direction and mean absolute deviation. ASTM Standards and Engineering Digital LibraryGoogle Scholar
  24. 24.
    JSVM 9.13.1. CVS Server [Online]. Available Telnet: garcon.ient.rwth Scholar
  25. 25.
    Karim HA, Hewage CTER, Worrall S, Kondoz AM (2008) Scalable multiple description video coding for stereoscopic 3D. IEEE Int Conf Consum Electron 54(2):745–752CrossRefGoogle Scholar
  26. 26.
    Keimel C, Klimpke M, Habigt J, Diepold K (2011) No-reference video quality metric for HDTV based on H.264/AVC bit stream features. 18th IEEE International Conference on Image Processing (ICIP), Brussels, Belgium, 11–14 Sept. 2011Google Scholar
  27. 27.
    Lebreton P, Raake A, Barkowsky M, Le Callet P (2012) Evaluating depth perception of 3D stereoscopic videos. IEEE J Sel Top Signal Proc 6(6):710–720CrossRefGoogle Scholar
  28. 28.
    Lin X, Ma H, Luo L, Chen Y (2012) No-reference video quality assessment in the compressed domain. IEEE Trans Consum Electron 58(2):505–512CrossRefGoogle Scholar
  29. 29.
    Lin X, Tian X, Chen Y (2012) No-reference video quality assessment based on region of interest. International Conference on Consumer Electronics, Communications and Networks (CECNet), Yichang, 21–23 April 2012Google Scholar
  30. 30.
    Liu Z, Chen T (2009) Distance measurement system based on binocular stereo vision. IEEE International Joint Conference on Artificial IntelligenceGoogle Scholar
  31. 31.
    Martini MG, Villarini B, Fiorucci F (2012) A reduced-reference perceptual image and video quality metric based on edge preservation. EURASIP J Adv Sig Process (1):1–13Google Scholar
  32. 32.
    Mittal A, Moorthy AK, Ghosh J, Bovik AC (2011) Algorithm assessment of 3D quality of experience for images and videos. IEEE Digital Signal Processing Workshop, 4–7 Jan. 2011.Google Scholar
  33. 33.
    Moorthy AK, Bovik AC (2011) Video quality assessment algorithms: what does the future hold? Springer Multimed Tools Appl 51(2):675–696CrossRefGoogle Scholar
  34. 34.
    Naccari M, Tagliasacchi M, Tubaro S (2009) No-reference video quality monitoring for H.264/AVC coded video. IEEE Transactions on Multimedia 11(5):932–946CrossRefGoogle Scholar
  35. 35.
    Narvekar ND, Karam LJ (2009) An Iterative Deblurring algorithm based on the concept of just noticeable blur. International Workshop on Video Processing and Quality Metrics for Consumer Electronics, Jan. 2009Google Scholar
  36. 36.
    Nian Q (1997) Binocular disparity and the perception of depth. Neuron 18:359–368CrossRefGoogle Scholar
  37. 37.
    Nur G, Bozdagi Akar G, Gokmen H (2012) A reduced reference 3D video quality assessment based on cartoon effect. NEM Summit Conference, Istanbul, Turkey, 16–18 June 2012Google Scholar
  38. 38.
    Nur G, Kodikara Arachchi H, Dogan S, Kondoz AM (2012) Advanced adaptation techniques for improved video perception. IEEE Trans Circ Syst Video Technol 22(2):225–240CrossRefGoogle Scholar
  39. 39.
    Oelbaum T, Diepold K (2007) Building a reduced reference video quality metric with very low overhead using multivariate data analysis. 15th European Signal Processing Conference, Poznan, Poland, 3–7 Sep. 2007Google Scholar
  40. 40.
    Oliver K (2001) Depth perception in media design: from sensory psychology cues to interactive tools. J Vis Lit 21(1):1–14Google Scholar
  41. 41.
    Pfautz JD (2002) Depth perception in computer graphics. PhD Thesis, University of Cambridge, Trinity CollegeGoogle Scholar
  42. 42.
    Pinson MH, Wolf S (2004) A new standardized method for objectively measuring video quality. IEEE Trans Broadcast 50(3):312–322CrossRefGoogle Scholar
  43. 43.
    Read J (2005) Early computational processing in binocular vision and depth perception. Prog Biophys Mol Biol 87:77–108CrossRefGoogle Scholar
  44. 44.
    Reibman AR, Vaishmpayan VA, Sermadevi Y (2004) Quality monitoring of video over a packet network. IEEE Trans Multimed 6(2):327–334CrossRefGoogle Scholar
  45. 45.
    Saxena A, Schulte J, Ng AY (2007) Depth estimation using monocular and stereo cues. Proceedings of the 20th International Joint Conference on Artificial Intelligence, San Francisco, USAGoogle Scholar
  46. 46.
    Solh M, AlRegib G (2011) A no-reference quality measure for DIBR-based 3D videos. IEEE International Workshop on Hot Topics in 3D, Barcelona, Spain, July 11–15, 2011Google Scholar
  47. 47.
    Staelens N, Deschrijver D, Vladislavleva E, Vermeulen B, Dhaene T, Demeester P (2013) Constructing a no-reference H.264/AVC bitstream-based video quality metric using genetic programming-based symbolic regression. IEEE Transactions on Circuits and Systems for Video Technology 23(8):1322–1333CrossRefGoogle Scholar
  48. 48.
    Staelens N et al (2010) Viqid: a no-reference bit stream-based visual quality impairment detector. IEEE Workshop on Quality of Multimedia Experience (QoMEx), Trondheim, Norway, Jun. 2010Google Scholar
  49. 49.
    Tagliasacchi M, Valenzise G, Naccari M, Tubaro S (2010) A reduced-reference structural similarity approximation for videos corrupted by channel errors. Springer Multimed Tools Appl 48(3):471–492CrossRefGoogle Scholar
  50. 50.
    Turaga DS, Chen Y, Caviedes J (2004) No-reference PSNR estimation for compressed pictures. Image Commun 19(2):173–184 (Special Issue Objective Video Quality Metrics)Google Scholar
  51. 51.
    Valenzise G, Magni S, Tagliasacchi M, Tubaro S (2010) Estimating channel-induced distortion in H.264/AVC video without bitstream information. In: Proc. of Quality of Multimedia Experience (QoMEX), pp. 100–105Google Scholar
  52. 52.
    Wang X, Ishii K (2009) Depth perception using a monocular vision system. Lect Notes Comput Sci 5506:779–786CrossRefGoogle Scholar
  53. 53.
    Wang Z, Lu L, Bovik AC (2004) Video quality assessment based on structural distortion measurement. Proc Signal Process Image Commun 19(2):121–132CrossRefGoogle Scholar
  54. 54.
    Wolf S, Pinson MH (2005) Low bandwidth reduced reference video quality monitoring system. First International Workshop on Video Processing and Quality Metrics for Consumer Electronics, Scottsdale, Arizona, 23–25 Jan, 2005Google Scholar
  55. 55.
    Xuan G, Zhang W, Chai P (2001) EM algorithms of Gaussian Mixture Model and hidden markov model. International Conference on Image Processing, Thessaloniki, Greece, 7–10 Oct. 2001Google Scholar
  56. 56.
    Zhu K, Asari V, Saupe D (2013) No-reference quality assessment of H.264/AVC encoded video based on natural scene features. Proc. SPIE 8755 Mobile Multimedia/Image Processing, Security, and Applications, 875505, 28 May 2013Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Electrical and Electronics Engineering DepartmentKirikkale UniversityKirikkaleTurkey

Personalised recommendations