Skip to main content
Log in

Foveated mean squared error—a novel video quality metric

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Efficiency of a video coding process, as well as accuracy of an objective video quality evaluation can be significantly improved by introduction of the human visual system (HVS) characteristics. In this paper we analyze one of these characteristics; namely, visual acuity reduction due to the foveated vision and object movements in a video sequence. We propose a new video quality metric called Foveated Mean Squared Error (FMSE) that takes into account a variable resolution of the HVS across the visual field. The highest visual acuity is at the point of fixation that falls into fovea, an area at retina with the highest density of photoreceptors. Visual acuity decreases rapidly for image regions which are further with respect to the fixation point. FMSE also utilizes the effect of additional spatial acuity reduction due to motion in a video sequence. The quality measures calculated by FMSE have shown a high correlation with experimental results obtained by subjective video quality assessment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Albanese M, Chianese A, Moscato V, Sansone L (2004) A formal model for video shot segmentation and its application via animate vision. Multimed Tool Appl 24:253–272

    Article  Google Scholar 

  2. Arnow TL, Geisler WS (1996) Visual detection following retinal damage: Prediction of an inhomogeneus retino-cortical model. In: Proc. SPIE 2674: 119-130

  3. Banks MS, Sekuler AB, Anderson SJ (1991) Peripheral spatial vision: limits imposed by optics, photoreceptors, and receptor pooling. J Optical Soc A 8:1775–1787

    Article  Google Scholar 

  4. Barten PGJ (1999) Contrast sensitivity of the human eye and its effects on image quality. SPIE Optical Engineering, Washington

    Book  Google Scholar 

  5. Boccignone G, Chianese A, Moscato V, Picariello A (2005) Foveated shot detection for video segmentation. IEEE Trans Circuits Syst Video Technol 15(3):365–377

    Article  Google Scholar 

  6. Boccignone G, Marcelli A, Napoletano P, Di Fiore G, Iacovoni G, Morsa S (2008) Bayesian integration of face and low-level cues for foveated video coding. IEEE Trans Circuits Syst Video Technol 18(12):1727–1739

    Article  Google Scholar 

  7. Daly S (1993) The visible difference predictor: an algorithm for the assesment of image fidelity. In: Watson AB (ed) Digital images and human vision. The MIT, Cambridge, pp 179–206

    Google Scholar 

  8. Daly S (1998) Engineering observation from spatio velocity and spatiotemporal visual models. In: Proc. SPIE 3299: 180-191

  9. Eckert MP, Buchsbaum G (1993) The significance of eye movements and image acceleration for coding television image sequences. In: Watson AB (ed) Digital images and human vision. The MIT, Cambridge, pp 89–98

    Google Scholar 

  10. ftp://vqeg.its.bldrdoc.gov

  11. Geisler WS, Perry JS (1998) A real time foveated multiresolution system for low-bandwidth video communication. In: Proc. SPIE 3299: 294-305

  12. Ho CC, Wu JL, Cheng WH (2005) A practical foveation-based rate-shaping mechanism for MPEG videos. IEEE Trans Circuits Syst Video Technol 15(11):1365–1372

    Article  Google Scholar 

  13. http://iphome.hhi.de/suehring/tml/download

  14. Itti L (2004) Automatic foveation for video compression using a neurobiological model of visual attention. IEEE Trans Image Process 13(10):1304–1318

    Article  Google Scholar 

  15. Lee S, Bovik AC (2003) Fast algorithms for foveated video processing. IEEE Trans Circuits Syst Video Technol 13(2):149–162

    Article  Google Scholar 

  16. Lee H, Lee S (2006) Visual entropy gain for wavelet image coding. IEEE Signal Process Lett 13(9):553–556

    Article  Google Scholar 

  17. Lee S, Pattchis MS, Bovik AC (2002) Foveated video quality assessment. IEEE Trans Multimedia 4(1):129–132

    Article  Google Scholar 

  18. Linde I, Rajashekar U, Novik AC, Cormack LK (2008) DOVES: a database of visual eye movements. Spat Vis 22(2):161–177

    Article  Google Scholar 

  19. Lisberg SG, Evinger C, Johnson GW, Fuchs AF (1981) Relation between eye acceleration and retinal image velocity during foveal pursuit in man and monkey. J Neurophysiol 46:229–249

    Google Scholar 

  20. Masry M, Hemami SS, Sermadevi Y (2006) A scalable wavelet-based video distortion metric and applications. IEEE Trans Circuits Syst Video Technol 16:260–274

    Article  Google Scholar 

  21. Meur OL, Callet PL, Barba D (2007) Predicting visual fixations on video based on low level visual features. Vis Res 47:2483–2498

    Article  Google Scholar 

  22. MSU Graphics&Media Lab, Video Group, MSU codecs, www.compression.ru/video/

  23. Privitera CM, Stark LW (2000) Algorithms for defining visual regions-of interest: comparison with eye fixation. IEEE Trans Pattern Anal Mach Intell 22(9):970–982

    Article  Google Scholar 

  24. Rajashekar U, Linde I, Bovik AC, Cormack LK (2008) GAFFE: A gaze-attentive fixation finding engine. IEEE Trans Image Process 17(4):564–573

    Article  MathSciNet  Google Scholar 

  25. Rec. ITU-T P.910 (1999) Subjective video quality assessment methods for multimedia applications

  26. Rec. ITU-R BT.500-11 (2002) Methodology for the subjective assessment of the quality of television pictures

  27. Rimac-Drlje S, Žagar D, Martinović G (2009) Spatial masking and perceived video quality in multimedia applications. In: Proc. IWSSIP 2009, Chalkida, Greece

  28. Robson JG, Graham N (1981) Probability summation and regional variation in contrast sensitivity across the visual field. Vis Res 21:409–418

    Article  Google Scholar 

  29. Stelmach LB, Pam WJ (1994) Processing image sequences based on eye movements. In: Proc. SPIE 2179: 90-98

  30. Vranješ M, Rimac-Drlje S, Nemčić O (2009) Influence of Foveated Vision on Video Quality Perception, In: Proc. Elmar 2009, Zadar, pp 29-32

  31. Wang Z, Bovik AC (2001) Embedded foveation image coding. IEEE Trans Image Process 10(10):1397–1410

    Article  MATH  Google Scholar 

  32. Wang Z, Bovik AC, Lu L, Kouloheris J (2001) Foveated wavelet image quality index. In: Proc. SPIE 4472: 42-52

  33. Wang Z, Lu L, Bovik AC (2004) Video quality assessment based on structural distortion measurement. Signal Process Image Comm 19:121–132

    Article  Google Scholar 

  34. Watson AB, Hu J, McGowan JF III (2001) DVQ: A digital video quality metric based on human vision. J Electronic Imaging 10(1):20–29

    Article  Google Scholar 

  35. Winkler S (2005) Digital video quality: Vision models and metrics. Wiley, West Sussex

    Google Scholar 

  36. Wu HR, Rao KR (2006) Digital video image quality and perceptual coding. CRC, New York

    Google Scholar 

  37. www.xvid.org/Downloads/xvidcore-1.1.0.zip

  38. Xiao F. DCT-based video quality evaluation. http://www.compression.ru/video/quality_measure/vqm.pdf

Download references

Acknowledgements

This work is supported by the Croatian Ministry of Education, Science and Sports through the projects 165-0361630-1636 and 165-0362027-1479.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Snježana Rimac-Drlje.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rimac-Drlje, S., Vranješ, M. & Žagar, D. Foveated mean squared error—a novel video quality metric. Multimed Tools Appl 49, 425–445 (2010). https://doi.org/10.1007/s11042-009-0442-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-009-0442-1

Keywords

Navigation