Advertisement

Digital Video Quality Assessment Algorithms

  • Anush K. Moorthy
  • Kalpana Seshadrinathan
  • Alan C. Bovik
Chapter

Abstract

In this chapter we first describe some HVS-based approaches which try to model the visual processing stream described above, since these approaches were originally used to predict visual quality. We then describe recently proposed structural and information-theoretic approaches and feature-based approaches which are commonly used. Further, we describe recent motion-modeling based approaches, and detail performance evaluation and validation techniques for VQA algorithms. Finally, we touch upon some possible future directions for research on VQA and conclude the chapter.

Keywords

Video Sequence Discrete Cosine Transform Video Quality Human Visual System Gabor Filter 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Z. Wang and A. C. Bovik, Modern Image Quality Assessment. New York: Morgan and Claypool Publishing Co., 2006.Google Scholar
  2. 2.
    A. K. Moorthy and A. C. Bovik, “Perceptually Significant Spatial Pooling techniques for Image quality assessment ,” in SPIE Conference on Human Vision and Electronic Imaging, Jan. 2009.Google Scholar
  3. 3.
    “Methodology for the subjective assessment of the quality of television pictures,” ITU-R Recommendation BT.500–11.Google Scholar
  4. 4.
    B. Hiremath, Q. Li and Z. Wang “Quality-aware video,”IEEE International Conference on Image Processing, San Antonio, TX, Sept. 16–19, 2007.Google Scholar
  5. 5.
    H. R. Sheikh, A. C. Bovik, and L. Cormack, “No-reference quality assessment using natural scene statistics: JPEG2000,”Image Processing, IEEE Transactions on, vol. 14, no. 11, pp. 1918–1927, 2005.CrossRefGoogle Scholar
  6. 6.
    C. M. Liu, J. Y. Lin, K. G. Wu and C. N. Wang, “Objective image quality measure for block-based DCT coding,” IEEE Trans. Consum. Electron., vol. 43, pp. 511–516, 1997.CrossRefGoogle Scholar
  7. 7.
    Z. Wang, A. C. Bovik, and B. L. Evans, “Blind measurement of blocking artifacts in images,” in IEEE Intl. Conf. Image Proc, 2000.Google Scholar
  8. 8.
    X. Li, “Blind image quality assessment”, IEEE International Conference on Image Processing, New York, 2002.Google Scholar
  9. 9.
    Patrick Le Callet, Christian Viard-Gaudin, Stéphane Péchard and Emilie Caillault, “No reference and reduced reference video quality metrics for end to end QoS monitoring”, Special Issue on multimedia Qos evaluation and management technologies, E89, (2), Pages: 289–296, February 2006. Google Scholar
  10. 10.
    W. S. Geisler and M. S. Banks, “Visual performance,” in Handbook of Optics, M. Bass, Ed. McGraw-Hill, 1995.Google Scholar
  11. 11.
    B. A. Wandell, Foundations of Vision. Sunderland, MA: Sinauer Associates Inc., 1995.Google Scholar
  12. 12.
    N. C. Rust, V Mante, E. P. Simoncelli, and J. A. Movshon, “How MT cells analyze the motion of visual patterns ”, Nature Neuroscience, vol.9(11), pp. 1421–1431, Nov 2006.CrossRefGoogle Scholar
  13. 13.
    Z. Wang, G. Wu, H. R. Sheikh, E. P. Simoncelli, E.-H. Yang and A. C. Bovik, ”Quality -aware images” IEEE Transactions on Image Processing, vol. 15, no. 6, pp. 1680–1689, June 2006.CrossRefGoogle Scholar
  14. 14.
    R. T. Born and D. C. Bradley, “Structure and function of visual area MT,” Annual Rev Neuroscience, vol. 28, pp. 157–189, 2005.CrossRefGoogle Scholar
  15. 15.
    M. A. Smith, N. J. Majaj, and J. A. Movshon, “Dynamics of motion signaling by neurons in macaque area MT,” Nature Neuroscience, vol. 8, no. 2, pp. 220–228, Feb. 2005.CrossRefGoogle Scholar
  16. 16.
    S. Daly, “The visible differences predictor: an algorithm for the assessment of image fidelity,” in Digital Images and Human Vision (A. B. Watson, ed.), pp. 179–206, Cambridge, MA: The MIT Press, 1993.Google Scholar
  17. 17.
    J. Lubin, “The use of psychophysical data and models in the analysis of display system performance,” in Digital Images and Human Vision (A. B. Watson, ed.), pp. 163–178, Cambridge, MA: The MIT Press, 1993.Google Scholar
  18. 18.
    R. J. Safranek and J. D. Johnston, “A perceptually tuned sub-band image coder with image dependent quantization and post-quantization data compression,” in Proc. ICASSP-89, vol. 3, (Glasgow, Scotland), pp. 1945–1948, May 1989.Google Scholar
  19. 19.
    A. B.Watson, “DCTune: a technique for visual optimization of dct quantization matrices for individual images,” Society for Information Display Digest of Technical Papers, vol. 24, pp. 946–949, 1993.Google Scholar
  20. 20.
    K. Seshadrinathan, R. J. Safranek, J. Chen, T. N. Pappas, H. R. Sheikh, E. P. Simoncelli, Z. Wang and A. C. Bovik. Image quality assessment. In A. C. Bovik, editor, The Essential Guide to Image Processing, chapter 20. Academic Press, 2009.Google Scholar
  21. 21.
    C. J. van den Branden Lambrecht and O. Verscheure, “Perceptual quality measure using a spatiotemporal model of the human visual system,” in Proc. SPIE, vol. 2668, no. 1. San Jose, CA, USA: SPIE, Mar. 1996, pp. 450–461.Google Scholar
  22. 22.
    S. Winkler, “Perceptual distortion metric for digital color video,” Proc. SPIE, vol. 3644, no. 1, pp. 175–184, May 1999.CrossRefGoogle Scholar
  23. 23.
    E. P. Simoncelli, W. T. Freeman, E. H. Adelson, and D. J. Heeger, “Shiftable multiscale transforms,” IEEE Trans. Inform. Theory, vol. 38, pp. 587–607, Mar. 1992.CrossRefMathSciNetGoogle Scholar
  24. 24.
    A. B. Watson, J. Hu, and J. F. McGowan III, “Digital video quality metric based on human vision,” J. Electron. Imaging, vol. 10, no. 1, pp. 20–29, Jan. 2001.CrossRefGoogle Scholar
  25. 25.
    M. Masry, S. S. Hemami, and Y. Sermadevi, “A scalable wavelet-based video distortion metric and applications,”Circuits and Systems for Video Technology, IEEE Transactions on, vol. 16, no. 2, pp. 260–273, 2006.CrossRefGoogle Scholar
  26. 26.
    H. Peterson, A.J. Ahumada, Jr. and A. Watson, ”An Improved Detection Model for DCT Coefficient Quantization,” Human Vision and Electronic Imaging, Proc. SPIE, 1913, 191–201Google Scholar
  27. 27.
    M. Carnec, P. Le Callet, and D. Barba, “Objective quality assessment of color images based on a generic perceptual reduced reference,” Signal Processing: Image Communication, Volume 23 , Issue 4, Pages 239–256, April 2008. CrossRefGoogle Scholar
  28. 28.
    K. Seshadrinathan and A. C. Bovik. Video quality assessment. In A. C. Bovik, editor, The Essential Guide to Video Processing, chapter 14. Academic Press, 2009.Google Scholar
  29. 29.
    Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process, vol. 13, no. 4, pp. 600–612, 2004.CrossRefGoogle Scholar
  30. 30.
    Z. Wang and A. C. Bovik, “A universal image quality index,” IEEE Signal Processing Letters, vol. 9, no. 3, pp. 81–84, 2002.CrossRefGoogle Scholar
  31. 31.
    Z. Wang, E. P. Simoncelli, and A. C. Bovik, “Multiscale structural similarity for image quality assessment,” in Thirty-Seventh Asilomar Conf. on Signals, Systems and Computers, Pacific Grove, CA, 2003.Google Scholar
  32. 32.
    Z. Wang and E. P. Simoncelli, “Translation insensitive image similarity in complex wavelet domain,” in IEEE Intl. Conf. Acoustics, Speech, and Signal Process., Philadelphia, PA, 2005.Google Scholar
  33. 33.
    M. P. Sampat, Z. Wang, S. Gupta, A. C. Bovik and M. K. Markey, ”Complex wavelet structural similarity: A new image similarity index,” IEEE Transactions on Image Processing, to appear 2009.Google Scholar
  34. 34.
    Z. Wang and X. Shang, “Spatial pooling strategies for perceptual image quality assessment,” in IEEE International Conference on Image Processing, Jan. 1996Google Scholar
  35. 35.
    A. K. Moorthy and A. C. Bovik, “Visual importance pooling for image quality assessment,” IEEE Journal of Selected Topics in Signal Processing, Special Issue on Visual Media Quality Assessment, to appear, April 2009.Google Scholar
  36. 36.
    Z. Wang, L. Lu, and A. C. Bovik, “Video quality assessment based on structural distortion measurement,” Signal Processing: Image Communication, vol. 19, no. 2, pp. 121–132, Feb. 2004.CrossRefGoogle Scholar
  37. 37.
    A. Srivastava, A. B. Lee, E. P. Simoncelli, and S.-C. Zhu, “On advances in statistical modeling of natural images,” J. Math. Imag. Vis., vol. 18, pp. 17–33, 2003.MATHCrossRefMathSciNetGoogle Scholar
  38. 38.
    E. P. Simoncelli and B. A. Olshausen, “Natural image statistics and neural representation,” Annu. Rev. Neurosci., vol. 24, pp. 1193–1216, May 2001.CrossRefGoogle Scholar
  39. 39.
    H. R. Sheikh and A. C. Bovik, “A visual information fidelity approach to video quality assessment,” First International Workshop on Video Processing and Quality Metrics for Conusmer Electronics, Jan. 2005.Google Scholar
  40. 40.
    H. R. Sheikh and A. C. Bovik, “Image information and visual quality,” IEEE Trans. Image Process, vol. 15, no. 2, pp. 430–444, 2006.CrossRefGoogle Scholar
  41. 41.
    H. R. Sheikh, A. C. Bovik, and G. de Veciana, “An information fidelity criterion for image quality assessment using natural scene statistics,” IEEE Trans. Image Process., vol. 14, no. 12, pp. 2117–2128, 2005.CrossRefGoogle Scholar
  42. 42.
    J. Malo, I. Epifanio, R. Navarro, and E. P. Simoncelli, “Non-linear image representation for efficient perceptual coding”, IEEE Transactions on Image Processing, vol.15(1), pp. 68–80, Jan 2006.CrossRefGoogle Scholar
  43. 43.
    J. Portilla and E. P. Simoncelli, “ A parametric texture model based on joint statistics of complex wavelet coefficients”, International Journal of Computer Vision, vol.40(1), pp. 49–71, Dec 2000.MATHCrossRefGoogle Scholar
  44. 44.
    J. A. Guerrero-Colón, E. P. Simoncelli , and J. Portilla, “Image denoising using mixtures of Gaussian scale mixtures “, IEEE International Conference on Image Processing, pp. 565–568, Oct 2008.Google Scholar
  45. 45.
    M. J. Wainwright, E. P. Simoncelli, and A. S. Wilsky, “Random cascades on wavelet trees and their use in analyzing and modeling natural images,” Applied and Computational Harmonic Analysis, vol. 11, pp. 89–123, 2001.MATHCrossRefMathSciNetGoogle Scholar
  46. 46.
    M. J. Wainwright and E. P. Simoncelli, “Scale Mixtures of Gaussians and the statistics of natural images”, Adv. Neural Information Processing Systems (NIPS’99), vol.12 pp. 855–861, May 2000.Google Scholar
  47. 47.
    A. P. Hekstra, J. G. Beerends, D. Ledermann, F. E. de Caluwe, S. Kohler, R. H. Koenen, S. Rihs, M. Ehrsam, and D. Schlauss, “PVQM - A perceptual video quality measure,” Signal Proc.: Image Comm. vol. 17, pp. 781–798, 2002.CrossRefGoogle Scholar
  48. 48.
  49. 49.
    M. Malkowski and D. Claben, “Performance of video telephony services in UMTS using live measurements and network emulation,” Wireless Personal Comm., vol. 1, pp. 19–32, 2008.CrossRefGoogle Scholar
  50. 50.
    M. Barkowsky, J. Bialkowski, R. Bitto, and A. Kaup, “Temporal registration using 3D phase correlation and a maximum likelihood approach in the perceptual evaluation of video quality,” in IEEE Workshop on Multimedia Signal Proc., 2007.Google Scholar
  51. 51.
    The Video Quality Experts Group. (2000) Final report from the video quality experts group on the validation of objective quality metrics for video quality assessment. [Online]. Available: http://www.its.bldrdoc.gov/vqeg/projects/frtv phaseI
  52. 52.
    Objective perceptual multimedia video quality measurement in the presence of a full reference, International Telecommunications Union Std. ITU-T Rec. J. 247, 2008.Google Scholar
  53. 53.
    M. H. Pinson and S. Wolf, “A new standardized method for objectively measuring video quality,” IEEE Trans. Broadcast., vol. 50, no. 3, pp. 312–322, Sep. 2004.CrossRefGoogle Scholar
  54. 54.
    The Video Quality Experts Group. (2003) Final VQEG report on the validation of objective models of video quality assessment. [Online]. Available: http://www.ts.bldrdoc.gov/vqeg/projects/frtv phaseII
  55. 55.
    Objective perceptual video quality measurement techniques for digital cable television in the presence of a full reference, International Telecommunications Union Std. ITU-T Rec. J. 144, 2004.Google Scholar
  56. 56.
    “Video quality metric.” [Online]. Available: http://www.its.bldrdoc.gov/n3/video/VQM_ software.php
  57. 57.
    M. Yuen and H. R. Wu, “A survey of hybrid MC/DPCM/DCT video coding distortions,” Signal Processing, vol. 70, no. 3, pp. 247–278, Nov. 1998.MATHCrossRefGoogle Scholar
  58. 58.
    J. A. Movshon and W. T. Newsome, “Visual response properties of striate cortical neurons projecting to Area MT in macaque monkeys,” J. Neurosci., vol. 16, no. 23, pp. 7733–7741, 1996.Google Scholar
  59. 59.
    Z.Wang and Q. Li, “Video quality assessment using a statistical model of human visual speed perception.” J Opt Soc Am A Opt Image Sci Vis, vol. 24, no. 12, pp. B61–B69, Dec 2007.CrossRefGoogle Scholar
  60. 60.
    A. A. Stocker and E. P. Simoncelli, “Noise characteristics and prior expectations in human visual speed perception,” Nature Neuroscience, 9, 578–585 (2006).CrossRefGoogle Scholar
  61. 61.
    Black, M. J. and Anandan, P., “The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields,” Computer Vision and Image Understanding, 63, 75–104 (1996).CrossRefGoogle Scholar
  62. 62.
    K. Seshadrinathan and A. C. Bovik, “Spatio-temporal quality assessment of natural videos,” IEEE Transactions on Image Processing, submitted for publication.Google Scholar
  63. 63.
    K. Seshadrinathan and A. C. Bovik, “A structural similarity metric for video based on motion models,” IEEE International Conference on Acoustics, Speech, and Signal Processing, 2007.Google Scholar
  64. 64.
    D. J. Fleet and A. D. Jepson, “Computation of component image velocity from local phase information,” International Journal of Computer Vision, vol. 5, no. 1, pp. 77–104, 1990.CrossRefGoogle Scholar
  65. 65.
    D. J. Heeger, “Optical flow using spatiotemporal filters,” International Journal of Computer Vision, vol. 1, no. 4, pp. 279–302, 1987.CrossRefGoogle Scholar
  66. 66.
    E. H. Adelson and J. R. Bergen, “Spatiotemporal energy models for the perception of motion.” J Opt Soc Am A, vol. 2, no. 2, pp. 284–299, Feb 1985.CrossRefGoogle Scholar
  67. 67.
    N. J. Priebe, S. G. Lisberger, and J. A. Movshon, “Tuning for spatiotemporal frequency and speed in directionally selective neurons of macaque striate cortex.” J Neurosci, vol. 26, no. 11, pp. 2941–2950, Mar 2006.CrossRefGoogle Scholar
  68. 68.
    E. P. Simoncelli and D. J. Heeger, “A model of neuronal responses in visual area MT,” Vision Res, vol. 38, no. 5, pp. 743–761, Mar 1998.CrossRefGoogle Scholar
  69. 69.
    J. G. Daugman, “Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters,” Journal of the Optical Society of America A (Optics and Image Science), vol. 2, no. 7, pp. 1160–1169, 1985.CrossRefGoogle Scholar
  70. 70.
    P. C. Teo and D. J. Heeger, “Perceptual image distortion,” in Proceedings of the IEEE International Conference on Image Processing. IEEE, 1994, pp. 982–986 vol.2.Google Scholar
  71. 71.
    K. Seshadrinathan and A. C. Bovik, “Unifying analysis of full reference image quality assessment,” in IEEE Intl. Conf. on Image Proc., 2008.Google Scholar
  72. 72.
    A. B. Watson and J. Ahumada, A. J., “Model of human visual-motion sensing,” Journal of the Optical Society of America A (Optics and Image Science), vol. 2, no. 2, pp. 322–342, 1985.CrossRefGoogle Scholar
  73. 73.
    H. Frank and S. C. Althoen, “The coefficient of variation,” in Statistics: Concepts and Applications. Cambridge, Great Britan: Cambridge University Press., 1995, pp. 58–59.Google Scholar
  74. 74.
    K. Seshadrinathan, “Video quality assessment based on motion models,” Ph.D. dissertation, University of Texas at Austin, 2008.Google Scholar
  75. 75.
    H. R. Sheikh, M. F. Sabir, and A. C. Bovik, “A statistical evaluation of recent full reference image quality assessment algorithms,” IEEE Transactions on Image Processing, vol. 15, no. 11, pp. 3440–3451, Nov. 2006.CrossRefGoogle Scholar
  76. 76.
    LIVE image quality assessment database. [Online]. Available: http://live.ece.utexas.edu/research/quality/subjective.html
  77. 77.
    Wang, Z. and Bovik, A. C., “Mean squared error: Love it or leave it? - a new look at fidelity measures.” IEEE Signal Processing Magazine. January 2009.Google Scholar
  78. 78.
    “Video coding for low bit rate communication”, ITU Recommendation H.263.Google Scholar
  79. 79.
    “Generic coding of moving pictures and associated audio information - part 2: Video,” 1994, ITU-T and ISO/IEC JTC 1. ITU-T Recommendation H.262 and ISO/IEC 13 818–2 (MPEG-2).Google Scholar
  80. 80.
    “Advanced video coding,” 2003, ISO/IEC 14496–10 and ITU-T Rec. H.264.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Anush K. Moorthy
    • 1
  • Kalpana Seshadrinathan
    • 1
  • Alan C. Bovik
    • 1
  1. 1.Department of Electrical and Computer EngineeringThe University of Texas at AustinAustinUSA

Personalised recommendations