Abstract
Objective video quality assessment (VQA) refers to evaluation of the quality of a video by an algorithm. The performance of any such VQA algorithm is gaged by how well the algorithmic scores correlate with human perception of quality. Research in the area of VQA has produced a host of full-reference (FR) VQA algorithms. FR VQA algorithms are those in which the algorithm has access to both the original reference video and the distorted video whose quality is being assessed. However, in many cases, the presence of the original reference video is not guaranteed. Hence, even though many FR VQA algorithms have been shown to correlate well with human perception of quality, their utility remains constrained. In this chapter, we analyze recently proposed reduced/no-reference (RR/NR) VQA algorithms. RR VQA algorithms are those in which some information about the reference video and/or the distorting medium is embedded in the video under test. NR VQA algorithms are expected to assess the quality of videos without any knowledge of the reference video or the distorting medium. The utility of RR/NR algorithms has prompted the Video Quality Experts Group (VQEG) to devote resources towards forming a RR/NR test group. In this chapter, we begin by discussing how performance of any VQA algorithm is evaluated. We introduce the popular VQEG Phase-I VQA dataset and comment on its drawbacks. New datasets which allow for objective evaluation of algorithms are then introduced. We then summarize some properties of the human visual system (HVS) that are frequently utilized in developing VQA algorithms. Further, we enumerate the paths that current RR/NR VQA algorithms take in order to evaluate visual quality. We enlist some considerations that VQA algorithms need to consider for HD videos. We then describe exemplar algorithms and elaborate on possible shortcomings of these algorithms. Finally, we suggest possible future research directions in the field of VQA and conclude this chapter.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bt-500-11: Methodology for the subjective assessment of the quality of television pictures. International telecommuncation union
Sekuler, R., Blake, R.: Perception. McGraw Hill, New York (2002)
Wandell, B.: Foundations of vision. Sinauer Associates (1995)
Seshadrinathan, K., Bovik, A.C.: Video quality assessment. In: Bovik, A.C. (ed.) The Essential Guide to Video Processing. Academic Press, London (2009)
Moorthy, A.K., Seshadrinathan, K., Bovik, A.C.: Digital Video Quality Assessment Algorithms. Springer, Heidelberg (2009)
Final report from the video quality experts group on the validation of objective quality metrics for video quality assessment, http://www.its.bldrdoc.gov/vqeg/projects/frtv_phaseI
Girod, B.: What’s wrong with mean-squared error? In: Watson, A.B. (ed.) Digital images and human vision, pp. 207–220 (1993)
Wang, Z., Bovik, A.C.: Mean squared error: Love it or leave it? - a new look at fidelity measures. IEEE Signal Processing Magazine (2009)
Seshadrinathan, K.: Video quality assessment based on motion models. Ph.D. thesis, The University of Texas at Austin (2008)
Wang, Z., Li, Q.: Video quality assessment using a statistical model of human visual speed perception. Journal of the Optical Society of America 24(12), B61–B69 (2007)
Generic coding of moving pictures and associated audio information - part 2: Video, ITU-T and ISO/IEC JTC 1. ITU-T Recommendation H.262 and ISO/IEC 13 818-2 (MPEG-2) (1994)
Advanced video coding, ISO/IEC 14496-10 and ITU-T Rec. H.264 (2003)
Live wireless video database (2009), http://live.ece.utexas.edu/research/quality/live_wireless_video.html
Live video database (2009), http://live.ece.utexas.edu/research/quality/live_video.html
Seshadrinathan, K., Soundararajan, R., Bovik, A.C., Cormack, L.K.: Study of subjective and objective quality assessment of video. IEEE Transactions on Image Processing (to appear)
Moorthy, A.K., Bovik, A.C.: Wireless video quality assessment: A study of subjective scores and objective algorithms. IEEE Transactions on Circuits and Systems for Video Technology (to appear)
Sheskin, D.: Handbook of parametric and nonparametric statistical procedures. CRC Pr. I Llc., Boca Raton (2004)
Moorthy, A.K., Bovik, A.C.: A motion-compensated approach to video quality assessment. In: Proc. IEEE Asilomar Conference on Signals, Systems and Computers (2009)
Wang, Z., Lu, L., Bovik, A.: Foveation scalable video coding with automatic fixation selection. IEEE Transactions on Image Processing 12(2), 243 (2003)
Beauchemin, S., Barron, J.: The computation of optical flow. ACM Computing Surveys (CSUR) 27(3), 433–466 (1995)
Richardson, I.: H. 264 and MPEG-4 video compression: video coding for next-generation multimedia. John Wiley & Sons Inc., Chichester (2003)
Winkler, S.: A perceptual distortion metric for digital color video. In: Proc. SPIE, vol. 3644(1), pp. 175–184 (1999)
Sugimoto, O., Kawada, R., Wada, M., Matsumoto, S.: Objective measurement scheme for perceived picture quality degradation caused by MPEG encoding without any reference pictures. In: Proceedings of SPIE, vol. 4310, p. 932 (2000)
MacWilliams, F., Sloane, N.: Pseudo-random sequences and arrays. Proceedings of the IEEE 64(12), 1715–1729 (1976)
Campisi, P., Carli, M., Giunta, G., Neri, A.: Blind quality assessment system for multimedia communications using tracing watermarking. IEEE Transactions on Signal Processing 51(4), 996–1002 (2003)
Fu-zheng, Y., Xin-dai, W., Yi-lin, C., Shuai, W.: A no-reference video quality assessment method based on digital watermark. In: 14th IEEE Proceedings on Personal, Indoor and Mobile Radio Communications, PIMRC 2003, vol. 3 (2003)
Farias, M., Carli, M., Neri, A., Mitra, S.: Video quality assessment based on data hiding driven by optical flow information. In: Proceedings of the SPIE Human Vision and Electronic Imaging IX, San Jose, CA, USA, pp. 190–200 (2004)
Carli, M., Farias, M., Gelasca, E., Tedesco, R., Neri, A.: Quality assessment using data hiding on perceptually important areas. In: IEEE International Conference on Image Processing (2005)
Wolf, S., Pinson, M.: Video quality measurement techniques. National Telecommunications and Information Administration (NTIA) Report 02-392 (2002)
Wolf, S., Pinson, M.: Low bandwidth reduced reference video quality monitoring system. In: First International Workshop on Video Processing and Quality Metrics for Consumer Electronics, Scottsdale, AZ, USA (2005)
Oelbaum, T., Diepold, K.: Building a reduced reference video quality metric with very low overhead using multivariate data analysis. In: International Conference on Cybernetics and Information Technologies, Systems and Applications, CITSA 2007 (2007)
Oelbaum, T., Diepold, K.: A reduced reference video quality metric for avc/h.264. In: Proc. European Signal Processing Conference, pp. 1265–1269 (2007)
Hair, J.: Multivariate data analysis. Prentice Hall, Englewood Cliffs (2006)
Haykin, S.: Neural networks: a comprehensive foundation. Prentice Hall, Englewood Cliffs (2008)
Le Callet, P., Viard-Gaudin, C., Barba, D.: A convolutional neural network approach for objective video quality assessment. IEEE Transactions on Neural Networks 17(5), 1316 (2006)
Meng, F., Jiang, X., Sun, H., Yang, S.: Objective Perceptual Video Quality Measurement using a Foveation-Based Reduced Reference Algorithm. In: IEEE International Conference on Multimedia and Expo., pp. 308–311 (2007)
Wang, Z., Wu, G., Sheikh, H., Simoncelli, E., Yang, E., Bovik, A.: Quality-aware images. IEEE Transactions on Image Processing 15(6), 1680–1689 (2006)
Hiremath, B., Li, Q., Wang, Z.: Quality-aware video. In: IEEE International Conference on Image Processing, ICIP 2007, vol. 3 (2007)
Simoncelli, E., Freeman, W., Adelson, E., Heeger, D.: Shiftable multiscale transforms. IEEE Transactions on Information Theory 38(2), 587–607 (1992)
Cover, T., Thomas, J.: Elements of information theory. Wiley-Interscience, Hoboken (2006)
Yamada, T., Miyamoto, Y., Serizawa, M., Harasaki, H.: Reduced-reference based video quality-metrics using representative-luminance values. In: Third International Workshop on Video Processing and Quality Metrics for Consumer Electronics, Scottsdale, AZ, USA (2007)
Gunawan, I., Ghanbari, M.: Reduced-reference video quality assessment using discriminative local harmonic strength with motion consideration. IEEE Transactions on Circuits and Systems for Video Technology 18(1), 71–83 (2008)
Gonzalez, R., Woods, R.: Digital image processing. Prentice-Hall, Inc., Upper Saddle River (2002)
Oppenheim, A., Schafer, R.: Discrete-time signal processing. Prentice-Hall, Inc., Upper Saddle River (1989)
Valenzise, G., Naccari, M., Tagliasacchi, M., Tubaro, S.: Reduced-reference estimation of channel-induced video distortion using distributed source coding. In: Proceeding of the 16th ACM international conference on Multimedia (2008)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: From error measurement to structural similarity. IEEE Signal Processing Letters 13(4), 600–612 (2004)
Wang, Z., Lu, L., Bovik, A.C.: Video quality assesssment based on structural distortion measurement. Signal Processing: Image communication (2), 121–132 (2004)
Albonico, A., Valenzise, G., Naccari, M., Tagliasacchi, M., Tubaro, S.: A Reduced-Reference Video Structural Similarity metric based on no-reference estimation of channel-induced distortion. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP (2009)
Naccari, M., Tagliasacchi, M., Pereira, F., Tubaro, S.: No-reference modeling of the channel induced distortion at the decoder for H. 264/AVC video coding. In: Proceedings of the International Conference on Image Processing, San Diego, CA, USA (2008)
Tan, K., Ghanbari, M.: Blockiness detection for MPEG2-coded video. IEEE Signal Processing Letters 7(8), 213–215 (2000)
Vlachos, T.: Detection of blocking artifacts in compressed video. Electronics Letters 36(13), 1106–1108 (2000)
Winkler, S., Sharma, A., McNally, D.: Perceptual video quality and blockiness metrics for multimedia streaming applications. In: Proceedings of the International Symposium on Wireless Personal Multimedia Communications, pp. 547–552 (2001)
Suthaharan, S.: Perceptual quality metric for digital video coding. Electronics Letters 39(5), 431–433 (2003)
Muijs, R., Kirenko, I.: A no-reference blocking artifact measure for adaptive video processing. In: Proceedings of the 13th European Signal Processing Conference, EUSIPCO 2005 (2005)
Caviedes, J., Oberti, F.: No-reference quality metric for degraded and enhanced video. In: Proceedings of SPIE, vol. 5150, p. 621 (2003)
Babu, R., Bopardikar, A., Perkis, A., Hillestad, O.: No-reference metrics for video streaming applications. In: International Packet Video Workshop (2004)
Farias, M., Mitra, S.: No-reference video quality metric based on artifact measurements. In: IEEE International Conference on Image Processing, vol. 3, pp. 141–144 (2005)
Massidda, F., Giusto, D., Perra, C.: No reference video quality estimation based on human visual system for 2.5/3G devices. In: Proceedings of SPIE, vol. 5666, p. 168 (2005)
Marziliano, P., Dufaux, F., Winkler, S., Ebrahimi, T.: Perceptual blur and ringing metrics: Application to JPEG 2000. Signal Processing: Image Communication 19(2), 163–172 (2004)
Dosselmann, R., Yang, X.: A Prototype No-Reference Video Quality System. In: Fourth Canadian Conference on Computer and Robot Vision, CRV 2007, pp. 411–417 (2007)
Kawayoke, Y., Horita, Y.: NR objective continuous video quality assessment model based on frame quality measure. In: 15th IEEE International Conference on Image Processing, ICIP 2008, pp. 385–388 (2008)
Moorthy, A.K., Bovik, A.C.: Visual importance pooling for image quality assessment. IEEE Journal of Selected Topics in Signal Processing, Issue on Visual Media Quality Assessment 3(2), 193–201 (2009)
Yang, F., Wan, S., Chang, Y., Wu, H.: A novel objective no-reference metric for digital video quality assessment. IEEE Signal processing letters 12(10), 685–688 (2005)
Lu, J.: Image analysis for video artifact estimation and measurement. In: Proceedings of SPIE, vol. 4301, p. 166 (2001)
Pastrana-Vidal, R., Gicquel, J.: Automatic quality assessment of video fluidity impairments using a no-reference metric. In: Proc. of Int. Workshop on Video Processing and Quality Metrics for Consumer Electronics (2006)
Pastrana-Vidal, R., Gicquel, J., Colomes, C., Cherifi, H.: Sporadic frame dropping impact on quality perception. In: Proceedings of SPIE, vol. 5292, p. 182 (2004)
Pastrana-Vidal, R., Gicquel, J., Colomes, C., Cherifi, H.: Frame dropping effects on user quality perception. In: 5th International Workshop on Image Analysis for Multimedia Interactive Services (2004)
Lu, Z., Lin, W., Seng, B., Kato, S., Ong, E., Yao, S.: Perceptual Quality Evaluation on Periodic Frame-Dropping Video. In: Proc. of IEEE Conference on Image Processing, pp. 433–436 (2007)
Pastrana-Vidal, R., Gicquel, J.: A no-reference video quality metric based on a human assessment model. In: Third International Workshop on Video Processing and Quality Metrics for Consumer Electronics, VPQM, vol. 7, pp. 25–26 (2007)
Yang, K., Guest, C., El-Maleh, K., Das, P.: Perceptual temporal quality metric for compressed video. IEEE Transactions on Multimedia 9(7), 1528–1535 (2007)
Cotsaces, C., Nikolaidis, N., Pitas, I.: Video shot detection and condensed representation. a review. IEEE signal processing magazine 23(2), 28–37 (2006)
Yamada, T., Miyamoto, Y., Serizawa, M.: No-reference video quality estimation based on error-concealment effectiveness. In: Packet Video 2007, pp. 288–293 (2007)
Keimel, C., Oelbaum, T., Diepold, K.: No-Reference Video Quality Evaluation for High-Definition Video. In: Proceedings of the International Conference on Image Processing, San Diego, CA, USA (2009)
Ong, E., Wu, S., Loke, M., Rahardja, S., Tay, J., Tan, C., Huang, L.: Video quality monitoring of streamed videos. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1153–1156 (2009)
Knee, M.: A single-ended picture quality measure for MPEG-2. In: Proc. International Broadcasting Convention, pp. 7–12 (2000)
Leontaris, A., Reibman, A.: Comparison of blocking and blurring metrics for video compression. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2005, vol. 2 (2005)
Leontaris, A., Cosman, P.C., Reibman, A.: Quality evaluation of motion-compensated edge artifacts in compressed video. IEEE Transactions on Image Processing 16(4), 943–956 (2007)
Hands, D., Bourret, A., Bayart, D.: Video QoS enhancement using perceptual quality metrics. BT Technology Journal 23(2), 208–216 (2005)
Kanumuri, S., Cosman, P., Reibman, A., Vaishampayan, V.: Modeling packet-loss visibility in MPEG-2 video. IEEE transactions on Multimedia 8(2), 341–355 (2006)
Yuen, M., Wu, H.: A survey of hybrid MC/DPCM/DCT video coding distortions. Signal Processing 70(3), 247–278 (1998)
Wang, Z., Simoncelli, E.P.: Maximum differentiation (MAD) competition: A methodology for comparing computational models of perceptual quantities. Journal of Vision 8(12), 1–13 (2008)
Charrier, C., Knoblauch, K., Moorthy, A.K., Bovik, A.C., Maloney, L.T.: Comparison of image quality assessment algorithms on compressed images. In: SPIE conference on Image quality and System Performance (to appear, 2010)
Charrier, C., Maloney, L.T., Cheri, H., Knoblauch, K.: Maximum likelihood difference scaling of image quality in compression-degraded images. Journal of the Optical Society of America 24(11), 3418–3426 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Moorthy, A.K., Bovik, A.C. (2010). Automatic Prediction of Perceptual Video Quality: Recent Trends and Research Directions. In: Mrak, M., Grgic, M., Kunt, M. (eds) High-Quality Visual Experience. Signals and Communication Technology. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12802-8_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-12802-8_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12801-1
Online ISBN: 978-3-642-12802-8
eBook Packages: EngineeringEngineering (R0)