Advertisement

Multimedia Tools and Applications

, Volume 72, Issue 3, pp 2871–2893 | Cite as

Quality assessment of perceptual color video based on a top-down framework and quaternion

  • Junli Li
  • Xiuying Wang
  • Gang Li
  • Fuqiang Zhang
  • David Feng
Article

Abstract

Objective video quality assessment is of great importance in a variety of video processing applications. Most existing video quality metrics either focus primarily on capturing spatial artifacts in the video signal, or are designed to assess only grayscale video thereby ignoring important chrominance information. In this paper, on the basis of the top-down visual analysis of cognitive understanding and video features, we propose and develop a novel full-reference perceptual video assessment technique that accepts visual information inputs in the form of a quaternion consisting of contour, color and temporal information. Because of the more important role of chrominance information in the “border-to-surface” mechanism at early stages of cognitive visual processing, our new metric takes into account the chrominance information rather than the luminance information utilized in conventional video quality assessment. Our perceptual quaternion model employs singular value decomposition (SVD) and utilizes the human visual psychological features for SVD block weighting to better reflect perceptual focus and interest. Our major contributions include: a new perceptual quaternion that takes chrominance as one spatial feature, and temporal information to model motion or changes across adjacent frames; a three-level video quality measure to reflect visual psychology; and the two weighting methods based on entropy and frame correlation. Our experimental validation on the video quality experts’ group (VQEG) Phase I FR-TV test dataset demonstrated that our new assessment metric outperforms PSNR, SSIM, PVQM (P8) and has high correlation with perceived video quality.

Keywords

Video quality assessment Top-down framework Cognitive analysis Quaternion Singular value decomposition Edge Chrominance Temporal correlation 

Notes

Acknowledgments

This research was supported by the National Natural Science Foundation of China under Grant No. 60832003, Sichuan scientific and technical support plan under Grant No. 2013SZ0085 and the natural science foundation of Ningbo City under Grant No. 2012A610047.

References

  1. 1.
    Alexiadis DS, Sergiadis GD (2009) Motion estimation, segmentation and separation, using hypercomplex phase correlation, clustering techniques and graph-based optimization (a). Comput Vis Image Underst 113(2):212–234CrossRefGoogle Scholar
  2. 2.
    Alexiadis DS, Sergiadis GD (2009) Estimation of motions in color image sequences using hypercomplex Fourier transforms (b). IEEE Trans Image Process 18(1):168–187CrossRefMathSciNetGoogle Scholar
  3. 3.
    Bihan NL, Mars J (2004) Singular value decomposition of quaternion matrices: a new tool for vectorsensor signal processing. Signal Process 84(7):1177–1199CrossRefMATHGoogle Scholar
  4. 4.
    Cermak G, Corriveau P et al. (2003) Final report from the video quality experts group on the validation of objective models of video quality assessment, phase II 2003. http://www.vqeg.org/
  5. 5.
    Chandler DM, Hemami SS (2007) VSNR: a wavelet-based visual signal-to-noise ratio for natural images. IEEE Trans Image Process 16(9):2284–2298CrossRefMathSciNetGoogle Scholar
  6. 6.
    Chikkerur S, Sundaram V, Reisslein M, Karam LJ (2011) Objective video quality assessment methods: a classification, review and performance comparison. IEEE Trans Broadcast 57(2):165–182CrossRefGoogle Scholar
  7. 7.
    Elder JH, Zucker SW (1998) Evidence for boundary-specific grouping. Vis Res 38(1):143–152CrossRefGoogle Scholar
  8. 8.
    Eskicioglu AM, Fisher PS (1995) Image quality measures and their performance. IEEE Trans Commun 43(1):2959–2965CrossRefGoogle Scholar
  9. 9.
    Findlay J (1980) The visual stimulus for saccadic eye movement in human observers. Perception 9:7–21CrossRefGoogle Scholar
  10. 10.
    Gonzalez W (2002) Digital image processing. Prentice HallGoogle Scholar
  11. 11.
    Grossberg S, Hong S (2006) A neuralmodel of surface perception: lightness, anchoring and filling-in. Spat Vis 19(2–4):263–321CrossRefGoogle Scholar
  12. 12.
    Hamilton WR (1844) On quaternions. In: Proceeding of the Royal Irish Academy 11Google Scholar
  13. 13.
    Hekstra AP, Beerends JG, Ledermann D (2002) PVQM—a perceptual video quality measure. Signal Process Image Commun 17(10):781–798CrossRefGoogle Scholar
  14. 14.
    Hung CP, Ramsden BM, Roe AW (2007) A functional circuitry for edge-induced brightness perception. Nat Neurosci 10(9):1185–1190CrossRefGoogle Scholar
  15. 15.
    ITU-T Study Group 12, Contribution COM 12-60, Evaluation of new methods for objective testing of video quality: objective test plan 1998. http://www.vqeg.org/
  16. 16.
    Kantor IL, Solodovnikov AS (1989) Hypercomplex numbers: an elementary introduction to algebras. Springer-VerlagGoogle Scholar
  17. 17.
    Lee C, Cho S, Choe J (2006) Objective video quality assessment. Opt Eng 45(1):1–11Google Scholar
  18. 18.
    Ma X, Xu Y, Song L, Yang X, Burkhardt H (2008) Color image watermarking using local quaternion Fourier spectral analysis. In: Proceedings of IEEE International Conference on Multimedia and Expo 2008:233–236Google Scholar
  19. 19.
    Moorthy AK, Bovik AC (2009) A motion compensated approach to video quality assessment. The 43 Annual Asilomar Conference on Signals, Systems and ComputersGoogle Scholar
  20. 20.
    Osberger W, Anthony JM (1998) Automatic identification of perceptually important regions in an image. The 14th International Conference on. Pattern Recogn 1:701–704Google Scholar
  21. 21.
    Pei SC, Cheng CM (1996) A novel block truncation coding of color images by using quaternion-moment preserving principle. IEEE Int Symp Circ Syst 2:684–687Google Scholar
  22. 22.
    Richardson EG (2003) H.264 and MPEG-4 video compression. John Wiley & SonsGoogle Scholar
  23. 23.
    Rohaky AM, Libert J et al. (2000) Final report from the video quality experts group on the validation of objective models of video quality assessment. http://www.vqeg.org/
  24. 24.
    Senders J (1997) Distribution of visual attention in static and dynamic displays. In: Proc SPIE 3016:186–194Google Scholar
  25. 25.
    Seshadrinathan K, Bovik AC (2010) Motion tuned spatio-temporal quality assessment of natural videos. IEEE Trans Image Process 19(2):335–350CrossRefMathSciNetGoogle Scholar
  26. 26.
    Sheikh HR, Bovik AC, de Veciana G (2005) An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Trans Image Process 14(12):2117–2128CrossRefGoogle Scholar
  27. 27.
    Shnayderman A, Gusev A, Eskicioglu AM (2003) A multidimensional image quality measure using singular value decomposition. In: Proc of SPIE 5294:82–92Google Scholar
  28. 28.
    Shnayderman A, Gusev A, Eskicioglu AM (2005) Assessment of full color image quality with singular value decomposition. In: Proc of SPIE 5668:70–81Google Scholar
  29. 29.
    Shnayderman A, Gusev A, Eskicioglu AM (2006) An SVD-based grayscale image quality measure for local and global assessment. IEEE Trans Image Process 1(2):422–429CrossRefGoogle Scholar
  30. 30.
    Song J, Bi H (1993) Perceptual quality metric for compressed video. Acta Electronica Sinica 28(7):80–83Google Scholar
  31. 31.
    Storn R, Price K (1997) Differential evolution—a simple and efficient adaptive scheme for global optimization over continuous spaces. J Glob Optim 11(4):341–359CrossRefMATHMathSciNetGoogle Scholar
  32. 32.
    Subakan ÖN, Vemuri BC (2009) Quaternion-based color image smoothing using a spatially varying kernel. Lect Notes Comput Sci 5681:415–428CrossRefGoogle Scholar
  33. 33.
    Tsui TK, Zhang XP, Androutsos D (2008) Color image watermarking using multidimensional Fourier transforms. IEEE Trans Inf Forensic Secur 3(1):16–28CrossRefGoogle Scholar
  34. 34.
    VQEG: video quality experts group. http://www.vqeg.org/
  35. 35.
    VQEG: final report (2000) on the validation of objective quality metrics for video quality assessment. http://www.its.bldrdoc.gov/vqeg/projects/frtvphaseI
  36. 36.
    Wang Z, Bovik AC (2006) Modern image quality assessment. Morgan & Claypool PublishersGoogle Scholar
  37. 37.
    Wang Z, Bovik A, Sheikh H (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612CrossRefGoogle Scholar
  38. 38.
    Wang Z, Li Q (2007) Video quality assessment using a statistical model of human visual speed perception. J Opt Soc Am 24(12):B61–B69CrossRefGoogle Scholar
  39. 39.
    Wang Y, Liu W, Wang Y (2008) Color image quality assessment based on quaternion singular value decomposition. In: Proceedings of International Congress on Image and Signal Processing 2008:433–439Google Scholar
  40. 40.
    Wang Z, Lu L, Bovik AC (2004) Video quality assessment based on structural distortion measurement. Signal Process Image Commun 19(2):121–132CrossRefGoogle Scholar
  41. 41.
    Wang Z, Shang X (2006) Spatial pooling strategies for perceptual image quality assessment. IEEE Int Conf Image Proc 2006:2945–2948Google Scholar
  42. 42.
    Watson AB, Hu J, MeGowan JF (2001) DVQ: a digital video quality metric based on human vision. J Electron Imaging 10(1):20–29CrossRefGoogle Scholar
  43. 43.
    Webster AA, Jones CT, Pinson MH (1993) An objective video quality assessment system based on human perception. In Proceedings of the SPIE Conference on Human Vision, Visual Processing and Digital Display IV:15–26Google Scholar
  44. 44.
    Winkler S (1999) A perceptual distortion metric for digital color video. In: Proceedings of the SPIE Conference on Human Vision and Electronic Imaging 3644:175–1841Google Scholar
  45. 45.
    Winkler S (2005) Digital video quality vision models and metrics. John Wiley & SonsGoogle Scholar
  46. 46.
    Xu Y, Yang X, Zhang P, Song L, Traversoni L (2007) Cooperative stereo matching using quaternion wavelets and top-down segmentation. In: Proceedings of IEEE International Conference on Multimedia and Expo 2007:1954–1957Google Scholar
  47. 47.
    Yang C, Chen G, Xie S (2007) Gradient information based image quality assessment. Acta Electronica Sinica 35(7):1313–1317Google Scholar
  48. 48.
    Yang W, Zhao Y, Xu D (2008) Method of image quality assessment based on human visual system and structural similarity. J Beijing Univ Aeronaut Astronaut 34(1):1–4Google Scholar
  49. 49.
    Zhang F (1997) Quaternions and matrices of quaternions. Linear Algebra Appl 251:21–57CrossRefMATHMathSciNetGoogle Scholar
  50. 50.
    Zhang F, Li J, Man J, Chen G (2009) Assessment of color video quality based on quaternion singular value decomposition. Sixth Int Conference Fuzzy Syst Knowl Discov 4:7–10Google Scholar
  51. 51.
    Zhang Q, Wang L, Li H, Ma Z (2012) Video fusion performance evaluation based on structural similarity and human visual perception. Signal Process 92:912–925CrossRefGoogle Scholar
  52. 52.
    Zucker SW (1986) Encyclopedia of artificial intelligence. John Wiley, New YorkGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Junli Li
    • 1
  • Xiuying Wang
    • 2
  • Gang Li
    • 3
  • Fuqiang Zhang
    • 3
  • David Feng
    • 2
  1. 1.College of Computer ScienceSichuan Normal UniversityChengduChina
  2. 2.Biomedical & Multimedia Information Technology Research Group, School of Information TechnologiesUniversity of SydneySydneyAustralia
  3. 3.College of Information Science and EngineeringNingbo UniversityNingboChina

Personalised recommendations