A New Color Image Database TID2013: Innovations and Results

  • Nikolay Ponomarenko
  • Oleg Ieremeiev
  • Vladimir Lukin
  • Lina Jin
  • Karen Egiazarian
  • Jaakko Astola
  • Benoit Vozel
  • Kacem Chehdi
  • Marco Carli
  • Federica Battisti
  • C. -C. Jay Kuo
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8192)


A new database of distorted color images called TID2013 is designed and described. In opposite to its predecessor, TID2008, this database contains images with five levels of distortions instead of four used earlier and a larger number of distortion types (24 instead of 17). The need for these modifications is motivated and new types of distortions are briefly considered. Information on experiments already carried out in five countries with the purpose of obtaining mean opinion score (MOS) is presented. Preliminary results of these experiments are given and discussed. Several popular metrics are considered and Spearman rank order correlation coefficients between these metrics and MOS are presented and discussed. Analysis of the obtained results is performed and distortion types difficult for assessment by existing metrics are noted.


full reference metrics image visual quality mean opinion score color image database subjective experiments 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Keelan, B.W.: Handbook of Image Quality. Marcel Dekker, Inc., New York (2002)CrossRefGoogle Scholar
  2. 2.
    Wu, H.R., Lin, W., Karam, L.: An Overview of Perceptual Processing for Digital Pictures. In: Proceedings of International Conference on Multimedia and Expo Workshops, Melbourne, pp. 113–120 (2012)Google Scholar
  3. 3.
    Larson, E.C., Chandler, D.M.: Most apparent distortion: full-reference image quality assessment and the role of strategy. Journal of Electronic Imaging 19(1), 011006 (2010)CrossRefGoogle Scholar
  4. 4.
    Ponomarenko, N., Krivenko, S., Lukin, V., Egiazarian, K.: Lossy Compression of Noisy Images Based on Visual Quality: A Comprehensive Study. EURASIP Journal on Advances in Signal Processing, 13 (2010), doi:10.1155/2010/976436Google Scholar
  5. 5.
    Carli, M.: Perceptual Aspects in Data Hiding. Thesis for the degree of Doctor of Technology, Tampere University of Technology (2008)Google Scholar
  6. 6.
    Moorthy, A.K., Bovik, A.C.: Visual Quality Assessment Algorithms: What Does the Future Hold? Multimedia Tools and Applications 51(2), 675–696 (2011)CrossRefGoogle Scholar
  7. 7.
    Fevralev, D., Lukin, V., Ponomarenko, N., Abramov, S., Egiazarian, K., Astola, J.: Efficiency analysis of DCT-based filters for color image database. In: Proceedings of SPIE Conference Image Processing: Algorithms and Systems VII, San Francisco, vol. 7870 (2011)Google Scholar
  8. 8.
    Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13(4), 600–612 (2004)CrossRefGoogle Scholar
  9. 9.
    Chandler, D.M.: Seven Challenges in Image Quality Assessment: Past, Present and Future Research. In: ISNR Signal Processing, vol. 2913, pp. 1–53 (2013)Google Scholar
  10. 10.
    Jin, L., Egiazarian, K., Jay Kuo, C.-C.: Perceptual Image Quality Assessment Using Block-Based Milti-Metric Fusion (BMMF). In: Proceedings of ICASSP, Kyoto, pp. 1145–1148 (2012)Google Scholar
  11. 11.
    Sheikh, H.R., Wang, Z., Cormack, L., Bovik, A.C.: LIVE Image Quality Assessment Database Release 2,
  12. 12.
    Sheikh, H.R., Sabir, M.F., Bovik, A.C.: A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms. IEEE Transactions on Image Processing 15(11), 3441–3452 (2006)CrossRefGoogle Scholar
  13. 13.
    Horita, Y., Shibata, K., Parvez Saddad, Z.M.: Subjective quality assessment toyama database,
  14. 14.
    Ponomarenko, N., Lukin, V., Zelensky, A., Egiazarian, K., Carli, M., Battisti, F.: TID2008 - A Database for Evaluation of Full-Reference Visual Quality Assessment Metrics. Advances of Modern Radioelectronics 10, 30–45 (2009)Google Scholar
  15. 15.
    Uss, M., Vozel, B., Lukin, V., Abramov, S., Baryshev, I., Chehdi, K.: Image Informative Maps for Estimating Noise Standard Deviation and Texture Parameters. EURASIP Journal on Advances in Signal Processing, 961–964 (2011)Google Scholar
  16. 16.
    Lukin, V., Ponomarenko, N., Egiazarian, K.: HVS-Metric-Based Performance Analysis of Image Denoising Algorithms. In: Proceedings of EUVIP, pp. 156–161 (2011)Google Scholar
  17. 17.
    Vu, C.T., Phan, T.D., Chandler, D.M.: S3: a Spectral and Spatial Measure of Local Perceived Sharpness in Natural Images. IEEE Transactions on Image Processing 21(3), 934–945 (2012)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Kendall, M.G.: The advanced theory of statistics, vol. 1. Charles Griffin & Company limited, London (1945)Google Scholar
  19. 19.
    Wang, Z., Simoncelli, E.P., Bovik, A.C.: Multi-scale structural similarity for image quality assessment. In: IEEE Asilomar Conference on Signals, Systems and Computers, pp. 1398–1402 (2003)Google Scholar
  20. 20.
    Jin, L., Egiazarian, K., Jay Kuo, C.-C.: Performance comparison of decision fusion strategies in BMMF-based image quality assessment. In: Proceedings of APSIPA, Hollywood, pp. 1–4 (2012)Google Scholar
  21. 21.
    Zhang, L., Mou, X., Zhang, D.: FSIM: a feature similarity index for image quality assessment. IEEE Transactions on Image Processing 20(5), 2378–2386 (2011)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Ponomarenko, N., Eremeev, O., Lukin, V., Egiazarian, K., Carli, M.: Modified image visual quality metrics for contrast change and mean shift accounting. In: Proceedings of CADSM, Polyana-Svalyava, pp. 305–311 (2011)Google Scholar
  23. 23.
    Larson, E.C., Chandler, D.M.: Most apparent distortion: full-reference image quality assessment and the role of strategy. Journal of Electronic Imaging 19(1), 011006 (2010)CrossRefGoogle Scholar
  24. 24.
    Pedersen, M., Bonnier, N., Hardeberg, J.Y., Albregtsen, F.: Attributes of Image Quality for Color Prints. Journal of Electronic Imaging 19(1), 011016-1–011016-13 (2010)Google Scholar
  25. 25.
    Hassan, M., Bhagvati, C.: Structural Similarity Measure for Color Images. International Journal of Computer Applications 43(14), 7–12 (2012)CrossRefGoogle Scholar
  26. 26.
    Ponomarenko, N.N., Lukin, V.V., Ieremeyev, O.I., Egiazarian, K., Astola, J.: Visual quality analysis for images degraded by different types of noise. In: Proceedings of SPIE Symposium on Electronic Imaging, San Francisco, vol. 8655, p. 12. SPIE (2013)Google Scholar
  27. 27.
    Oh, B.T., Jay Kuo, C.-C., Sun, S., Lei, S.: Film Grain Noise Modeling in Advanced Video Coding, SPIE Proceedings, Vol. In: SPIE Proceedings, San Jose, vol. 6508, p. 12 (2007)Google Scholar
  28. 28.
    Petrescu, D., Pincenti, J.: Quality and noise measurements in mobile phone video capture. In: SPIE Proceedings, San Francisco, vol. 7881, p. 14 (2011)Google Scholar
  29. 29.
    Danielyan, A., Foi, A., Katkovnik, V., Egiazarian, K.: Spatially adaptive filtering as regularization in inverse imaging: compressive sensing, upsampling, and super-resolution. In: Milanfar, P. (ed.) Super-Resolution Imaging, CRC Press / Taylor & Francis (2010)Google Scholar
  30. 30.
    Paredes, J.L., Arce, G.R.: Compressive Sensing Signal Reconstruction by Weighted Median Regression Estimate. IEEE Transactions on Signal Processing 59(6), 2585–2601 (2011)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Damera-Venkata, N., Kite, T., Geisler, W., Evans, B., Bovik, A.: Image Quality Assessment Based on a Degradation Model. IEEE Transactions on Image Processing 9, 636–650 (2000)CrossRefGoogle Scholar
  32. 32.
    Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13(4), 600–612 (2004)CrossRefGoogle Scholar
  33. 33.
    Sheikh, H.R., Bovik, A.C.: Image Information and Visual Quality. IEEE Transactions on Image Processing 15, 430–444 (2006)CrossRefGoogle Scholar
  34. 34.
    Chandler, D.M., Hemami, S.S.: VSNR: A Wavelet-Based Visual Signal-to-Noise Ratio for Natural Images. IEEE Transactions on Image Processing 16(9), 2284–2298 (2007)MathSciNetCrossRefGoogle Scholar
  35. 35.
    Mitsa, T., Varkur, K.: Evaluation of contrast sensitivity functions for the formulation of quality measures incorporated in halftoning algorithms. In: IEEE International Conference on Acoustic, Speech, and Signal Processing, Minneapolis, vol. 5, pp. 301–304 (1993)Google Scholar
  36. 36.
    Gaubatz, M.: Metrix MUX Visual Quality Assessment Package,
  37. 37.
    Egiazarian, K., Astola, J., Ponomarenko, N., Lukin, V., Battisti, F., Carli, M.: New full-reference quality metrics based on HVS. In: Proceedings of the Second International Workshop on Video Processing and Quality Metrics, Scottsdale, p. 4 (2006)Google Scholar
  38. 38.
    Ponomarenko, N., Silvestri, F., Egiazarian, K., Carli, M., Astola, J., Lukin, V.: On between-coefficient contrast masking of DCT basis functions. In: Proc. of the Third International Workshop on Video Processing and Quality Metrics, USA, p. 4 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Nikolay Ponomarenko
    • 1
  • Oleg Ieremeiev
    • 1
  • Vladimir Lukin
    • 1
  • Lina Jin
    • 2
  • Karen Egiazarian
    • 2
  • Jaakko Astola
    • 2
  • Benoit Vozel
    • 3
  • Kacem Chehdi
    • 3
  • Marco Carli
    • 4
  • Federica Battisti
    • 4
  • C. -C. Jay Kuo
    • 5
  1. 1.Dept of Transmitters, Receivers and Signal ProcessingNational Aerospace UniversityKharkovUkraine
  2. 2.Institute of Signal ProcessingTampere University of TechnologyTampereFinland
  3. 3.University of Rennes 1 - IETR, CS 80518Lannion CedexFrance
  4. 4.University of Rome IIIRomeItaly
  5. 5.Media Communications LabUSC Viterbi School of EngineeringUSA

Personalised recommendations