Advertisement

Comprehensive image quality assessment via predicting the distribution of opinion score

  • Anan Liu
  • Jingting Wang
  • Jing LiuEmail author
  • Yuting Su
Article
  • 41 Downloads

Abstract

Image quality assessment is a challenge problem in image processing area. Previous works usually predict the mean opinion score (MOS) to evaluate image quality. However, it is found that the distribution of opinion scores provides richer and more precise semantics information. Therefore, in this work, we focus on the distribution of opinion scores (DOS) and aims to comprehensively evaluate image quality via automatically predicting DOS. Specifically, we first extract image features via convolutional neural network and then adopt the label distribution support vector regressor (LDSVR) algorithm to predict score distribution. To the best of our knowledge, we are the first to introduce label distribution learning approach for image quality assessment. Extensive experiments have been carried out and validate that the proposed algorithm can well predict the DOS and provide a comprehensive assessment to image quality.

Keywords

Image quality assessment Label distribution learning Convolutional neural network Support vector regressor 

Notes

References

  1. 1.
    Cabral RS, Torre F, Costeira JP, Bernardino A (2011) Matrix completion for multi-label image classification. In: Advances in neural information processing systems, pp 190–198Google Scholar
  2. 2.
    Ciocca G, Corchs S, Gasparini F, Schettini R (2014) How to assess image quality within a workflow chain: an overview. Int J Digit Libr 15(1):1–25CrossRefGoogle Scholar
  3. 3.
    Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2014) Decaf: a deep convolutional activation feature for generic visual recognition. In: International conference on machine learning, pp 647–655Google Scholar
  4. 4.
    Drucker H, Burges CJ, Kaufman L, Smola AJ, Vapnik V (1997) Support vector regression machines. In: Advances in neural information processing systems, pp 155–161Google Scholar
  5. 5.
    Fattal R, Lischinski D, Werman M (2002) Gradient domain high dynamic range compression. ACM Trans Graph 21(3):249–256CrossRefGoogle Scholar
  6. 6.
    Gatys L, Ecker AS, Bethge M (2015) Texture synthesis using convolutional neural networks, pp 262–270Google Scholar
  7. 7.
    Gatys LA, Ecker AS, Bethge M (2015) A neural algorithm of artistic style. Computing research repository. arXiv:1508.06576
  8. 8.
    Geng X (2013) Label distribution learning. IEEE Trans Knowl Data Eng 28 (7):1734–1748CrossRefGoogle Scholar
  9. 9.
    Geng X, Peng H (2015) Pre-release prediction of crowd opinion on movies by label distribution learning. In: IEEE international conference on artificial intelligence, pp 3511–3517Google Scholar
  10. 10.
    Geng X, Yin C, Zhou ZH (2013) Facial age estimation by learning from label distributions. IEEE Trans Pattern Anal Mach Intell 35(10):2401–12CrossRefGoogle Scholar
  11. 11.
    Guo Y, Liu Y, Bakker EM, Guo Y, Lew MS (2018) CNN-RNN: a large-scale hierarchical image classification framework. Multimed Tools Appl 77(8):10, 251–10, 271CrossRefGoogle Scholar
  12. 12.
    HDRsoft (Accessed on May, 2016) Photomatix. http://www.hdrsoft.com/
  13. 13.
    He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition, pp 770–778Google Scholar
  14. 14.
    Jing P, Su Y, Nie L, Gu H, Liu J, Wang M (2018) A framework of joint low-rank and sparse regression for image memorability prediction. Accepted by IEEE Transactions on Circuits and Systems for Video Technology, pp 1–1.  https://doi.org/10.1109/TCSVT.2018.2832095
  15. 15.
    Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. In: European conference on computer vision. Springer, pp 694–711Google Scholar
  16. 16.
    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: International conference on neural information processing systems, pp 1097–1105Google Scholar
  17. 17.
    Kundu D, Ghadiyaram D, Bovik AC, Evans BL (2017) No-reference quality assessment of tone-mapped HDR pictures. IEEE Trans Image Process 26(6):2957–2971MathSciNetCrossRefGoogle Scholar
  18. 18.
    Larson GW, Rushmeier H, Piatko C (1997) A visibility matching tone reproduction operator for high dynamic range scenes. IEEE Trans Vis Comput Graph 3(4):291–306CrossRefGoogle Scholar
  19. 19.
    Li J, Lu K, Huang Z, Zhu L, Tao Shen H (2018) Transfer independently together: a generalized framework for domain adaptation. Accepted by IEEE Transactions on Cybernetics.  https://doi.org/10.1109/TCYB.2018.2820174
  20. 20.
    Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain. IEEE Trans Image Process 21(12):4695–4708MathSciNetCrossRefGoogle Scholar
  21. 21.
    Moorthy AK, Bovik AC (2011) Blind image quality assessment: from natural scene statistics to perceptual quality. IEEE Trans Image Process 20(12):3350–3364MathSciNetCrossRefGoogle Scholar
  22. 22.
    Nie L, Wang M, Zha ZJ, Chua TS (2012) Oracle in image search: a content-based approach to performance prediction. ACM Trans Inf Syst 30(2):13, 1–13, 23CrossRefGoogle Scholar
  23. 23.
    Nocedal J, Wright SJ (2006) Numerical optimization. Springer, BerlinzbMATHGoogle Scholar
  24. 24.
    Pece F, Kautz J (2013) Bitmap movement detection: HDR for dynamic scenes. Journal of Virtual Reality and Broadcasting 10(2):1–8Google Scholar
  25. 25.
    Pérez-Cruz F, Alarcón-Diana PL, Navia-Vázquez A, Artés-Rodríguez A (2001) Fast training of support vector classifiers. In: Advances in neural information processing systems, pp 734–740Google Scholar
  26. 26.
    Pietra SD, Pietra VD, Lafferty J (1995) Inducing features of random fields. IEEE Trans Pattern Anal Mach Intell 19(4):380–393CrossRefGoogle Scholar
  27. 27.
    Ponomarenko N, Ieremeiev O, Lukin V, Egiazarian K, Jin L, Astola J, Vozel B, Chehdi K, Carli M, Battisti F (2013) Color image database TID2013: peculiarities and preliminary results. In: European workshop on visual information processing, pp 106–111Google Scholar
  28. 28.
    Qi S, Jing P, Wang X, Nie L (2016) Quality biased multimedia data retrieval in microblogs. J Vis Commun Image Represent 40:838–846CrossRefGoogle Scholar
  29. 29.
    Raman S, Chaudhuri S (2009) Bilateral filter based compositing for variable exposure photography. In: Eurographics (short papers), pp 1–4Google Scholar
  30. 30.
    Razavian AS, Azizpour H, Sullivan J, Carlsson S (2014) CNN Features off-the-shelf: an astounding baseline for recognition. In: IEEE conference on computer vision and pattern recognition workshops, pp 512–519Google Scholar
  31. 31.
    Rubin TN, Chambers A, Smyth P, Steyvers M (2012) Statistical topic models for multi-label document classification. Mach Learn 88(1–2):157–208MathSciNetCrossRefGoogle Scholar
  32. 32.
    Saad MA, Bovik AC, Charrier C (2012) Blind image quality assessment: a natural scene statistics approach in the DCT domain. IEEE Trans Image Process 21 (8):3339–3352MathSciNetCrossRefGoogle Scholar
  33. 33.
    Sánchez-Fernández M, de Prado-Cumplido M, Arenas-García J, Pérez-Cruz F (2004) SVM multiregression for nonlinear channel estimation in multiple-input multiple-output systems. IEEE Trans Signal Process 52(8):2298–2307MathSciNetCrossRefGoogle Scholar
  34. 34.
    Schölkopf B, Smola A (2001) Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT Press, CambridgeGoogle Scholar
  35. 35.
    Sheikh HR, Bovik AC, De VG (2005) An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Trans Image Process 14 (12):2117–2128CrossRefGoogle Scholar
  36. 36.
    Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:14091556
  37. 37.
    Song S, Huang H, Ruan T (2018) Abstractive text summarization using LSTM-CNN based deep learning. Accepted by Multimedia Tools and Applications.  https://doi.org/10.1007/s11042-018-5749-3
  38. 38.
    Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: IEEE conference on computer vision and pattern recognition, pp 1–9Google Scholar
  39. 39.
    Vedaldi A, Lenc K (2015) MatConvNet – convolutional neural network for MATLAB. In: ACM international conference on multimedia, pp 689–692Google Scholar
  40. 40.
    Wang J, Zhao Y, Wu X, Hua XS (2011) A transductive multi-label learning approach for video concept detection. Pattern Recogn 44(10-11):2274–2286CrossRefGoogle Scholar
  41. 41.
    Wang D, Wang B, Zhao S, Yao H, Liu H (2017) Off-the-shelf CNN features for 3D object retrieval. Multimedia Tools and Applications (1):1–17Google Scholar
  42. 42.
    Wei X, Huang H, Nie L, Feng F, Hong R, Chua TS (2018) Quality matters: assessing cQA pair quality via transductive multi-view learning. In: International joint conferences on artificial intelligenceGoogle Scholar
  43. 43.
    Xie L, Shen J, Han J, Zhu L, Shao L (2017) Dynamic multi-view hashing for online image retrieval. In: International joint conferences on artificial intelligenceGoogle Scholar
  44. 44.
    Xu C, Yang J, Gao J (2017) Coupled-learning convolutional neural networks for object recognition. Accepted by Multimedia Tools and Applications.  https://doi.org/10.1007/s11042-017-5262-0
  45. 45.
    Zhu L, Huang Z, Chang X, Song J, Shen HT (2017) Exploring consistent preferences: discrete hashing with pair-exemplar for scalable landmark search. In: Proceedings of the 2017 ACM on multimedia conference, pp 726–734Google Scholar
  46. 46.
    Zhu L, Shen J, Xie L, Cheng Z (2017) Unsupervised visual hashing with semantic assistant for content-based image retrieval. IEEE Trans Knowl Data Eng 29 (2):472–486CrossRefGoogle Scholar
  47. 47.
    Zhu L, Huang Z, Li Z, Xie L, Shen HT (2018) Exploring auxiliary context: discrete semantic transfer hashing for scalable image retrieval. IEEE Trans Neural Netw Learn Syst 29(11):5264–5276CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Electrical and Information EngineeringTianjin UniversityTianjinChina

Personalised recommendations