Advertisement

Blind Image Quality Assessment via Deep Recursive Convolutional Network with Skip Connection

  • Qingsen Yan
  • Jinqiu SunEmail author
  • Shaolin Su
  • Yu Zhu
  • Haisen Li
  • Yanning Zhang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11257)

Abstract

The performance of traditional image quality assessment (IQA) methods are not robust, due to those methods exploit shallow hand-designed features. It has been demonstrated that deep neural network can learn more effective features compared with the traditional methods. In this paper we propose a multi-scale recursive deep neural network to accurately predict image quality. In order to learn more effective feature representations for IQA, many deep learning based works focus on using more layers and deeper network structure. However, deeper network layers introduce large numbers of parameters, which causes huge difficulty in training. The proposed recursive convolution layer ensures both the depth of the network and the light of parameters, which guarantees the convergence of training procedure. Moreover, extracting multi-scale features is the most prevalent approach in IQA. Based on this criteria, we using skip connection to combine information among layers, and it further enriches the coarse and fine features for quality assessment. The experimental results on the LIVE, CISQ and TID2013 databases show that the proposed algorithm outperforms all of the state-of-the-art methods, which verifies the effectiveness of our network architecture.

Keywords

Image quality assessment (IQA) Feature extraction Deep learning Convolutional neural networks (CNN) Skip layer No-reference (NR) 

References

  1. 1.
    Bianco, S., Celona, L., Napoletano, P., Schettini, R.: On the use of deep learning for blind image quality assessment. Signal Image Video Process. 3, 1–8 (2016)Google Scholar
  2. 2.
    Bosse, S., Maniry, D., Müller, K.R., Wiegand, T., Samek, W.: Deep neural networks for no-reference and full-reference image quality assessment. IEEE Trans. Image Process. 27(1), 206–219 (2018)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Gong, D., et al.: From motion blur to motion flow: a deep learning solution for removing heterogeneous motion blur. In: Computer Vision and Pattern Recognition, pp. 1827–1836 (2017)Google Scholar
  4. 4.
    Gu, K., Zhai, G., Yang, X., Zhang, W.: Deep learning network for blind image quality assessment. In: IEEE International Conference on Image Processing, pp. 511–515 (2014)Google Scholar
  5. 5.
    Hou, W., Gao, X., Tao, D., Liu, W.: Blind image quality assessment via deep learning 26(6), 1275–1286 (2015)Google Scholar
  6. 6.
    Kang, L., Ye, P., Li, Y., Doermann, D.: Convolutional neural networks for no-reference image quality assessment. In: Computer Vision and Pattern Recognition, pp. 1733–1740 (2014)Google Scholar
  7. 7.
    Kavukcuoglu, K., Boureau, Y.L., Gregor, K., Lecun, Y.: Learning convolutional feature hierarchies for visual recognition. In: International Conference on Neural Information Processing Systems, pp. 1090–1098 (2010)Google Scholar
  8. 8.
    Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1637–1645 (2016)Google Scholar
  9. 9.
    Larson, E.C., Chandler, D.M.: Most apparent distortion: full-reference image quality assessment and the role of strategy. J. Electron. Imaging 19(1), 6–11 (2010)Google Scholar
  10. 10.
    Li, C., Bovik, A.C., Wu, X.: Blind image quality assessment using a general regression neural network. IEEE Trans. Neural Netw. 22(5), 793–799 (2011)CrossRefGoogle Scholar
  11. 11.
    Mittal, A., Moorthy, A.K., Bovik, A.C.: No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. Publ. IEEE Signal Process. Soc. 21(12), 4695–4708 (2012)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Mittal, A., Soundararajan, R., Bovik, A.C.: Making a completely blind image quality analyzer. IEEE Signal Process. Lett. 20(3), 209–212 (2013)CrossRefGoogle Scholar
  13. 13.
    Moorthy, A.K., Bovik, A.C.: Blind image quality assessment: from natural scene statistics to perceptual quality. IEEE Trans. Image Process. Publ. IEEE Signal Process. Soc. 20(12), 3350 (2011)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Moorthy, A.K., Bovik, A.C.: A two-step framework for constructing blind image quality indices. IEEE Signal Process. Lett. 17(5), 513–516 (2010)CrossRefGoogle Scholar
  15. 15.
    Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: International Conference on International Conference on Machine Learning, pp. 807–814 (2010)Google Scholar
  16. 16.
    Ponomarenko, N., et al.: Color image database TID2013: peculiarities and preliminary results. In: European Workshop on Visual Information Processing, pp. 106–111 (2013)Google Scholar
  17. 17.
    Saad, M.A., Bovik, A.C., Charrier, C.: Blind image quality assessment: a natural scene statistics approach in the DCT domain. IEEE Trans. Image Process. Publ. IEEE Signal Process. Soc. 21(8), 3339–3352 (2012)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Schmidhuber, J.: Multi-column deep neural networks for image classification. In: Computer Vision and Pattern Recognition, pp. 3642–3649 (2012)Google Scholar
  19. 19.
    Sheikh, H.R., Bovik, A.C.: Image information and visual quality. IEEE Trans. Image Process. Publ. IEEE Signal Process. Soc. 15(2), 430 (2006)CrossRefGoogle Scholar
  20. 20.
    Sheikh, H.R., Sabir, M.F., Bovik, A.C.: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process. 15(11), 34–51 (2006)CrossRefGoogle Scholar
  21. 21.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (2015)Google Scholar
  22. 22.
    Soundararajan, R., Bovik, A.C.: RRED indices: reduced reference entropic differencing for image quality assessment. IEEE Trans. Image Process. Publ. IEEE Signal Process. Soc. 21(2), 517–526 (2012)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Wang, Z., Bovik, A.: Modern Image Quality Assessment. Morgan and Claypool, San Rafael (2006)CrossRefGoogle Scholar
  24. 24.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRefGoogle Scholar
  25. 25.
    Wang, Z., Simoncelli, E.P.: Reduced-reference image quality assessment using a wavelet-domain natural image statistic model. Proc. SPIE 56, 149–159 (2005)CrossRefGoogle Scholar
  26. 26.
    Wang, Z., Simoncelli, E.P., Bovik, A.C.: Multi-scale structural similarity for image quality assessment (2004)Google Scholar
  27. 27.
    Xue, W., Zhang, L., Mou, X.: Learning without human scores for blind image quality assessment. In: Computer Vision and Pattern Recognition, pp. 995–1002 (2013)Google Scholar
  28. 28.
    Xue, W., Zhang, L., Mou, X., Bovik, A.C.: Gradient magnitude similarity deviation: a highly efficient perceptual image quality index. IEEE Trans. Image Process. Publ. IEEE Signal Process. Soc. 23(2), 684–695 (2013)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Yan, Q., Sun, J., Li, H., Zhu, Y., Zhang, Y.: High dynamic range imaging by sparse representation. Neurocomputing 269, 160–169 (2017)CrossRefGoogle Scholar
  30. 30.
    Yang, J., Gong, D., Liu, L., Shi, Q.: Seeing deeply and bidirectionally: a deep learning approach for single image reflection removal. In: European Conference on Computer Vision (2018)Google Scholar
  31. 31.
    Ye, P., Kumar, J., Kang, L., Doermann, D.: Unsupervised feature learning framework for no-reference image quality assessment. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1098–1105 (2012)Google Scholar
  32. 32.
    Zhang, D.: FSIM: a feature similarity index for image quality assessment. IEEE Trans. Image Process. Publ. IEEE Signal Process. Soc. 20(8), 2378–2386 (2011)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Zhang, L., et al.: Adaptive importance learning for improving lightweight image super-resolution network. arXiv preprint arXiv:1806.01576 (2018)
  34. 34.
    Zhang, L., Wei, W., Zhang, Y., Shen, C., van den Hengel, A., Shi, Q.: Cluster sparsity field: an internal hyperspectral imagery prior for reconstruction. Int. J. Comput. Vis. 126(8), 797–821 (2018)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Qingsen Yan
    • 1
    • 3
  • Jinqiu Sun
    • 2
    Email author
  • Shaolin Su
    • 1
  • Yu Zhu
    • 1
  • Haisen Li
    • 1
  • Yanning Zhang
    • 1
  1. 1.School of Computer Science and EngineeringNorthwestern Polytechnical UniversityXi’anChina
  2. 2.School of AstronauticsNorthwestern Polytechnical UniversityXi’anChina
  3. 3.School of Computer ScienceThe University of AdelaideSouth AustraliaAustralia

Personalised recommendations