Advertisement

CNN-GRNN for Image Sharpness Assessment

  • Shaode Yu
  • Fan Jiang
  • Leida LiEmail author
  • Yaoqin XieEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10116)

Abstract

Image sharpness is key to readability and scene understanding. Because of the inaccessible reference information, blind image sharpness assessment (BISA) is useful and challenging. In this paper, a shallow convolutional neural network (CNN) is proposed for intrinsic representation of image sharpness and general regression neural network (GRNN) is utilized for precise score prediction. The hybrid CNN-GRNN model tends to build functional relationship between retrieved features and subjective human scores by supervised learning. Superior to traditional algorithms based on handcrafted features and machine learning, CNN-GRNN fuses feature extraction and score prediction into an optimization procedure. Experiments on Gaussian blurring images in LIVE, CSIQ, TID2008 and TID2013 show that CNN-GRNN outperforms the state-of-the-art algorithms and gets closer to human subjective judgment.

Keywords

Mean Opinion Score Convolutional Neural Network General Regression Neural Network Image Quality Assessment Image Sharpness 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgement

The authors would like to thank reviewers for their valuable suggestion that has helped to improve the paper quality. This work is supported from National Natural Science Foundation of China (Grant No. 81501463 and 61379143), Guangdong Innovative Research Team Program (Grant No. 2011S013), National 863 Programs of China (Grant No. 2015AA043203), the Shenzhen Fundamental Research Program (Grant Nos. JCYJ20140417113430726, JCYJ20140417113430665 and JCYJ20150401145529039), the Qing Lan Project of Jiangsu Province and the China Postdoctoral Science Foundation (Grant No. 2016M590827).

References

  1. 1.
    Bahrami, K., Kot, A.C.: A fast approach for no-reference image sharpness assessment based on maximum local variation. IEEE Sig. Process. Lett. 21(6), 751–755 (2014)CrossRefGoogle Scholar
  2. 2.
    Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)CrossRefGoogle Scholar
  3. 3.
    Chandler, D.M.: Seven challenges in image quality assessment: past, present, and future research. ISRN Sig. Process. 2013, 1–53 (2013)CrossRefGoogle Scholar
  4. 4.
    Ciancio, A., Costa, A.D., da Silva, E., Said, A., Samadani, R., Obrador, P.: No-reference blur assessment of digital pictures based on multifeature classifiers. IEEE Trans. Image Process. 21(3), 934–945 (2012)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Ferzli, R., Karam, L.J.: A no-reference objective image sharpness metric based on the notion of just noticeable blur (JNB). IEEE Trans. Image Process. 18(4), 717–728 (2009)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Hassen, R., Wang, Z., Salama, M.M.: Image sharpness assessment based on local phase coherence. IEEE Trans. Image Process. 22(7), 2798–2810 (2013)CrossRefGoogle Scholar
  7. 7.
    Hou, W., Gao, X.: Saliency-guided deep framework for image quality assessment. IEEE Multimedia 22(2), 46–55 (2015)CrossRefGoogle Scholar
  8. 8.
    Hou, W., Gao, X., Tao, D., Li, X.: Blind image quality assessment via deep learning. IEEE Trans. Neural Netw. Learn. Syst. 26(6), 46–55 (2015)MathSciNetGoogle Scholar
  9. 9.
    Kang, L., Ye, P., Li, Y., Doermann, D.: Convolutional neural networks for no-reference image quality assessment. IEEE Conference on Computer Vision and Pattern Recognition, pp. 1733–1740 (2014)Google Scholar
  10. 10.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  11. 11.
    Larson, E.C., Chandler, D.M.: Most apparent distortion: full-reference image quality assessment and the role of strategy. J. Electron. Imaging 19(1), 11006 (2010)CrossRefGoogle Scholar
  12. 12.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015)CrossRefGoogle Scholar
  13. 13.
    Li, C., Bovik, A.C., Wu, X.: Blind image quality assessment using a general regression neural network. IEEE Trans. Neural Netw. 22(5), 793–799 (2011)CrossRefGoogle Scholar
  14. 14.
    Li, L., Lin, W., Wang, X., Yang, G., Bahrami, K., Kot, A.C.: No-reference image blur assessment based on discrete orthogonal moments. IEEE Trans. Cybern. 46(1), 39–50 (2016)CrossRefGoogle Scholar
  15. 15.
    Li, Y., Po, L., Xu, X., Feng, L., Yuan, F., Cheung, C.H., Cheung, K.W.: No-reference image quality assessment with shearlet transform and deep neural networks. Neurocomputing 154, 94–109 (2015)CrossRefGoogle Scholar
  16. 16.
    Li, L., Wu, D., Wu, J., Li, H., Lin, W., Kot, A.C.: Image sharpness assessment by sparse representation. IEEE Trans. Multimedia 18(6), 1085–1097 (2016)CrossRefGoogle Scholar
  17. 17.
    Lin, W., Kuo, C.: Perceptual visual quality metrics: a survey. J. Vis. Commun. Image Represent. 22(4), 297–312 (2011)CrossRefGoogle Scholar
  18. 18.
    Lv, Q., Jiang, G., Yu, M., Xu, H., Shao, F., Liu, S.: Difference of Gaussian statistical features based blind image quality assessment: a deep learning approach. In: IEEE Conference on Image Processing, pp. 2344–2348 (2015)Google Scholar
  19. 19.
    Manap, R.A., Shao, L.: Non-distortion-specific no-reference image quality assessment: a survey. Inf. Sci. 301, 141–160 (2015)CrossRefGoogle Scholar
  20. 20.
    Mittal, A., Moorthy, A.K., Bovik, A.C.: No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21(12), 4695–4708 (2012)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Mittal, A., Soundararajan, R., Bovik, A.C.: Making a “completely” blind image quality analyzer. IEEE Sig. Process. Lett. 20(3), 209–212 (2013)CrossRefGoogle Scholar
  22. 22.
    Moorthy, A.K., Bovik, A.C.: A two-step framework for constructing blind image quality indices. IEEE Sig. Process. Lett. 17(5), 513–516 (2010)CrossRefGoogle Scholar
  23. 23.
    Narvekar, N.D., Karam, L.J.: A no-reference image blur metric based on the cumulative probability of blur detection (CPBD). IEEE Trans. Image Process. 20(9), 2678–2683 (2009)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Niu, X., Suen, C.: A novel hybrid CNN-SVM classifier for recognizing handwritten digits. Pattern Recogn. 45(4), 1318–1325 (2012)CrossRefGoogle Scholar
  25. 25.
    Palm, R.B.: Prediction as a candidate for learning deep hierarchical models of data. Technical University of Denmar (2012)Google Scholar
  26. 26.
    Ponomarenko, N., Lukin, V., Zelensky, A., Egiazarian, K., Astola, J., Carli, M., Battisti, F.: TID2008 - a database for evaluation of full-reference visual quality assessment metrics. Adv. Mod. Radioelectron. 10(4), 30–45 (2009)Google Scholar
  27. 27.
    Ponomarenko, N., Jin, L., Ieremeiev, O., Lukin, V., Egiazarian, K., Astola, J., Vozel, B., Chehdi, K., Carli, M., Battisti, F., Kuo, C.C.J.: Image database TID2013: peculiarities, results and perspectives. Sig. Process. Image Commun. 20, 57–77 (2015)CrossRefGoogle Scholar
  28. 28.
    Russakovsky, O., Deng, J., Su, H., Jonathan, K., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A., Li, F.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Saad, M.A., Bovik, A.C., Christophe, C.: Blind image quality assessment: a natural scene statistics approach in the DCT domain. IEEE Trans. Image Process. 21(8), 3339–3352 (2012)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Sang, Q., Qi, H., Wu, X., Bovic, A.C.: No-reference image blur index based on singular value curve. J. Vis. Commun. Image Represent. 25(7), 1625–1630 (2014)CrossRefGoogle Scholar
  31. 31.
    Sheikh, H.R., Sabir, M.F., Bovik, A.C.: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process. 15(11), 3440–3451 (2006)CrossRefGoogle Scholar
  32. 32.
    Specht, D.F.: A general regression neural network. IEEE Trans. Neural Netw. 2(6), 568–576 (1991)CrossRefGoogle Scholar
  33. 33.
    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)Google Scholar
  34. 34.
    Virtanen, T., Nuutinen, M., Vaahteranoksa, M., Oittinen, P.: CID2013: a database for evaluating no-reference image quality assessment algorithms. IEEE Trans. Image Process. 24(1), 390–402 (2015)MathSciNetCrossRefGoogle Scholar
  35. 35.
    Vu, P.V., Chandler, D.M.: A fast wavelet-based algorithm for global and local image sharpness estimation. IEEE Sig. Process. Lett. 19(7), 423–426 (2012)CrossRefGoogle Scholar
  36. 36.
    Vu, C.T., Phan, T.D., Chandler, D.M.: S3: a spectral and spatial measure of local perceived sharpness in natural images. IEEE Trans. Image Process. 21(3), 934–945 (2012)MathSciNetCrossRefGoogle Scholar
  37. 37.
    Wang, Z., Bovik, A.C.: Reduced- and no-reference image quality assessment. IEEE Sig. Process. Mag. 28(6), 29–40 (2011)CrossRefGoogle Scholar
  38. 38.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRefGoogle Scholar
  39. 39.
    Yu, S., Zhang, W., Wu, S., Li, X., Xie, Y.: Applications of edge preservation ratio in image processing. In: IEEE International Conference on Signal Processing, pp. 698–702 (2014)Google Scholar
  40. 40.
    Zhang, L., Zhang, L., Mou, X., Zhang, D.: FSIM: a feature similarity index for image quality assessment. IEEE Trans. Image Process. 20(8), 2378–2386 (2011)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Shenzhen Institutes of Advanced TechnologyChinese Academy of SciencesShenzhenChina
  2. 2.Shenzhen College of Advanced TechnologyUniversity of Chinese Academy of SciencesShenzhenChina
  3. 3.School of Information and Electronic EngineeringChinese University of Mining and TechnologyXuzhouChina

Personalised recommendations