Skip to main content
SpringerLink
Log in

A local structural information representation method for image quality assessment

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Image is a typical example of visual data, and its quality inevitably affects its application. Hence, measuring image quality accurately is a beneficial task. In practical application, there are different image types e.g. natural image and screen content image (SCIs). And the distortion types contained in images are various. Most image quality assessment (IQA) methods concentrate on a single image type with limited distortion types. In this paper, we present a no-reference IQA method which can accurately measure the quality for both natural image and SCI, and is robust for various distortion types. Human visual system is sensitive to the changes in image structural information which are usually caused by image quality degradation. Therefore, the new method employs local structural information representation for IQA. We first analyze the gray-scale fluctuation of each pixel in four detection directions to obtain four gray-scale fluctuation maps (GFMs) and one gray-scale fluctuation direction map (GFD). And then, the structural features extracted from GFMs and GFD are used for representing local structural information. Finally, the mapping function from the features to image subjective scores is trained by support vector regression (SVR). The experimental results on the public databases demonstrate that SVR is suitable for IQA and the proposed method can accurately predict the quality of both natural images and SCIs with various distortion types.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.
Fig. 11.
Fig. 12.
Fig. 13.

Similar content being viewed by others

References

  1. Asadi Amiri S, Hassanpour H, Marouzi OR (2018) No-reference image quality assessment based on localized discrete cosine transform for JPEG compressed images[J]. Multimed Tools Appl 77(1):787–803

    Google Scholar 

  2. Ashwini K, Amutha R (2018) Fast and secured cloud assisted recovery scheme for compressively sensed signals using new chaotic system[J]. Multimed Tools Appl 77(24):31581–31606

    Google Scholar 

  3. Chang CC, Lin CJ (2011) LIBSVM: a library for support vector machines[J]. ACM Trans Intell Syst Technol 2(3):1–27

    Google Scholar 

  4. Fang Y, Yan J, Li L, Wu J, Lin W (2018) No reference quality assessment for screen content images with both local and global feature representation[J]. IEEE Trans Image Process 27(4):1600–1610

    MathSciNet  MATH  Google Scholar 

  5. Freitas PG, Akamine WYL, Farias MCQ (2016) No-reference image quality assessment based on statistics of local ternary pattern[C]// 2016 eighth international conference on quality of multimedia experience (QoMEX). IEEE

  6. Freitas PG, Akamine WYL, Farias MCQ (2018) No-reference image quality assessment using orthogonal color Planes patterns[J]. IEEE Transact Multimed 20(12):3353–3360

    Google Scholar 

  7. Ghadiyaram D, Bovik AC (2015) Massive online Crowdsourced study of subjective and objective picture quality[J]. IEEE Trans Image Process 25(1):372–387

    MathSciNet  MATH  Google Scholar 

  8. Ghadiyaram D, Bovik AC (2017) Perceptual quality prediction on authentically distorted images using a bag of features approach.[J]. J Vis 17(1):32

    Google Scholar 

  9. Gu K, Wang S, Yang H, Lin W, Zhai G, Yang X, Zhang W (2016) Saliency-guided quality assessment of screen content images[J]. IEEE Transact Multimed 18(6):1098–1110

    Google Scholar 

  10. Gu K, Zhai G, Lin W, Yang X, Zhang W (2016) Learning a blind quality evaluation engine of screen content images[J]. Neurocomputing 196(C):140–149

    Google Scholar 

  11. Gu K, Zhou J, Qiao JF, Zhai G, Lin W, Bovik AC (2017) No-reference quality assessment of screen content pictures[J]. IEEE Trans Image Process 26(8):4005–4018

    MathSciNet  MATH  Google Scholar 

  12. Gu K, Qiao J, Min X, Yue G, Lin W, Thalmann D (2018) Evaluating quality of screen content images via structural variation analysis[J]. IEEE Transact Visual Comp Graph 24(10):2689–2701

    Google Scholar 

  13. Hu M, Yang Y, Shen F, Zhang L, Shen HT, Li X (2017) Robust web image annotation via exploring multi-facet and structural knowledge.[J]. IEEE Trans Image Process 26(10):4871–4884

    MathSciNet  Google Scholar 

  14. Hu B, Li L, Qian J (2018) Perceptual quality evaluation for motion deblurring[J]. IET Comput Vis 12(6):796–805

    Google Scholar 

  15. Jerripothula KR, Cai J, Yuan J (2018) Quality-guided fusion-based co-saliency estimation for image co-segmentation and Colocalization[J]. IEEE Transact Multimed 20(9):2466–2477

    Google Scholar 

  16. Ji H, Liu C (2008) Motion blur identification from image gradients[C] Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on. IEEE :1–8

  17. Kim J, Lee S (2017) Fully deep blind image quality predictor[J]. IEEE J Select Top Signal Process 11(1):206–220

    Google Scholar 

  18. Larson EC, Chandler DM (2010) Most apparent distortion: full-reference image quality assessment and the role of strategy[J]. J Electron Imaging 19(1):011006

    Google Scholar 

  19. Li Q, Lin W, Xu J, Fang Y (2016) Blind image quality assessment using statistical structural and luminance features[J]. IEEE Transact Multimed 18(12):2457–2469

    Google Scholar 

  20. Li Q, Lin W, Fang Y (2017) BSD: blind image quality assessment based on structural degradation[J]. Neurocomputing 236(C):93–103

    Google Scholar 

  21. Li Y, Liu W, Huang J (2018) Sub-selective quantization for learning binary codes in large-scale image search.[J]. IEEE Transact Patt Analysis Mach Intell 40(6):1526–1532

    Google Scholar 

  22. Liu L, Hua Y, Zhao Q et al (2015) Blind image quality assessment by relative gradient statistics and AdaBoosting neural network[J]. Signal Process Image Commun 40(C):1–15

    Google Scholar 

  23. Liu Y, Zhai G, Gu K, Liu X, Zhao D, Gao W (2018) Reduced-reference image quality assessment in free-energy principle and sparse representation[J]. IEEE Transact Multimed 20(2):379–391

    Google Scholar 

  24. Liu L, Wang T, Huang H et al (2019) Pre-attention and spatial dependency driven no-reference image quality assessment[J]. IEEE Transact Multimed 21(9):2305–2318

    Google Scholar 

  25. Liu H, Zhang Y, Zhang H, Fan C, Kwong S, Kuo CCJ, Fan X (2020) Deep learning-based picture-wise just noticeable distortion prediction model for image compression[J]. IEEE Trans Image Process 29:641–656

    MathSciNet  Google Scholar 

  26. Ma K, Liu W, Liu T, Wang Z, Tao D (2017) dipIQ: blind image quality assessment by learning-to-rank discriminable image pairs[J]. IEEE Trans Image Process 26(8):3951–3964

    MathSciNet  MATH  Google Scholar 

  27. Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain[J]. IEEE Trans Image Process 21(12):4695–4708

    MathSciNet  MATH  Google Scholar 

  28. Mittal A, Soundararajan R, Bovik AC (2013) Making a “completely blind” image quality analyzer[J]. IEEE Signal Process Lett 20(3):209–212

    Google Scholar 

  29. Moorthy AK, Bovik AC (2010) A two-step framework for constructing blind image quality indices[J]. IEEE Signal Process Lett 17(5):513–516

    Google Scholar 

  30. Moorthy AK, Bovik AC (2011) Blind image quality assessment: from natural scene statistics to perceptual quality[J]. IEEE Trans Image Process 20(12):3350–3364

    MathSciNet  MATH  Google Scholar 

  31. Ponomarenko N, Lukin V, Zelensky A et al (2009) TID2008-a database for evaluation of full-reference visual quality assessment metrics [J]. Adv Mod Radioelectron 10(4):30–45

    Google Scholar 

  32. Ponomarenko N, Jin L, Ieremeiev O, Lukin V, Egiazarian K, Astola J, Vozel B, Chehdi K, Carli M, Battisti F, Jay Kuo CC (2015) Image database TID2013: peculiarities, results and perspectives[J]. Signal Process Image Commun 30:57–77

    Google Scholar 

  33. Sheikh HR, Sabir MF, Bovik AC (2006) A statistical evaluation of recent full reference image quality assessment algorithms.[J]. IEEE Trans Image Process 15(11):3440–3451

    Google Scholar 

  34. Song X, Peng X, Xu J, Shi G, Wu F (2017) Distributed compressive sensing for cloud-based wireless image transmission[J]. IEEE Transact Multimed 19(6):1351–1364

    Google Scholar 

  35. Sun W, Liao Q, Xue JH, Zhou F (2018) SPSIM: a Superpixel-based similarity index for full-reference image quality assessment[J]. IEEE Trans Image Process 27(9):4232–4244

    MathSciNet  MATH  Google Scholar 

  36. Tang L, Li Q, Li L, Gu K, Qian J (2018) Training-free referenceless camera image blur assessment via hypercomplex singular value decomposition[J]. Multimed Tools Appl 77(5):5637–5658

    Google Scholar 

  37. VQEG, Final report from the video quality experts group on the validation of objective models of video quality assessment, March 2000 (http://www.vqeg.org/)

  38. Wang Z, Simoncelli EP, Bovik AC (2003) Multiscale structural similarity for image quality assessment[C]. Thrity-Seventh Asilomar Conf Signals Sys Comput 2:1398–1402

    Google Scholar 

  39. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity[J]. IEEE Trans Image Process 13(4):600–612

    Google Scholar 

  40. Wang S, Gu K, Zhang X, Lin W, Zhang L, Ma S, Gao W (2016) Subjective and objective quality assessment of compressed screen content images[J]. IEEE J Emerg Select Top Circ Syst 6(4):532–543

    Google Scholar 

  41. Wang S, Gu K, Zhang X, Lin W, Ma S, Gao W (2018) Reduced-reference quality assessment of screen content images[J]. IEEE Transact Circ Syst Vid Technol 28(1):1–14

    Google Scholar 

  42. Yang X, Sun Q, Wang T (2014). Completely blind image quality assessment based on gray-scale fluctuations [C]. International conference on digital image processing. 915916.

  43. Yang H, Fang Y, Lin W (2015) Perceptual quality assessment of screen content images[J]. IEEE Trans Image Process 24(11):4408–4421

    MathSciNet  MATH  Google Scholar 

  44. Yang X, Sun Q, Wang T (2018) Image quality assessment via spatial structural analysis[J]. Comput Electr Eng 70:349–365

    Google Scholar 

  45. Yang X, Sun Q, Wang T (2018) Completely blind image quality assessment via image gray-scale fluctuations and fractal dimension analysis[J]. Appl Opt 57(12):3268–3280

    Google Scholar 

  46. Yang J, Sim K, Jiang B, Lu W (2018) Blind image quality assessment utilising local mean eigenvalues[J]. Electron Lett 54(12):754–756

    Google Scholar 

  47. Yang X, Sun Q, Wang T (2019) No-reference image quality assessment based on sparse representation[J]. Neural Comput Applic 31(10):6643–6658

    Google Scholar 

  48. Yao J, Liu G (2018) Improved SSIM image quality assessment of contrast distortion based on the contrast sensitivity characteristics of human visual system[J]. IET Image Process 12(6):872–879

    Google Scholar 

  49. Yeh CH, Lo SH, Lin W (2019) Visual-quality guided global backlight dimming for video display on Mobile devices[J]. IEEE Transact Circ Syst Vid Technol 29(11):3393–3403

    Google Scholar 

  50. Yue G, Hou C, Gu K, Ling N, Li B (2018) Analysis of structural characteristics for quality assessment of multiply distorted images[J]. IEEE Transact Multimed 20(10):2722–2732

    Google Scholar 

  51. Zhang L, Zhang L, Mou X et al (2011) FSIM: a feature similarity index for image quality assessment[J]. IEEE Trans Image Process 20(8):2378–2386

    MathSciNet  MATH  Google Scholar 

  52. Zhang L, Zhang L, Bovik AC (2015) A feature-enriched completely blind image quality evaluator.[J]. IEEE Trans Image Process 24(8):2579–2591

    MathSciNet  MATH  Google Scholar 

Download references

Funding

This work was supported in part by the National Natural Science Foundation of China under Grant No. 41971343.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Nanjing Normal University, No. 1 Wenyuan Road Qixia District, Nanjing, 210046, China

    Xichen Yang & Genlin Ji

  2. School of Artificial Intelligence and Information Technology, Nanjing University of Chinese Medicine, No. 138 Xianlin Road Qixia District, Nanjing, 210023, China

    Tianshu Wang

Authors
  1. Xichen Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tianshu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Genlin Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceived and designed the experiments: Xichen Yang. Performed the experiments: Xichen Yang. Analyzed the data: Xichen Yang. Wrote and reviewed the paper: Xichen yang, Genlin Ji, Tianshu Wang. Approved the final version of the paper: Xichen yang, Genlin Ji, Tianshu Wang.

Corresponding author

Correspondence to Xichen Yang.

Ethics declarations

Declaration

The authors have declared that there are no competing interests exist.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, X., Wang, T. & Ji, G. A local structural information representation method for image quality assessment. Multimed Tools Appl 79, 22797–22823 (2020). https://doi.org/10.1007/s11042-020-09022-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09022-1

Keywords

Search

Navigation