Skip to main content
SpringerLink
Log in

User models of subjective image quality assessment on virtual viewpoint in free-viewpoint video system

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper, we present a user model of subjective quality assessment on virtual viewpoint image (VVI) for free-viewpoint video system. VVIs are rendered through neighbor viewpoint color and depth images, and it is a new type of image that generated for human-computer interaction (HCI) in free-viewpoint video system. In this system, a natural scene is captured by multi-viewpoint cameras, and users can view the scene from any desired viewpoint, regardless the real or virtual one. The subjective quality of VVIs is crucial for the quality of experiences for HCI, because the magnitude of VVI is much greater than the real. In order to find the user model of VVI quality assessment, we organize three sets of stimuli, including Symmetric Stimuli, Asymmetric Stimuli Part I and Part II, to reveal the psychological responses of participants. A psychometric function is consequently obtained to determine the relationship between stimulus and psychological responses. Further discussions on the factors of distortion level, gender, age and academic background are examined to find the influence on the user model. We find that the distortion level of neighbor viewpoint color images has the dominant impact on the user model, while other factors contribute little.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Bex P (2008) Sensitivity to spatial distortion in natural scenes. J Vis 8(6):688

    Article  Google Scholar 

  2. Bex PJ, Solomon SG, Dakin SC (2009) Contrast sensitivity in natural scenes depends on edge as well as spatial frequency structure. J Vis 9(10)

  3. Bosc E, Pepion R, Le Callet P, Koppel M, Ndjiki-Nya P, Pressigout M, Morin L (2011) Towards a new quality metric for 3-d synthesized view assessment. IEEE J Sel Top Sign Process 5(7):1332–1343

    Article  Google Scholar 

  4. Buehren T, Collins MJ (2006) Accommodation stimulus–response function and retinal image quality. Vis Res 46(10):1633–1645

    Article  Google Scholar 

  5. Call for proposals on 3d video coding technology. ISO/IEC JTC1/SC29/WG11, Doc W12036 (2011)

  6. Chan S, Shum HY, Ng KT (2007) Image-based rendering and synthesis. IEEE Signal Proc Mag 24(6):22–33

    Article  Google Scholar 

  7. Chen L, Singer B, Guirao A, Porter J, Williams DR (2005) Image metrics for predicting subjective image quality. Optom Vis Sci 82(5):358–369

    Article  Google Scholar 

  8. Gao Y, Tang J, Hong R, Yan S, Dai Q, Zhang N, Chua TS (2012) Camera constraint-free view-based 3-d object retrieval. IEEE Trans Image Process 21(4):2269–2281

    Article  MathSciNet  Google Scholar 

  9. Gao Y, Wang M, Ji R, Wu X, Dai Q (2014) 3d object retrieval with hausdorff distance learning. IEEE Trans Ind Electron 61(4):2088–2098

    Article  Google Scholar 

  10. Gao Y, Wang M, Tao D, Ji R, Dai Q (2012) 3d object retrieval and recognition with hypergraph analysis. IEEE Trans Image Process 21(9):4290–4303

    Article  MathSciNet  Google Scholar 

  11. Ji R, Gao Y, Hong R, Liu Q, Tao D, Li X (2014) Spectral-spatial constraint hyperspectral image classification. IEEE Trans Geosci Remote Sens 52(3):1811–1824

    Article  Google Scholar 

  12. Ji R, Gao Y, Zhong B, Yao H, Tian Q (2011) Mining flickr landmarks by modeling reconstruction sparsity. ACM Trans Multimedia Comput, Commun, and Appl (TOMCCAP) 7(1):31

    Google Scholar 

  13. Karsten M, Aljoscha S, Kristina D, Philipp M, Peter K, Thomas W et al (2009) View synthesis for advanced 3d video systems. EURASIP J Image Video Process 2008

  14. Klimaszewski K, Wegner K, Domanski M (2009) Distortions of synthesized views caused by compression of views and depth maps. In: IEEE of 3dtv conference: The true vision-capture, transmission and display of 3d video, pp 1–4

  15. Lee C, Ho YS (2008) View synthesis tools for 3d video. ISO/IEC JTC1/SC29/WG11, Doc M15851

  16. Liu Q, Yang Y, Gao Y, Hong R (2013) Texture-adaptive hole-filling algorithm in raster-order for three-dimensional video applications. Neurocomputing 111:154–160

    Article  Google Scholar 

  17. Liu Q, Yang Y, Gao Y, Ji R, Yu L (2013) A bayesian framework for dense depth estimation based on spatial–temporal correlation. Neurocomputing 104:1–9

    Article  Google Scholar 

  18. Liu Y, Huang Q, Ma S, Zhao D, Gao W (2009) Joint video/depth rate allocation for 3d video coding based on view synthesis distortion model. Signal Process Image Commun 24(8):666–681

    Article  Google Scholar 

  19. Marr D, Hildreth E (1980) Theory of edge detection. Proc R Soc B Biol Sci 207(1167):187–217

    Article  Google Scholar 

  20. McIlhagga WH, May KA (2012) Optimal edge filters explain human blur detection. J Vis 12(10)

  21. Methodology for the subjective assessment of the quality of television pictures. ITU-R BT.500-11 (2002)

  22. Mittal A, Soundararajan R, Bovik A (2013) Prediction of image naturalness and quality. J Vis 13(9):1056–1056

    Article  Google Scholar 

  23. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: From error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  24. Wiegand T, Sullivan GJ, Bjontegaard G, Luthra A (2003) Overview of the h. 264/avc video coding standard. IEEE Trans Circ Syst for Video Technol 13(7):560–576

    Article  Google Scholar 

  25. Wolfe JM (1994) Visual search in continuous, naturalistic stimuli. Vis Res 34(9):1187–1195

    Article  MathSciNet  Google Scholar 

  26. Yang Y, Dai Q (2010) Contourlet-based image quality assessment for synthesised virtual image. IET Electron Lett 46(7):492–494

    Article  Google Scholar 

  27. Yang Y, Liu Q, Ji R, Gao Y (2012) Dynamic 3d scene depth reconstruction via optical flow field rectification. PloS ONE 7(11):e47,041

    Article  Google Scholar 

  28. Yang Y, Liu Q, Liu H, Yu L, Wang F (2014) Dense depth image synthesis via energy minimization for three-dimensional video. Signal Process. doi:10.1016/j.sigpro.2014.07.020

    Google Scholar 

  29. Zhang L, Gao Y, Lu K, Shen J, Ji R (2014) Representative discovery of structure cues for weakly-supervised image segmentation. IEEE Trans Multimedia 16(2):470–479

    Article  Google Scholar 

  30. Zhang L, Gao Y, Roger Z, Tian Q, Li X (2014) Fusion of multichannel local and global structural cues for photo aesthetics evaluation. IEEE Trans Image Process 23(3):1419–1429

    Article  MathSciNet  Google Scholar 

  31. Zhang L, Han Y, Yang Y, Song M, Yan S, Tian Q (2013) Discovering discriminative graphlets for aerial image categories recognition. IEEE Trans Image Process 22(12):5071–5084

    Article  MathSciNet  Google Scholar 

  32. Zhang L, Song M, Liu X, Bu J, Chen C (2013) Fast multi-view segment graph kernel for object classification. Signal Process 93(6):1597–1607

    Article  Google Scholar 

  33. Zhang L, Song M, Zhao Q, Liu X, Bu J, Chen C (2013) Probabilistic graphlet transfer for photo cropping. IEEE Trans Image Process 22(2):802–815

    Article  MathSciNet  Google Scholar 

  34. Zhang L, Tam WJ (2005) Stereoscopic image generation based on depth images for 3d tv. IEEE Trans Broadcast 51(2):191–199

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by Natural Science Foundation of China (NSFC) (Grant Nos. 61170194 and 61202301).

Author information

Authors and Affiliations

  1. School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan, 430074, China

    You Yang & Qiong Liu

  2. Department of Automation, Tsinghua University, Beijing, 100086, China

    You Yang

  3. Department of Computer Science, City University of Hong Kong, Kowloon Tong, Hong Kong

    Xu Wang

  4. School of Information Engineering at Zhengzhou University, Zhengzhou, China

    Mingliang Xu

  5. China Academy of Civil Aviation Science and Technology, Beijing, China

    Wei Wu

Authors
  1. You Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mingliang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wei Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qiong Liu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, Y., Wang, X., Liu, Q. et al. User models of subjective image quality assessment on virtual viewpoint in free-viewpoint video system. Multimed Tools Appl 75, 12499–12519 (2016). https://doi.org/10.1007/s11042-014-2321-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-014-2321-7

Keywords

Search

Navigation