Skip to main content
SpringerLink
Log in

A perceptual metric for stereoscopic image quality assessment based on the binocular energy

  • Published:
Multidimensional Systems and Signal Processing Aims and scope Submit manuscript

Abstract

Stereoscopic imaging is becoming very popular and its deployment by means of photography, television, cinema. . .is rapidly increasing. Obviously, the access to this type of images imposes the use of compression and transmission that may generate artifacts of different natures. Consequently, it is important to have appropriate tools to measure the quality of stereoscopic content. Several studies tried to extend well-known metrics, such as the PSNR or SSIM, to 3D. However, the results are not as good as for 2D images and it becomes important to have metrics dealing with 3D perception. In this work, we propose a full reference metric for quality assessment of stereoscopic images based on the binocular fusion process characterizing the 3D human perception. The main idea consists of the development of a model allowing to reproduce the binocular signal generated by simple and complex cells, and to estimate the associated binocular energy. The difference of binocular energy has shown a high correlation with the human judgement for different impairments and is used to build the Binocular Energy Quality Metric (BEQM). Extensive experiments demonstrated the performance of the BEQM with regards to literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Adelson E. H., Bergen J. R. (1985) Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America 2(2): 284–299

    Article  Google Scholar 

  • Akhter, R., Sazzad, Z. M. P., Horita, Y., & Baltes, J. (2010). No reference stereoscopic image quality assessment. In Image quality and system performance (vol. 7524, pp. 17–21). San Jose, California, USA.

  • Ates H. F., Orchard M. T. (2003) A nonlinear image representation in wavelet domain using complex signals with single quadrant spectrum. Asilomar Conference on Signals, Systems, Computers 2: 1966–1970

    Google Scholar 

  • Avcibas I., Sankur B., Sayood K. (2002) Statistical evaluation of image quality measures. Journal of Electronic Imaging 11(2): 206–223

    Article  Google Scholar 

  • Barlow H. B., Blakemore C., Pettigrew J. D. (1967) The neural mechanism of binocular depth discrimination. Journal of Physiology 193: 327–342

    Google Scholar 

  • Bensalma, R., & Larabi, M. C. (2010). Stereo image coding based on binocular energy modeling. In International conference image processing (ICIP) (pp. 2989–2992).

  • Bensalma, R., & Larabi, M. C. (2010). Towards a perceptual quality metric for color stereo images. In International conference image processing (ICIP) (pp. 4037–4040). Hong Kong.

  • Blake R., Wilson H. R. (1991) Neural models of stereoscopic vision. Trends in neurosciences. International Journal of Computer Vision 14: 445–452

    Google Scholar 

  • Boev, A., Gotchev, A., Egiazarian, K. O., Aksay, A., & Akar, G. B. (2010). Towards compound stereo-video quality metric: A specific encoder-based framework. In IEEE SSIAI (pp. 218–222). Denver, Colorado, USA.

  • Campbell F. W., Cooper G. F., Enroth-Cugell C. (1969) The spatial selectivity of the visual cells of the cat. Journal of Physiology 203: 223–235

    Google Scholar 

  • Campisi, P., LeCallet, P., & Marini, E. (2007). Stereoscopic images quality assessment. In European signal processing conference. Poznan, Poland.

  • Candès E., ans Demanet L., Donoho D., Ying L. (2006) Fast discrete curvelet transforms. Multiscale Modelling and Simulation 5(3): 861–899

    Article  MATH  Google Scholar 

  • Cheng, I., & Boulanger, P. (2005). A 3D perceptual metric using just-noticeable-difference. In: In Eurographics Short Presentations (pp. 97–100).

  • Damera-Venkata N., Kite T. D., Evans B. L., Geisler W. S., Bovik A. C. (2000) Image quality assessment based on a degradation model. IEEE Transaction Image Processing 4(4): 636–650

    Article  Google Scholar 

  • DeAngelis G. C., Ohzawa I., Freeman R. D. (1991) Depth is encoded in the visual cortex by a specialized receptive field structure. Journal on Nature 352: 156–195

    Article  Google Scholar 

  • Do, M., & Vetterli, M. (2001). Pyramidal directional filter banks and curvelets. IEEE proceedings on international conference image processing.

  • Donghyun K., Dongbo M., Juhyun O., Kwanghoon S., Seonggyu J. (2009) Depth map quality metric for three-dimensional video. Image Quality and System Performance 7237(29): 723719

    Google Scholar 

  • Ellinas J. N., Sangriotis M. S. (2004) Stereo image compression using wavelet coefficients morphology. Image and Vision Computing 22(4): 281–290

    Article  Google Scholar 

  • Field D. J., Tolhurst D. J. (1986) The structure and symmetry of simple-cell receptive field profiles in the cat’s visual cortex. Proceedings of the Royal Society of London 228: 379–400

    Article  Google Scholar 

  • Fleet D. J., Wagner H., Heeger D. J. (1996) Neural encoding of binocular disparity: Energy model, position shifts and phase shifts. Vision Research 36(12): 1839–1857

    Article  Google Scholar 

  • Foster K.H., Gaska J. P., Marcelja S., Pollen D.A. (1983) Phase relationships between adjacent simple cells in the feline visual cortex. Journal of Physiology 345: 22

    Google Scholar 

  • Goldmann, L., & Ebrahimi, T. (2010) 3D quality is more than just the sum of 2D and depth. In IEEE International workshop on hot topics in 3D.

  • Gorley, P., & Holliman, N. (2008). Stereoscopic image quality metrics and compression. In Image quality and system performance, vol. 6803 (pp. 1–11). San Jose, California, USA.

  • Hewage, C., & Martini, M.G. (2010). Reduced-reference quality metric for 3D depth map transmission. In IEEE 3DTV conference (pp. 1–4). Tampere, Finland.

  • Hubel D. H., Wiesel T. N. (1962) Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. Journal of Physiology 160: 106–154

    Google Scholar 

  • Hubel D. H., Wiesel T. N. (1970) Stereoscopic vision in macaque monkey. cells sensitive to binocular depth in area 18 of the macaque monkey cortex. Journal of Nature 225: 41–42

    Article  Google Scholar 

  • Jones J. P., Palmer L. A. (1987) An evaluation of the two-dimensional gabor filter model of simple receptive fields in cat striate cortex. Journal of Neurophysiology 58(6): 1233–1258

    Google Scholar 

  • Kaptein, R. G., Kuijsters, A., Lambooij, M. T. M., IJsselsteijn, W. A., & Heynderickx, I. (2008). Performance evaluation of 3D-TV systems. In image quality and system performance (pp. 1–11). San Jose, California, USA.

  • Keita, T., & Takeshi, N. (2005). Unstructured light field rendering using on-the-fiy focus measurements. In IEEE international conference on multimedia and expo.

  • Kingsbury N. (1997) Image processing with complex wavelets. Philosophical Transactions on Royal Society London A 357: 2543–2560

    Article  Google Scholar 

  • Kuffler S. W. (1953) Discharge patterns and functional organization of mammalian retina. Journal of Physiology 16: 37–68

    Google Scholar 

  • Lavoué G., Gelasca E. D., Dupont F., Baskurt A., Ebrahimi T. (2006) Perceptually driven 3D distance metrics with application to watermarking. Image Quality and System Performance 6312(29): 63120L

    Google Scholar 

  • Le Pennec E., Mallat S. (2005) Bandelet image approximation and compression. SIAM Multiscale Modeling and Simulation 4(3): 992–1039

    Article  MathSciNet  MATH  Google Scholar 

  • Liu A., Gaska J. P., Jacobson L. D., Pollen D. A. (1992) Interneuronal interaction between members of quadrature phase and anti-phase pairs in the cat’s visual cortex. Vision Research 32: 1193–1198

    Article  Google Scholar 

  • Mallat S. (1989) A theory for multiresolution signal decomposition : the wavelet represntation. IEEE, PAMI 11(7): 674–693

    Article  MATH  Google Scholar 

  • Mallat S., Peyré G. (2006) Orthogonal bandelet bases for geometric image approximation. Communications on Pure and Applied Mathematics 61(9): 1173–1212

    Google Scholar 

  • Media Information and Communication Technology (MICT) Lab-oratory. (2011). Mict image quality evaluation database. http://mict.eng.u-toyama.ac.jp/mict/index2.html. Accessed 24 Sept 2011.

  • Meesters L. M. J., IJsselsteijn W. A., Seuntiens P. J. H. (2004) A survey of perceptual evaluations and requirements of three-dimensional TV. IEEE Transactions on Circuits and Systems for Video Technology 14(3): 381–391

    Article  Google Scholar 

  • Miyahara M., Kotani K., Algazi V. R. (1998) Objective picture quality scale (PQS) for image coding. IEEE Transaction Communications 46(9): 1215–1225

    Article  Google Scholar 

  • Nath S. K., Dubois E. (2006) An improved, wavelet-based, stereoscopic image sequence codec with SNR and spatial scalability. Signal processing Image communication 21(3): 181–199

    Article  Google Scholar 

  • Ohzawa I., Freeman R. D. (1986) The binocular organization of simple cells in the cat’s visual cortex. Journal of Neurophysiology 56: 221–242

    Google Scholar 

  • Ohzawa I., Freeman R. D. (1986) The binocular organization of complex cells in the cat’s visual cortex. Journal of Neurophysiology 56: 243–259

    Google Scholar 

  • Olsson, R., & Sjostrom, M. A. (2007). Depth dependent quality metric for evaluation of coded integral imaging based 3D-images. In IEEE 3DTV. Kos, Greece.

  • Palmer L. A., Davis T. L. (1981) Receptive-field structure in cat striate cortex. Vision Research 46: 260–276

    Google Scholar 

  • Peyre, G. (2005). Geometrie multi-échelles pour les images et les textures. Ph.D. thesis, Ecole Polytechnique.

  • Pollen D. A., Ronner S. (1981) Phase relationships between adjacent simple cells in the visual cortex. Science 212: 1409–1411

    Article  Google Scholar 

  • Rittermann, M. A. (2004). A proposal for the quality assessment of 3D video objects. In International workshop on image analysis for multimedia interactiveservices. Lisboa, Portugal.

  • Sarnoff Corporation (2003) JND metrix technology. evaluation (2003) Version available http://www.sarnoff.com/productsservices/videovision/jndmetrix/downloads.asp.

  • Schanda J. (2007) Colorimetry: Understanding the CIE System. Wiley, Hoboken, NJ, USA

    Google Scholar 

  • Scharstein D., Szeliski R. (2002) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision 47(1/2/3): 7–42

    Article  MATH  Google Scholar 

  • Scharstein, D., & Szeliski, R. (2002). Middlebury stereo vision page. vision middlebury data base (2002) Version available http://vision.middlebury.edu/stereo/.

  • Selesnick I.W., Baraniuk R.G., Kingsbury N.G. (2005) The dual-tree complex wavelet transform. IEEE Signal Processing Magazine 22(6): 123–151

    Article  Google Scholar 

  • Sheikh H. R., Bovik A. C., deVeciana G. (2005) An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Transactions on Image Processing 14(12): 2117–2128

    Article  Google Scholar 

  • Sheikh H. R., Bovik A. C. (2006) Image information and visual quality. IEEE Transactions on Image Processing 15(2): 430–444

    Article  Google Scholar 

  • Tikanmäki, A., Gotchev, A., Smolic, A., & Müller, K. (2008). Quality assessment of 3D video in rate allocation experiments. In IEEE international symposium on consumer electronics. Algarve, Portugal.

  • VQEG (2008). Final report from the video quality experts group on the validation of objective models of multimedia quality assessement. Technical Report on PHASE I 2008, VQEG.

  • Wang Z., Lu L., Bovik A. C. (2004) Video quality assessment based on structural distortion measurement. Signal Processing Image Communication 19(2): 121–132

    Article  Google Scholar 

  • Wang, Z., Simoncelli, E.P., & Bovik, A. C. (2003). Multi-scale structural similarity for image quality assessment. In IEEE asilomar conference on signals, systems and computers (pp. 1398–1402). Pacific Grove, CA.

  • Watson A. B. (1993) Dctune: A technique for visual optimization of dct quantization matrices for individual images. Society for Information Display Digest of Technical Papers 1: 946–949

    Google Scholar 

  • Weken D. V., Nachtegael M., Kerre E. E. (2004) Using similarity measures and homogeneity for the comparison of images. Image and Vision Computing 22(9): 695–702

    Article  Google Scholar 

  • Woo, W., Ortega, A., & Iwadate, Y. (1999). Stereo image coding based on binocular energy modeling. In International conference image processing (ICIP) (pp. 467–471).

  • Xing, L., You, J., Ebrahimi, T., & Perkis, A. (2010). A perceptual quality metric for stereoscopic crosstalk perception. In IEEE international conference on image processing (pp. 4033–4036). Hong Kong, China.

  • You, J., Xing, L., Perkis, A., & Wang, X. (2010). Assessment for stereoscopic images based on 2D image quality metrics and disparity analysis. In International workshop on video processing and quality metrics. Scottsdale, Arizona, USA.

Download references

Author information

Authors and Affiliations

  1. XLIM-SIC, UMR CNRS 7252, Université de Poitiers, Poitiers, France

    Rafik Bensalma & Mohamed-Chaker Larabi

Authors
  1. Rafik Bensalma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohamed-Chaker Larabi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohamed-Chaker Larabi.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bensalma, R., Larabi, MC. A perceptual metric for stereoscopic image quality assessment based on the binocular energy. Multidim Syst Sign Process 24, 281–316 (2013). https://doi.org/10.1007/s11045-012-0178-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11045-012-0178-3

Keywords

Search

Navigation