Skip to main content
SpringerLink
Log in

Hybrid linear weighted prediction and intra block copy based light field image coding

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Light field imaging can capture both spatial and angular information of a 3D scene and is considered as a prospective acquisition and display solution to supply a more natural and fatigue-free 3D visualization. However, one problem that occupies an important position to deal with the light field data is the sheer size of data volume. In this context, efficient coding schemes for this particular type of image are needed. In this paper, we propose a hybrid linear weighted prediction and intra block copy based light field image codec architecture based on high efficiency video coding screen content coding extensions (HEVC SCC) standard to effectively compress the light field image data. In order to improve the prediction accuracy, a linear weighted prediction method is integrated into HEVC SCC standard, where a locally correction weighted based method is used to derive the weight coefficient vector. However, for the non-homogenous texture area, a best match in linear weighted prediction method does not necessarily lead to a good prediction of the coding block. In order to alleviate such shortcoming, the proposed hybrid codec architecture explores the idea of using the intra block copy scheme to find the best prediction of the coding block based on rate-distortion optimization. For the reason that the used “try all then select best” intra mode decision method is time-consuming, we further propose a fast mode decision scheme for the hybrid codec architecture to reduce the computation complexity. Experimental results demonstrate the advantage of the proposed hybrid codec architecture in terms of different quality metrics as well as the visual quality of views rendered from decompressed light field content, compared to the HEVC intra-prediction method and several other prediction methods in this field.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Adelson EH, Bergen JR (1991) The plenoptic function and the elements of early vision. In: Computational Models of Visual Processing pp 3–20. Cambridge: MIT Press

  2. Aggoun A, Tsekleves E, Swash MR, Zarpalas D, Dimou A, Daras P, Nunes P, Soares LD (2013) Immersive 3D holoscopic video system. IEEE Multimedia 20:28–37

    Article  Google Scholar 

  3. Cherigui S, Guillemot C, Thoreau D, Guillotel P, Perez P (2013) Correspondence Map-Aided Neighbor Embedding for Image Intra Prediction. IEEE Trans Image Process 22(3):1161–1174

    Article  MathSciNet  Google Scholar 

  4. Conti C, Soares LD, Nunes P (2016) HEVC-based 3D holoscopic video coding using self-similarity compensated prediction. Signal Process Image Commun 42:59–78

    Article  Google Scholar 

  5. Conti C, Nunes P, Soares LD (2016) HEVC-based light field image coding with bi-predicted self-similarity compensation. 2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pp. 1–4

  6. Dai F, Zhang J, Ma Y and Zhang Y (2015) Lenselet image compression scheme based on subaperture images streaming. 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, pp. 4733–4737

  7. Ebrahimi T (2015) JPEG PLENO abstract and executive summary, ISO/IEC JTC 1/SC 29/WG1 N6922, Sydney, Australia

  8. Georgiev T 2013 (Online), Available: http://www.tgeorgiev.net, Website (Online)

  9. Helin P, Astola P, Rao B, Tabus I (2016) Sparse modelling and predictive coding of subaperture images for lossless plenoptic image compression. 2016 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON) pp. 1–4

  10. HEVC SCC Reference Software Ver. 3.0 (SCM-3.0). [Online]. Available: https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/tags/HM-16.2+SCM-3.0/

  11. Kalantari NK, Wang TC, Ramamoorthi R (2016) Learning-Based View Synthesis for Light Field Cameras. ACM Trans Graph 35(6):193

    Article  Google Scholar 

  12. Lei J, Li D, Pan Z, Sun Z, Kwong S, Hou C (2017) Fast Intra Prediction Based on Content Property Analysis for Low Complexity HEVC-Based Screen Content Coding. IEEE Trans Broadcast 63(1):48–58

    Article  Google Scholar 

  13. M. Levoy, “Light fields and computational imaging, Computer, vol. 39, pp. 46–55, (2006)

    Article  Google Scholar 

  14. Levoy M, Hanrahan P (1996) Light field rendering. In Proc. 23rd Annu. Conf Comput Graph Interact Techn pp. 31–42

  15. Li Y, Sjostrom M, Olsson R, Jennehag U (2016) Coding of Focused Plenoptic Contents by Displacement Intra Prediction. IEEE Transactions on Circuits and Systems for Video Technology 26(7):1308–1319

    Article  Google Scholar 

  16. Li L, Li Z, Li B, Liu D, Li H (2017) Pseudo Sequence Based 2-D Hierarchical Coding Structure for Light-Field Image Compression. 2017 Data Compression Conference (DCC), pp. 131–140

  17. Liu D, An P, Ma R, Shen L (2015) Disparity compensa-tion based 3D holoscopic image coding using HEVC. In 2015 IEEE China Summit & Int. Conf. Signal and Information Processing (ChinaSIP), pp. 201–205

  18. Liu Y, Nie L, Zhang L, Rosenblum DS (2015) Action2activity: Recognizing complex activities from sensor data. In IJCAI'15 Proceedings of the 24th International Conference on Artificial Intelligence, pp. 1617–1623

  19. Liu D, Wang L, Li L, Xiong Z, Wu F, Zeng W (2016) Pseudo-sequence-based light field image compression. 2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pp. 1–4

  20. Liu D, An P, Ma R, Yang C, Shen L, Li K (2016) Three-dimensional holoscopic image coding scheme using high-efficiency video coding with kernel-based minimum mean-square-error estimation. J Electron Imaging 25(4):043015-1–043015-9

    Article  Google Scholar 

  21. Liu D, An P, Ma R, Yang C, Shen L (2016) 3D holoscopic image coding scheme using HEVC with Gaussian process regression. Signal Process Image Commun 47:438–451

    Article  Google Scholar 

  22. Liu Y, Zhang L, Nie L, Yan Y, Rosenblum DS (2016) Fortune teller: Predicting your career path. In Proceedings of the Thirtieth AAAI conference on artificial intelligence, pp. 201–207

  23. Liu Y, Zheng Y, Liang Y, Liu S, Rosenblum DS (2016) Urban Water Quality Prediction based on Multi-task Multi-view Learning. In Proceedings of the 25th International Joint Conference on Artificial Intelligence, pp. 1–7

  24. Liu Y, Nie L, Liu L, Rosenblum DS (2016) From action to activity: Sensor-based activity recognition. Neurocomputing 181:108–115

    Article  Google Scholar 

  25. Liu F, Hou G, Sun Z, Tan T (2017) High quality depth map estimation of object surface from light-field images. Neurocomputing 252:3–16

    Article  Google Scholar 

  26. Liu D, An P, Yang C, Ma R, Shen L (2017) Coding of 3D holoscopic image by using spatial correlation of rendered view images. In 42th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2017), pp. 2002–2006

  27. Lucas LFR, Conti C, Nunes P, Soares LD, Rodrigues NMM, Pagliari CL, da Silva EAB, de Faria SMM (2014) Locally linear embedding-based predic-tion for 3D holoscopic image coding using HEVC. In 2014 Proceedings of the 22nd European Signal Processing Conference (EUSIPCO), pp. 11, 15, 1–5

  28. Monteiro R et al (2016) Light field HEVC-based image coding using locally linear embedding and self-similarity compensated prediction. 2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pp. 1–4

  29. Monteiro RJS, Nunes PJL, Rodrigues NMM et al (2017) Light Field Image Coding using High Order Intra Block Prediction. IEEE Journal on Selected Topics in Signal Processing 11(7):1120–1131

    Article  Google Scholar 

  30. Podder PK, Paul M, Murshed M (2016) A novel motion classification based inter mode selection strategy for HEVC performance improvement. Neurocomputing 173:1211–1220

    Article  Google Scholar 

  31. Rerabek M, Bruy lants T, Ebrahimi T, Pereira F, Schelkens P (2016) Call for Proposals and Evaluation Procedure. ICME 2016 Grand Challenge: Light Field Image Compression, Seattle, USA, pp. 1–8

  32. L. Shen et al. “Adaptive inter-mode decision for HEVC jointly utilizing inter-level and spatio-temporal correlations, ” IEEE Trans Syst Video Technol Vol. 24, no. 10, pp. 1709–1722, (2014)

  33. Tan TK, Boon CS, Suzuki Y (2006) Intra prediction by template matching. In IEEE Int Conf Image Processing (ICIP), IEEE, pp, 1693–1696

  34. Tehrani MP, Shimizu S, Lafruit G, Senoh T, Fujii T, Vetro A, et al. (2013) Use Cases and Requirements on Free-viewpoint Television (FTV), ISO/IEC JTC1/SC29/WG11 MPEG N14104, Geneva, Switzer-land

  35. Turkan M, Guillemot C (2012) Image prediction based on neighbor-embedding methods. IEEE Trans Image Process 21(4):1885–1898

    Article  MathSciNet  Google Scholar 

  36. Wang G, Xiang W, Pickering M, Chen CW (2016) Light Field Multi-View Video Coding With Two-Directional Parallel Inter-View Prediction. IEEE Trans Image Process 25(11):5104–5117

    Article  MathSciNet  Google Scholar 

  37. Xu J, Joshi R, Cohen RA (2016) Overview of the Emerging HEVC Screen Content Coding Extension. IEEE Transactions on Circuits and Systems for Video Technology 26(1):50–62

    Article  Google Scholar 

  38. Yang R, Huang X, Li S, Jaynes C (2008) Toward the light field display: Autostereoscopic rendering via a cluster of projectors. IEEE Trans Vis Comput Graphics 14(1):84–96

    Article  Google Scholar 

  39. Yu H, Cohen R, Rapaka K, Xu J (2016) Common test conditions for screen content coding, document JCTVC-X1015

  40. Zhang Q et al (2016) An efficient depth map filtering based on spatial and texture features for 3D video coding. Neurocomputing 188:82–89

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China, under Grants 61571285, U1301257, and Scientific Research Staring Foundation 055-170002004, and the Key Project on Anhui Provincial Natural Science Study by Colleges and Universities No. KJ2018A0361. This work is also supported by the Foundation of University Research and Innovation Platform Team for Intelligent Perception and Computing of Anhui Province.

Author information

Authors and Affiliations

  1. School of Computer and Information, Anqing Normal University, Anqing, China

    Deyang Liu

  2. The University Key Laboratory of Intelligent Perception and Computing of Anhui Province, Anqing, China

    Deyang Liu

  3. School of Communication and Information Engineering, Shanghai University, Shanghai, China

    Ping An, Ran Ma & Liquan Shen

Authors
  1. Deyang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ping An
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ran Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Liquan Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Deyang Liu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, D., An, P., Ma, R. et al. Hybrid linear weighted prediction and intra block copy based light field image coding. Multimed Tools Appl 77, 31929–31951 (2018). https://doi.org/10.1007/s11042-018-6255-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6255-3

Keywords

Search

Navigation