Multimedia Tools and Applications

, Volume 77, Issue 24, pp 31929–31951 | Cite as

Hybrid linear weighted prediction and intra block copy based light field image coding

  • Deyang LiuEmail author
  • Ping An
  • Ran Ma
  • Liquan Shen


Light field imaging can capture both spatial and angular information of a 3D scene and is considered as a prospective acquisition and display solution to supply a more natural and fatigue-free 3D visualization. However, one problem that occupies an important position to deal with the light field data is the sheer size of data volume. In this context, efficient coding schemes for this particular type of image are needed. In this paper, we propose a hybrid linear weighted prediction and intra block copy based light field image codec architecture based on high efficiency video coding screen content coding extensions (HEVC SCC) standard to effectively compress the light field image data. In order to improve the prediction accuracy, a linear weighted prediction method is integrated into HEVC SCC standard, where a locally correction weighted based method is used to derive the weight coefficient vector. However, for the non-homogenous texture area, a best match in linear weighted prediction method does not necessarily lead to a good prediction of the coding block. In order to alleviate such shortcoming, the proposed hybrid codec architecture explores the idea of using the intra block copy scheme to find the best prediction of the coding block based on rate-distortion optimization. For the reason that the used “try all then select best” intra mode decision method is time-consuming, we further propose a fast mode decision scheme for the hybrid codec architecture to reduce the computation complexity. Experimental results demonstrate the advantage of the proposed hybrid codec architecture in terms of different quality metrics as well as the visual quality of views rendered from decompressed light field content, compared to the HEVC intra-prediction method and several other prediction methods in this field.


Light field image Linear weighted prediction Intra block copy Fast mode decision HEVC SCC 



This work was supported in part by the National Natural Science Foundation of China, under Grants 61571285, U1301257, and Scientific Research Staring Foundation 055-170002004, and the Key Project on Anhui Provincial Natural Science Study by Colleges and Universities No. KJ2018A0361. This work is also supported by the Foundation of University Research and Innovation Platform Team for Intelligent Perception and Computing of Anhui Province.


  1. 1.
    Adelson EH, Bergen JR (1991) The plenoptic function and the elements of early vision. In: Computational Models of Visual Processing pp 3–20. Cambridge: MIT PressGoogle Scholar
  2. 2.
    Aggoun A, Tsekleves E, Swash MR, Zarpalas D, Dimou A, Daras P, Nunes P, Soares LD (2013) Immersive 3D holoscopic video system. IEEE Multimedia 20:28–37CrossRefGoogle Scholar
  3. 3.
    Cherigui S, Guillemot C, Thoreau D, Guillotel P, Perez P (2013) Correspondence Map-Aided Neighbor Embedding for Image Intra Prediction. IEEE Trans Image Process 22(3):1161–1174MathSciNetCrossRefGoogle Scholar
  4. 4.
    Conti C, Soares LD, Nunes P (2016) HEVC-based 3D holoscopic video coding using self-similarity compensated prediction. Signal Process Image Commun 42:59–78CrossRefGoogle Scholar
  5. 5.
    Conti C, Nunes P, Soares LD (2016) HEVC-based light field image coding with bi-predicted self-similarity compensation. 2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pp. 1–4Google Scholar
  6. 6.
    Dai F, Zhang J, Ma Y and Zhang Y (2015) Lenselet image compression scheme based on subaperture images streaming. 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, pp. 4733–4737Google Scholar
  7. 7.
    Ebrahimi T (2015) JPEG PLENO abstract and executive summary, ISO/IEC JTC 1/SC 29/WG1 N6922, Sydney, AustraliaGoogle Scholar
  8. 8.
    Georgiev T 2013 (Online), Available:, Website (Online)
  9. 9.
    Helin P, Astola P, Rao B, Tabus I (2016) Sparse modelling and predictive coding of subaperture images for lossless plenoptic image compression. 2016 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON) pp. 1–4Google Scholar
  10. 10.
    HEVC SCC Reference Software Ver. 3.0 (SCM-3.0). [Online]. Available:
  11. 11.
    Kalantari NK, Wang TC, Ramamoorthi R (2016) Learning-Based View Synthesis for Light Field Cameras. ACM Trans Graph 35(6):193CrossRefGoogle Scholar
  12. 12.
    Lei J, Li D, Pan Z, Sun Z, Kwong S, Hou C (2017) Fast Intra Prediction Based on Content Property Analysis for Low Complexity HEVC-Based Screen Content Coding. IEEE Trans Broadcast 63(1):48–58CrossRefGoogle Scholar
  13. 13.
    M. Levoy, “Light fields and computational imaging, Computer, vol. 39, pp. 46–55, (2006)CrossRefGoogle Scholar
  14. 14.
    Levoy M, Hanrahan P (1996) Light field rendering. In Proc. 23rd Annu. Conf Comput Graph Interact Techn pp. 31–42Google Scholar
  15. 15.
    Li Y, Sjostrom M, Olsson R, Jennehag U (2016) Coding of Focused Plenoptic Contents by Displacement Intra Prediction. IEEE Transactions on Circuits and Systems for Video Technology 26(7):1308–1319CrossRefGoogle Scholar
  16. 16.
    Li L, Li Z, Li B, Liu D, Li H (2017) Pseudo Sequence Based 2-D Hierarchical Coding Structure for Light-Field Image Compression. 2017 Data Compression Conference (DCC), pp. 131–140Google Scholar
  17. 17.
    Liu D, An P, Ma R, Shen L (2015) Disparity compensa-tion based 3D holoscopic image coding using HEVC. In 2015 IEEE China Summit & Int. Conf. Signal and Information Processing (ChinaSIP), pp. 201–205Google Scholar
  18. 18.
    Liu Y, Nie L, Zhang L, Rosenblum DS (2015) Action2activity: Recognizing complex activities from sensor data. In IJCAI'15 Proceedings of the 24th International Conference on Artificial Intelligence, pp. 1617–1623Google Scholar
  19. 19.
    Liu D, Wang L, Li L, Xiong Z, Wu F, Zeng W (2016) Pseudo-sequence-based light field image compression. 2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pp. 1–4Google Scholar
  20. 20.
    Liu D, An P, Ma R, Yang C, Shen L, Li K (2016) Three-dimensional holoscopic image coding scheme using high-efficiency video coding with kernel-based minimum mean-square-error estimation. J Electron Imaging 25(4):043015-1–043015-9CrossRefGoogle Scholar
  21. 21.
    Liu D, An P, Ma R, Yang C, Shen L (2016) 3D holoscopic image coding scheme using HEVC with Gaussian process regression. Signal Process Image Commun 47:438–451CrossRefGoogle Scholar
  22. 22.
    Liu Y, Zhang L, Nie L, Yan Y, Rosenblum DS (2016) Fortune teller: Predicting your career path. In Proceedings of the Thirtieth AAAI conference on artificial intelligence, pp. 201–207Google Scholar
  23. 23.
    Liu Y, Zheng Y, Liang Y, Liu S, Rosenblum DS (2016) Urban Water Quality Prediction based on Multi-task Multi-view Learning. In Proceedings of the 25th International Joint Conference on Artificial Intelligence, pp. 1–7Google Scholar
  24. 24.
    Liu Y, Nie L, Liu L, Rosenblum DS (2016) From action to activity: Sensor-based activity recognition. Neurocomputing 181:108–115CrossRefGoogle Scholar
  25. 25.
    Liu F, Hou G, Sun Z, Tan T (2017) High quality depth map estimation of object surface from light-field images. Neurocomputing 252:3–16CrossRefGoogle Scholar
  26. 26.
    Liu D, An P, Yang C, Ma R, Shen L (2017) Coding of 3D holoscopic image by using spatial correlation of rendered view images. In 42th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2017), pp. 2002–2006Google Scholar
  27. 27.
    Lucas LFR, Conti C, Nunes P, Soares LD, Rodrigues NMM, Pagliari CL, da Silva EAB, de Faria SMM (2014) Locally linear embedding-based predic-tion for 3D holoscopic image coding using HEVC. In 2014 Proceedings of the 22nd European Signal Processing Conference (EUSIPCO), pp. 11, 15, 1–5Google Scholar
  28. 28.
    Monteiro R et al (2016) Light field HEVC-based image coding using locally linear embedding and self-similarity compensated prediction. 2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pp. 1–4Google Scholar
  29. 29.
    Monteiro RJS, Nunes PJL, Rodrigues NMM et al (2017) Light Field Image Coding using High Order Intra Block Prediction. IEEE Journal on Selected Topics in Signal Processing 11(7):1120–1131CrossRefGoogle Scholar
  30. 30.
    Podder PK, Paul M, Murshed M (2016) A novel motion classification based inter mode selection strategy for HEVC performance improvement. Neurocomputing 173:1211–1220CrossRefGoogle Scholar
  31. 31.
    Rerabek M, Bruy lants T, Ebrahimi T, Pereira F, Schelkens P (2016) Call for Proposals and Evaluation Procedure. ICME 2016 Grand Challenge: Light Field Image Compression, Seattle, USA, pp. 1–8Google Scholar
  32. 32.
    L. Shen et al. “Adaptive inter-mode decision for HEVC jointly utilizing inter-level and spatio-temporal correlations, ” IEEE Trans Syst Video Technol Vol. 24, no. 10, pp. 1709–1722, (2014)Google Scholar
  33. 33.
    Tan TK, Boon CS, Suzuki Y (2006) Intra prediction by template matching. In IEEE Int Conf Image Processing (ICIP), IEEE, pp, 1693–1696Google Scholar
  34. 34.
    Tehrani MP, Shimizu S, Lafruit G, Senoh T, Fujii T, Vetro A, et al. (2013) Use Cases and Requirements on Free-viewpoint Television (FTV), ISO/IEC JTC1/SC29/WG11 MPEG N14104, Geneva, Switzer-landGoogle Scholar
  35. 35.
    Turkan M, Guillemot C (2012) Image prediction based on neighbor-embedding methods. IEEE Trans Image Process 21(4):1885–1898MathSciNetCrossRefGoogle Scholar
  36. 36.
    Wang G, Xiang W, Pickering M, Chen CW (2016) Light Field Multi-View Video Coding With Two-Directional Parallel Inter-View Prediction. IEEE Trans Image Process 25(11):5104–5117MathSciNetCrossRefGoogle Scholar
  37. 37.
    Xu J, Joshi R, Cohen RA (2016) Overview of the Emerging HEVC Screen Content Coding Extension. IEEE Transactions on Circuits and Systems for Video Technology 26(1):50–62CrossRefGoogle Scholar
  38. 38.
    Yang R, Huang X, Li S, Jaynes C (2008) Toward the light field display: Autostereoscopic rendering via a cluster of projectors. IEEE Trans Vis Comput Graphics 14(1):84–96CrossRefGoogle Scholar
  39. 39.
    Yu H, Cohen R, Rapaka K, Xu J (2016) Common test conditions for screen content coding, document JCTVC-X1015Google Scholar
  40. 40.
    Zhang Q et al (2016) An efficient depth map filtering based on spatial and texture features for 3D video coding. Neurocomputing 188:82–89CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Computer and InformationAnqing Normal UniversityAnqingChina
  2. 2.The University Key Laboratory of Intelligent Perception and Computing of Anhui ProvinceAnqingChina
  3. 3.School of Communication and Information EngineeringShanghai UniversityShanghaiChina

Personalised recommendations