Advertisement

HEVC optimization based on human perception for real-time environments

  • D. G. Fernández
  • Guillermo Botella
  • Alberto A. Del BarrioEmail author
  • Carlos García
  • Manuel Prieto-Matías
  • Christos Grecos
Article

Abstract

High-Efficiency Video Coding (HEVC) is the new emerging video coding standard of the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG). The HEVC standard provides a significant improvement in compression efficiency in comparison with existing standards such as H264/AVC by means of greater complexity. In this paper we will examine several HEVC optimizations based on image analysis to reduce its huge CPU, resource and memory expensive encoding process. The proposed algorithms optimize the HEVC quad-tree partitioning procedure, intra/inter prediction and mode decision by means of H264-based methods and spatial and temporal homogeneity analysis which is directly applied to the original video. The validation process of these approaches was conducted by taking into account the human visual system (HVS). The adopted solution makes it possible to perform HEVC real time encoding for HD sequences on a low cost processor with negligible quality loss. Moreover, the frames pre-processing leverages the logic units and embedded hardware available on an Intel GPU, so the execution time of these stages are negligible for the encoding processor.

Keywords

HEVC CU size decision Spatial homogeneity Temporal homogeneity HVS metrics GPU Mode decision Intra prediction Inter prediction Texture analysis 

Notes

Acknowledgments

This work has been partially supported by Spanish research projects TIN 2015-65277-R and TIN-2012-32180, as well as the UCM-Banco Santander Grant PR26-16/20B-1.

References

  1. 1.
    Ahn S, Lee B, Kim M (2015) A novel fast CU encoding scheme based on spatio-temporal encoding parameters for HEVC inter coding. IEEE Trans Circuits Syst Video Technol 25(3):422–435CrossRefGoogle Scholar
  2. 2.
    Alcocer E, Gutierrez R, Lopez-Granado O, Malumbres MP (2016) Design and implementation of an efficient hardware integer motion estimator for an HEVC video encoder. Journal of Real-Time Image ProcessingGoogle Scholar
  3. 3.
    Bjontegarrd G (2001) Calculation of average PSNR differences between RD curves. ITU-T SC16/Q6 13th VCEG meeting, AustinGoogle Scholar
  4. 4.
    Bossen F (2012) Common test conditions and software reference configurations. JCT-VC Document, JCTVC-K1100Google Scholar
  5. 5.
    Cho S, Kim M (2013) Fast CU splitting and pruning for suboptimal CU partitioning in HEVC intra coding. IEEE Trans Circuits Syst Video Technol 23(9):1555–1564CrossRefGoogle Scholar
  6. 6.
    de Frutos-López M, Orellana-Quirós D, Pujol-Alcolado JC, de María FD (2010) An improved fast mode decision algorithm for intraprediction in H.264/AVC video coding. Signal Process Image Commun 25(10):709–716CrossRefGoogle Scholar
  7. 7.
    Fernandez DG, Del Barrio AA, Botella G, Garcia C (2016) 4K-based intra and inter prediction techniques for HEVC. Proc. SPIE, Real-Time Image and Video Processing 2016, 98970BGoogle Scholar
  8. 8.
    Fernández DG, Del Barrio AA, Botella G, García C (2016) Fast CU size decision based on temporal homogeneity detection. In: Design of Circuits and Integrated Systems (DCIS), 2016 Conference on, 1–6Google Scholar
  9. 9.
    Fernández DG, Del Barrio AA, Botella G, García C (2018) Fast and effective CU size decision based on spatial and temporal homogeneity detection. Multimedia Tools and Applications 77(5):5907–5927CrossRefGoogle Scholar
  10. 10.
    Fernández DG, Del Barrio AA, Botella G, García C, Meyer-Baese U, Meyer-Baese A (2016) HEVC optimizations for medical environments. In: Proc. SPIE 9871, Sensing and Analysis Technologies for Biomedical and Cognitive Applications 2016, 98710BGoogle Scholar
  11. 11.
    Fernández DG, Del Barrio AA, Botella G, García C, Prieto M, Hermida R (2018) Complexity reduction in the HEVC/H265 standard based on smooth region classification. Digital Signal Processing 73:24–39CrossRefGoogle Scholar
  12. 12.
    D. G. Fernández, A. A. Del Barrio, Guillermo Botella, Uwe Meyer-Baese, Anke Meyer-Baese, Christos Grecos (2017) Information fusion based techniques for HEVC. Proc. SPIE 10223, Real-Time Image and Video Processing 2017, 102230M. 10.1117/12.2262604Google Scholar
  13. 13.
    Fraunhofer Institute for Telecommunications (2018) Perceptually optimized video coding. Retrieved from: https://www.hhi.fraunhofer.de/en/departments/vca/research-groups/image-video-coding/research-topics/perceptually-optimized-video-coding.html. Accessed 17 Dec 2018
  14. 14.
    Goswami K, Lee J-H, Jang K-S, Kim B-G, Kwon K-K (2014) Entropy difference-based early skip detection technique for high efficiency video coding. Journal of Real-Time Image ProcessingGoogle Scholar
  15. 15.
    He G, Zhou D, Goto S (2013) Transform-based fast mode and depth decision algorithm for HEVC intra prediction. IEEE 10th International Conference on ASIC (ASICON), pp. 1–4Google Scholar
  16. 16.
    Intel Corporation (2016) Introduction to advance motion extension for OpenCL. Retrieved from: https://software.intel.com/en-us/articles/intro-to-advanced-motion-estimation-extension-for-opencl. Accessed 17 Dec 2018
  17. 17.
    Intel Corporation White Paper (2012) Performance Interactions of OpenCL* Code and Intel® Quick Sync Video on Intel® HD Graphics 4000Google Scholar
  18. 18.
    ITU-R BT.500 (2012) Methodology for the subjective assessment of the quality of television pictures. International Telecommunication Union, GenevaGoogle Scholar
  19. 19.
    Jiang W, Ma H, Chen Y (2012) Gradient based fast mode decision algorithm for intra prediction in HEVC. In: 2nd International Conference on Consumer Electronics, Communications and Networks (CECNet), pp. 1836–1840Google Scholar
  20. 20.
    Jiang W, Ma H, Chen Y (2012) Gradient Based Fast Mode Decision Algorithm for Intra Prediction in HEVC. In: International Conference on Consumer Electronics, Communications and Networks (CECNet)Google Scholar
  21. 21.
    Khan MUK, Shafique M, Grellert M, Henkel J (2013) Hardware-software collaborative complexity reduction scheme for the emerging HEVC intra encoder. Design, Automation Test in Europe Conference Exhibition (DATE) 2013:125–128CrossRefGoogle Scholar
  22. 22.
    Khronos OpenCL Working Group (2011) The OpenCL specification version 1.1. Revision 44Google Scholar
  23. 23.
    Khronos OpenCL Working Group (2016) Online documentation for cl_intel_advaanced_motion_estimation. Retrieved from: https://www.khronos.org/registry/cl/extensions/intel/cl_intel_advanced_motion_estimation.txt. Accessed 17 Dec 2018
  24. 24.
    Koumaras H, Kourtis M, Martakos D (2012) Benchmarking the encoding efficiency of h.265/HEVC and h.264/AVC,” in Future Network Mobile Summit (FutureNetw)Google Scholar
  25. 25.
    Leal da Silva T, Agostini LV, da Silva Cruz LA (2015) Fast intra prediction algorithm based on texture analysis for 3D-HEVC encoders” in Journal of Real-Time Image ProcessingGoogle Scholar
  26. 26.
    Lee JH, Park CS, Kim BG, Jun DS, Jung SH, Choi JS (2013) Novel fast PU decision algorithm for the HEVC video standard”, IEEE International Conference on Image Processing (ICIP)Google Scholar
  27. 27.
    Li C, Bovik AC (2009) Three-component weighted structural similarity index. Proc. SPIE 7242–72420, Image Quality and System Performance VIGoogle Scholar
  28. 28.
    Lim K, Lee J, Kim S, Lee S (2015) Fast PU skip and split termination algorithm for HEVC intra. IEEE Transactions on Circuits Systems for Video Technology 25(8)Google Scholar
  29. 29.
    Liu X, Liu Y, Wang P, Lai C, Chao H (2016) An adaptive mode Decision algorithm based on video texture characteristics for HEVC intra prediction. IEEE Transactions on Circuits Systems for Video Technology, vol. PP: 99Google Scholar
  30. 30.
    Mallikarachchi T, Fernando A, Arachchi H (2014) Efficient coding unit size selection based on texture analysis for HEVC intra prediction. IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6Google Scholar
  31. 31.
    McCann K, Rosewarne C, Bross B, Naccari M, Sharman K, Sullivan G (2014) High efficiency video coding (HEVC) encoder description 0v16 (HM16). JCT-VC High Efficiency Video Coding N14 703Google Scholar
  32. 32.
    Min B, Cheung R (2014) A fast cu size decision algorithm for HEVC intra encoder. IEEE Transactions on Circuits and Systems for Video Technology PP(99):1Google Scholar
  33. 33.
    Moorthy AK, Bovik AC (2010) Efficient motion weighted spatio-temporal video SSIM Index. Proc. SPIE 7527–75271, Human Vision and Electronic Imaging XVGoogle Scholar
  34. 34.
    MSU Graphics & Media Lab (Video Group) (2016) MSU video quality measurement tool. Retrieved from: http://www.compression.ru/video/quality_measure/video_measurement_tool.html. Accessed 17 Dec 2018
  35. 35.
    Multimedia Signal Processing Group (MMSPG) (2016) VQMT: video quality measurement tool. Retrieved from: http://mmspg.epfl.ch/vqmt. Accessed 17 Dec 2018
  36. 36.
    Na S, Lee W, Yoo K (2014) Edge-based fast mode decision algorithm for intra prediction in HEVC. IEEE International Conference on Consumer Electronics (ICCE), pp. 11–14Google Scholar
  37. 37.
    Öztekin A, ErÇelebi E (2015) An early split and skip algorithm for fast intra CU selection in HEVC. In: Journal of Real-Time Image ProcessingGoogle Scholar
  38. 38.
    Pastuszak G, Trochimiuk M (2015) Algorithm and architecture design of the motion estimation for the H.265/HEVC 4K-UHD encoder. Journal of Real-Time Image ProcessingGoogle Scholar
  39. 39.
    Ponomarenko N, Silvestri F, Egiazarian K, Carli M, Astola J, Lukin V (2007) On Between-Coefficient Contrast Masking of DCT Basis Functions. Third International Workshop on Video Processing and Quality Metrics (VPQM), ScottsdaleGoogle Scholar
  40. 40.
    Ramezanpour M, Zargari F (2016) Fast HEVC I-frame coding based on strength of dominant direction of CUs. Journal of Real-Time Image ProcessingGoogle Scholar
  41. 41.
    Shang X, Wang G, Fan T, Li Y (2015) Fast CU size decision and PU mode decision algorithm in HEVC intra coding. IEEE International Conference on Image Processing (ICIP)Google Scholar
  42. 42.
    Sheikh HR, Bovik AC (2006) Image Information and Visual Quality. IEEE Trans Image Process 15(2):430–444CrossRefGoogle Scholar
  43. 43.
    Sheikh HR, Bovik AC, de Veciana G (2005) An Information Fidelity Criterion for Image Quality Assessment Using Natural Scene Statistics. IEEE Trans Image Process 14(12):2117–2128CrossRefGoogle Scholar
  44. 44.
    Shen L, Liu Z, Liu S, Zhang Z, An P (2009) Selective disparity estimation and variable size motion estimation based on motion homogeneity for multi-view coding. IEEE Transactions on Broadcasting 55(4)Google Scholar
  45. 45.
    Shen L, Liu Z, Yan T, Zhang Z, An P (2010) View-adaptive motion estimation and disparity estimation for low complexity multiview video coding. IEEE Transactions on Circuits and Systems for Video Technology 20(6)Google Scholar
  46. 46.
    Shen L, Liu Z, Zhang Z, Shi X (2008) Fast Inter Mode Decision Using Spatial Property of Motion Field. IEEE Transactions on Multimedia 10(6):1208–1214CrossRefGoogle Scholar
  47. 47.
    Shen L, Liu Z, Zhang X, Zhao W, Zhang Z (2013) An effective CU size decision method for HEVC encoders. IEEE Transactions on Multimedia 15(2):465–470CrossRefGoogle Scholar
  48. 48.
    Shen L, Zhang Z, An P (2013) Fast CU size decision and mode decision algorithm for HEVC intra coding. IEEE Transactions on Consumer Electronics 59(1)Google Scholar
  49. 49.
    Shen L, Zhang Z, Liu Z (2014) Effective CU size decision for HEVC intracoding. IEEE Trans Image Process 23(10):4232–4241MathSciNetCrossRefGoogle Scholar
  50. 50.
    Sullivan G, Ohm J, Han W-J, Wiegand T (2012) Overview of the high efficiency video coding (hevc) standard. IEEE Trans Circuits Syst Video Technol 22(12):1649–1668CrossRefGoogle Scholar
  51. 51.
    Sze V, Budagavi M, Sullivan GJ (2014) High Efficiency Video Coding (HEVC), 1st ed. Springer International Publishing. Available: https://link.springer.com/book/10.1007%2F978-3-319-06895-4. Accessed 17 Dec 2018
  52. 52.
    Tian G, Goto S (2012) Content adaptive prediction unit size decision algorithm for HEVC intra coding,” in Picture Coding Symposium (PCS), pp. 405–408Google Scholar
  53. 53.
    Ting Y-C, Chang T-S (2014) Gradient-based PU size selection for HEVC intra prediction. IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1929–1932Google Scholar
  54. 54.
    Vanne J, Viitanen M, Hämäläinen TD (2014) Efficient mode decision schemes for HEVC inter prediction. IEEE Transactions on Circuits and Systems for Video Technology 24(9):1579–1593CrossRefGoogle Scholar
  55. 55.
    Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Trans Image Process 13(4):600–612CrossRefGoogle Scholar
  56. 56.
    Wang H-M, Lin J-K, Yang J-F (2006) Fast inter mode decision based on hierarchical homogeneous detection and cost analysis for h.264/AVC coders. In: IEEE International Conference on Multimedia and Expo, pp. 709–712Google Scholar
  57. 57.
    Wang Z, Simoncelli EP, Bovik AC (2003) Multi-scale structural similarity for image quality assessment. In: Proceedings of the 37th IEEE Asiloma Conference on Signal, Systems and Computers, Pacific GroveGoogle Scholar
  58. 58.
    Watson AB (1998) Toward a perceptual video quality metric. Human Vision, Visual Processing, and Digital Display VIII(3299):139–147Google Scholar
  59. 59.
    Wu D, Pan F, Lim K, Wu S, Li Z, Lin X, Rahardja S, Ko C (2005) Fast intermode decision in h.264/AVC video coding. IEEE Transactions on Circuits and Systems for Video Technology 15(7):953–958CrossRefGoogle Scholar
  60. 60.
    x265 project (2017) http://x265.org/. GNU GPL 2 license, source code available at: https://bitbucket.org/multicoreware/x265/wiki/Home. Accessed 17 Dec 2018
  61. 61.
    Xiao F (2000) DCT-based Video Quality Evaluation---Final Project for EE392JGoogle Scholar
  62. 62.
    Xiong J, Li H, Meng F, Wu Q, Ngan KN (2015) Fast HEVC inter CU decision based on latent SAD estimation. IEEE Transactions on Multimedia 17(12):2147–2159CrossRefGoogle Scholar
  63. 63.
    Xiong J, Li H, Meng F, Zhu S, Wu Q, Zeng B (2014) MRF-based fast HEVC inter CU decision with the variance of absolute differences. IEEE Trans Multimedia 16(8):2141–2153CrossRefGoogle Scholar
  64. 64.
    Xiong J, Li H, Wu Q, Meng F (2014) A fast HEVC inter CU selection method based on pyramid motion divergence. IEEE Transactions on Multimedia 16(2):559–564CrossRefGoogle Scholar
  65. 65.
    xiph.org (2016) Derf’s test media collection. Retrieved from: https://media.xiph.org/video/derf/. Accessed 17 Dec 2018
  66. 66.
    Ye T, Zhang D, Dai F, Zhang Y (2013) Fast mode decision algorithm for intra prediction in HEVC. in Proceedings of the Fifth International Conference on Internet Multimedia Computing and Service, pp. 300–304Google Scholar
  67. 67.
    Ye T, Zhang D, Dai F, Zhang Y (2013) Fast mode decision algorithm for intra prediction in HEVC. in the Fifth International Conference on Internet Multimedia Computing and Service (ICIMCS'13)Google Scholar
  68. 68.
    Zhang Y, Li Z, Li B (2012) Gradient-based fast decision for intra prediction in HEVC. In Visual Communications and Image Processing (VCIP), pp. 1–6Google Scholar
  69. 69.
    Zhang H, Ma Z (2014) Fast Intra mode decision for high efficiency video coding (HEVC). IEEE Transactions on Circuits Systems for Video Technology 24(4):660–668CrossRefGoogle Scholar
  70. 70.
    Zhang H, Ma Z (2016) Fast intra mode and CU size decision for HEVC. IEEE Transactions on Circuits Systems for Video Technology PP(99):1–7Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Computer Architecture and AutomationComplutense University of MadridMadridSpain
  2. 2.Department of Computer ScienceCentral Washington UniversityWashingtonUSA

Personalised recommendations