Multimedia Tools and Applications

, Volume 78, Issue 23, pp 32735–32754 | Cite as

Predicting split decisions of coding units in HEVC video compression using machine learning techniques

  • Mahitab Hassan
  • Tamer ShanablehEmail author


In this work, we propose to reduce the complexity of HEVC video encoding by predicting the split decisions of coding units. We use a sequence-dependent approach in which a number of frames belonging to the video being encoded are used for generating a classification model. At each coding depth of the coding units, features representing the coding unit at that particular depth are extracted from both the present and previously encoded coding units. The feature vectors are then used for generating a dimensionality reduction model and a classification model. The generated models at each coding depth are then used to predict the split decisions of subsequent coding units. Stepwise regression, random forest reduction and principal component analysis are used for dimensionality reduction; whereas, polynomial networks and random forests are utilized for classification. The proposed solution is assessed in terms of classification accuracy, BD-rate, BD-PSNR and computational time complexity. Using seventeen video sequences with four different classes of resolution, an average classification accuracy of 86.5% is reported for the proposed classification system. In comparison to regular HEVC coding, the proposed solution resulted in a BD-rate loss of 0.55 and a BD-PSNR of −0.02 dB. The average reported computational complexity reduction is found to be 39.2%.


HEVC Pattern recognition Video compression 



  1. 1.
    Ahn S, Lee B, Kim M (2015) A Novel Fast CU Encoding Scheme Based on Spatiotemporal Encoding Parameters for HEVC Inter Coding. IEEE Transactions on Circuits and Systems for Video Technology 25(3):422–435CrossRefGoogle Scholar
  2. 2.
    Bjøntegaard G (2008) Improvements of the BD-PSNR model. document VCEG-AI11, ITU-T SG16/Q6Google Scholar
  3. 3.
    Breiman L (2001) Random Forests. Mach Learn 45(1):5–32CrossRefGoogle Scholar
  4. 4.
    Cho S, Kim M (2013) Fast CU Splitting and Pruning for Suboptimal CU Partitioning in HEVC Intra Coding. IEEE Transactions on Circuits and Systems for Video Technology 23(9):1555–1564CrossRefGoogle Scholar
  5. 5.
    Chung C-H, Peng W-H, Hu J-H (2017) HEVC/H.265 Coding Unit Split Decision Using Deep Reinforcement Learning. International Symposium on Intelligent Signal Processing and Communication Systems, XiamenCrossRefGoogle Scholar
  6. 6.
    Correa G, Assuncao PA, Agostini LV, da Silva Cruz LA (2015) Fast HEVC Encoding Decisions Using Data Mining. IEEE Transactions on Circuits and Systems for Video Technology 25(4):660–673CrossRefGoogle Scholar
  7. 7.
    Deng X, Xu M, Jiang L, Sun X, Wang Z (2016) Subjective-Driven Complexity Control Approach for HEVC. IEEE Transactions on Circuits and Systems for Video Technology 26(1):91–106CrossRefGoogle Scholar
  8. 8.
    Du B, Siu W-C, Yang X (2015) Fast CU Partition Strategy for HEVC Intra-Frame Coding Using Learning Approach via Random Forests. Proceedings of APSIPA Annual Summit and Conference, Hong KongCrossRefGoogle Scholar
  9. 9.
    Goswami K, Lee J, Kim B (2016) Fast algorithm for the High Efficiency Video Coding (HEVC) encoder using texture analysis. Inf Sci 364-365:72–90CrossRefGoogle Scholar
  10. 10.
    Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009) The WEKA data mining software: An update. ACM SIGKDD Explorations Newslett 11(1):10–18CrossRefGoogle Scholar
  11. 11.
    Hu Q, Zhang X, Shi Z, Gao Z (2016) Neyman-Pearson-Based Early Mode Decision for HEVC Encoding. IEEE Transactions on Multimedia 18(3):379–391CrossRefGoogle Scholar
  12. 12.
    Jiménez-Moreno A, Martínez-Enríquez E, Díaz-de-María F (2016) Complexity Control Based on a Fast Coding Unit Decision Method in the HEVC Video Coding Standard. IEEE Transactions on Multimedia 18(4):563–575CrossRefGoogle Scholar
  13. 13.
    Kim I-K, McCann KD, Sugimoto K, Bross B, Han W-J, Sullivan GJ (2013) High Efficiency Video Coding (HEVC) Test Model 13 (HM13) Encoder Description. Document: JCTVC-O1002, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 15th Meeting, GenevaGoogle Scholar
  14. 14.
    Kim H, Park R (2016) Fast CU Partitioning Algorithm for HEVC Using an Online-Learning-Based Bayesian Decision Rule. IEEE Transactions on Circuits and Systems for Video Technology 26(1):130–138CrossRefGoogle Scholar
  15. 15.
    Lee J, Kim S, Lim K, Lee S (2015) A Fast CU Size Decision Algorithm for HEVC. IEEE Transactions on Circuits and Systems for Video Technology 25(3):411–421CrossRefGoogle Scholar
  16. 16.
    Lim K, Lee J, Kim S, Lee S (2015) Fast PU Skip and Split Termination Algorithm for HEVC Intra Prediction. IEEE Transactions on Circuits and Systems for Video Technology 25(8):1335–1346CrossRefGoogle Scholar
  17. 17.
    Liu Z, Lin T, Chou C (2016) Efficient prediction of CU depth and PU mode for fast HEVC encoding using statistical analysis. J Vis Commun Image Represent 38:474–486CrossRefGoogle Scholar
  18. 18.
    Livingston F (2005) Implementation of Breiman’s random forest machine learning algorithm. ECE591Q Machine Learning Journal Paper 1–13Google Scholar
  19. 19.
    Mallikarachchi T, Talagala D, Arachchi H, Fernando A (2018) Content-adaptive feature-based CU size prediction for fast low-delay video encoding in HEVC. IEEE Transactions on Circuits and Systems for Video Technology 28(3)CrossRefGoogle Scholar
  20. 20.
    Genuer R, Poggi J-M, Tuleau-Malot C (2010) Variable selection using Random Forests. Pattern Recogn Lett 31(14):2225–2236CrossRefGoogle Scholar
  21. 21.
    Ohm JR, Sullivan GJ, Schwarz H, Tan TK, Wiegand T (2012) Comparison of the Coding Efficiency of Video Coding Standards—Including High Efficiency Video Coding (HEVC). IEEE Transactions on Circuits and Systems for Video Technology 22(12):1669–1684CrossRefGoogle Scholar
  22. 22.
    Park S (2016) CU encoding depth prediction, early CU splitting termination and fast mode decision for fast HEVC intra-coding. Signal Process Image Commun 42:79–89CrossRefGoogle Scholar
  23. 23.
    Peixoto E, Shanableh T, Izquierdo E (2014) H.264/AVC to HEVC Video Transcoder based on Dynamic Thresholding and Content Modeling. IEEE Transactions on Circuits and Systems for Video Technology 24(1)CrossRefGoogle Scholar
  24. 24.
    Radosavljević M, Georgakarakos G, Lafond S, Vukobratović D (2015) Fast coding unit selection based on local texture characteristics for HEVC intra frame. 2015 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Orlando, pp 1377–1381Google Scholar
  25. 25.
    Shanableh T (2011) Prediction of Structural Similarity Index of Compressed Video at a Macroblock Level. IEEE Signal Processing Letters 18(5):335–338CrossRefGoogle Scholar
  26. 26.
    Shanableh T, Assaleh K (2010) Feature modeling using polynomial classifiers and stepwise regression. Neurocomputing, Elsevier 73:10–12CrossRefGoogle Scholar
  27. 27.
    Shanableh T, Peixoto E, Izquierdo E (2013) MPEG-2 to HEVC video transcoding with content-based modeling. IEEE Transactions on Circuits and Systems for Video Technology 23(7)CrossRefGoogle Scholar
  28. 28.
    Shen L, Zhang Z, Liu Z (2014) Effective CU Size Decision for HEVC Intracoding. IEEE Trans Image Process 23(10):4232–4241MathSciNetCrossRefGoogle Scholar
  29. 29.
    Shen L, Zhang Z, Zhang X, An P, Liu Z (2015) Fast TU size decision algorithm for HEVC encoders using Bayesian theorem detection. Signal Process Image Commun 32:121–128CrossRefGoogle Scholar
  30. 30.
    Sullivan G, Ohm J, Han W, Wiegand T (2012) Overview of the High Efficiency Video Coding (HEVC) Standard. IEEE Transactions on Circuits and Systems for Video Technology 22(12):1649–1668CrossRefGoogle Scholar
  31. 31.
    Sullivan GJ, Wiegand T (1998) Rate-distortion optimization for video compression. IEEE Signal Process Mag 15(6):74–90CrossRefGoogle Scholar
  32. 32.
    Tai K-H, Hsieh M-Y, Chen M-J, Chen C-Y, Yeh C-H (2017) A fast HEVC encoding method using depth information of collocated CUs and RD cost characteristics of PU modes. IEEE Transactions on Broadcasting 63(4)CrossRefGoogle Scholar
  33. 33.
    Toh K, Tran Q, Srinivasan D (2004) Benchmarking a reduced multivariate polynomial pattern classifier. IEEE Trans Pattern Anal Mach Intell 26(6):740–755CrossRefGoogle Scholar
  34. 34.
    Vanne J, Viitanen M, Hamalainen TD, Hallapuro A (2012) Comparative rate-distortion-complexity analysis of HEVC and AVC video codecs. IEEE Trans Circuits Syst Video Technol 22(12):1885–1898CrossRefGoogle Scholar
  35. 35.
    Xiong J, Li H, Meng F, Wu Q, Ngan KN (2015) Fast HEVC Inter CU Decision Based on Latent SAD Estimation. IEEE Transactions on Multimedia 17(12):2147–2159CrossRefGoogle Scholar
  36. 36.
    Xiong J, Li H, Wu Q, Meng F (2014) A Fast HEVC Inter CU Selection Method Based on Pyramid Motion Divergence. IEEE Transactions on Multimedia 16(2):559–564CrossRefGoogle Scholar
  37. 37.
    Yang S-H, Zhong C-C (2017) Fast Coding-Unit Mode Decision for HEVC Transrating. IEEE International Conference on Computer and Information Technology, FinlandCrossRefGoogle Scholar
  38. 38.
    Yoo H, Suh J (2014) Fast coding unit decision based on skipping of inter and intra prediction units. Electron Lett 50(10):750–752CrossRefGoogle Scholar
  39. 39.
    Zhang Y, Kwong S, Wang X, Yuan H, Pan Z, Xu L (2015) Machine Learning-Based Coding Unit Depth Decisions for Flexible Complexity Allocation in High Efficiency Video Coding. IEEE Trans Image Process 24(7):2225–2238MathSciNetCrossRefGoogle Scholar
  40. 40.
    Zhao W, Onoye T, Song T (2015) Hierarchical Structure-Based Fast Mode Decision for H.265/HEVC. IEEE Transactions on Circuits and Systems for Video Technology 25(10):1651–1664CrossRefGoogle Scholar
  41. 41.
    Zupancic I, Blasi SG, Peixoto E, Izquierdo E (2016) Inter-Prediction Optimizations for Video Coding Using Adaptive Coding Unit Visiting Order. IEEE Transactions on Multimedia 18(9):1677–1690CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringAmerican University of SharjahSharjahUnited Arab Emirates
  2. 2.IBM CloudDubaiUnited Arab Emirates

Personalised recommendations