Adaptive CU partition and early skip mode detection for H.266/VVC

Multimedia Tools and Applications (2021)Cite this article

Abstract

The Joint Video Exploration Team (JVET) has started to develop the next-generation video coding standard-H.266/Versatile Video Coding (H.266/VVC) based on H.265/High Efficiency Video Coding (H.265/HEVC) to provide higher compression performance. The H.266/VVC supports the flexible quadtree with a nested multi-type tree (QTMT) partition structure including quadtree (QT), binary tree (BT), and ternary tree (TT). The coding unit (CU) sizes range from 128 to 4 for the luma component or from 64 to 2 for the chroma component in the QTMT splitting structure. The introduction of small CU size, i.e., 2×N, leads to inefficient hardware implementation because it causes pipeline delayed and needs to process 2×N pixels in the hardware architecture. In addition, the inter or bi-predicted of small CU requires a higher memory bandwidth than the bi-predicted of 8×8 CU in H.266/VVC. To solve the above issues, we introduce a fast method to accelerate CU partition and mode decision, including an adaptive CU partition method and early skip mode detection method. The proposed algorithm consists of two parts: (1) adaptive remove 2×N CUs by skipping BT and TT splitting mode; (2) early skip bi-predicted or inter prediction of small CU. The experimental results demonstrate that the proposed scheme can save 47% coding time while maintaining the coding performance.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

US$ 39.95

Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  1. 1.

    Bjontegaard G (2001) Calculation of Average PSNR differences between RD curves. Document ITU-T SG16 Q.6, VCEG Meeting, VCEG-M33, Austin, TX, USA

  2. 2.

    Bjøntegaard G (2008) Improvements of the BD-PSNR model. VCEG-AI11, input document of the 35th VCEG meeting

  3. 3.

    Bossen F, Boyce J, Li X, Seregin V and Sühring K (2019) “JVET common test conditions and software reference configurations for SDR video,” JVET-N1010, Geneva, CH

  4. 4.

    Bross B, Chen J, Liu S (2018) Versatile Video Coding (Draft 3),” in the 12th meeting of Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, document no. JVET-L1001, Macao, China,

  5. 5.

    Bross B, Chen J, Liu S (2019) Versatile video coding (Draft 4). Doc. JVET-M1001

  6. 6.

    Chen C, Chuang T (2018) Cel-related: Separate tree partitioning at 64x64-luma/32x32-chroma unit level. 11th JVET Meeting: Ljubljana, SI

  7. 7.

    Chen J, Chien WJ, Karczewicz M, Li X, Liu H, Said A, Zhang L, Zhao X (2015) Further improvements to HMKTA-1.0. ITU-T SG16/Q6, Doc. VCEG-AZ07, 52nd Meeting

  8. 8.

    Chen J, Alshina E, Sullivan GJ, Ohm JR, Boyce J (2016) “Algorithm description of Joint Exploration Test Model 4 (JEM4),” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Doc. JVET-C1001, 3rd Meeting, Geneva, May

  9. 9.

    Chen C-C, Xiu X, He Y, Ye Y (2016) EE3: Generalized bi-prediction. JVET-D0102, Input Docum 4th JVET Meet

  10. 10.

    Fan Y, Chen J, Sun H, Katto J, Jing M (2020) A Fast QTMT Partition Decision Strategy for VVC Intra Prediction. In: IEEE Access, vol 8, pp 107900–107911

    Google Scholar 

  11. 11.

    Feng Z, Liu P, Jia K, Duan K (2018) HEVC Fast Intra Coding Based CTU Depth Range Prediction. In: Proceedings of 2018 IEEE 3rd International Conference on Image, Vision and Computing (ICIVC), Chongqing, pp 551–555

    Google Scholar 

  12. 12.

    Fu T, Zhang H, Mu F, Chen H (2019) Fast CU Partitioning Algorithm for H.266/VVC Intra-Frame Coding. In: Proceedings of 2019 IEEE international conference on multimedia and Expo (ICME), Shanghai, China, vol 60, p 55

    Google Scholar 

  13. 13.

    Fu T, Zhang H, Mu F, Chen H (2019) Two-Stage Fast Multiple Transform Selection Algorithm for VVC Intra Coding. In: Proceedings of 2019 IEEE international conference on multimedia and expo (ICME), pp 61–66

    Google Scholar 

  14. 14.

    He Y (2019) BoG report on CE2 related contributions. Doc. JVET-M0862

  15. 15.

    Huang X, Zhang Q, Zhao X, Zhang W, Zhang Y, Gan Y (2017) Fast inter-prediction mode decision algorithm for HEVC. SIViP 11(1):33–40

    Article  Google Scholar 

  16. 16.

    Huang Y, Hsu C, Chen C, Chuang T (2020) A VVC Proposal with Quaternary Tree plus Binary-Ternary Tree Coding Block Structure and Advanced Coding Techniques In: IEEE Trans. Circuits Syst. Video Technol., vol. 30, no. 5, pp. 1311–1325

  17. 17.

    Li X, Bossen F, Boyce J , Shring K and Seregin V (2018) “JVET common test conditions and software reference configurations for SDR video,” document JVET-K1010, 11th JVET meeting: Ljubljana, SI

  18. 18.

    Lim K, Lee J, Kim S, Lee S (2015) Fast PU Skip and Split Termination Algorithm for HEVC Intra Prediction. IEEE Trans. Circuits Syst Video Technol 25(8):1335–1346

    Article  Google Scholar 

  19. 19.

    Lin T, Jiang H, Huang J, Chang P (2018) Fast Binary Tree Partition Decision in H.266/FVC Intra Coding,” in Proceedings of 2018 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW), Taichung, pp. 1–2

  20. 20.

    Lin T, Jiang H, Huang J (2020) Fast intra coding unit partition decision in H.266/FVC based on spatial features. J Real-Time Image Proc 17(3):493–510

    Article  Google Scholar 

  21. 21.

    Liu S, Brass B, Chen J (2018) “Versatile video coding (draft2),” 11th JVET Meeting: Ljubljana, SI 22:1649–1668

    Google Scholar 

  22. 22.

    Min B, Cheung, RCC 2015 “A Fast CU Size Decision Algorithm for the HEVC Intra Encoder,” In: IEEE Trans. Circuits Syst. Video Technol., vol. 25, no. 5, pp. 892–896.

  23. 23.

    Park S, Kang J (2019) Context-based ternary tree decision method in versatile video coding for fast intra coding. IEEE Access 7:172597–172605

    Article  Google Scholar 

  24. 24.

    Ramezanpour M, Zargari F (2015) Early termination algorithm for CU size decision in HEVC intra coding. In: Proceedings of 2015 9th Iranian conference on machine vision and image processing (MVIP), Tehran, pp 45–48

  25. 25.

    Shen L, Zhang Z, An P (2013) Fast CU size decision and mode decision algorithm for HEVC intra coding. IEEE Trans Consumer Electron 59(1):207–213

    Article  Google Scholar 

  26. 26.

    Sullivan GJ, Ohm J, Han W, Wiegand T (2012) Overview of the high efficiency video coding (HEVC) standard. IEEE Trans Circuits Syst Video Technol 22(12):1649–1668

    Article  Google Scholar 

  27. 27.

    “The vvc test model 1”, https://jvet.hhi.fraunhofer.de/svn/svn_VVCSoftware_ VTM/tags/VTM-1.0, [Online].

  28. 28.

    VTM-6.0, https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/tags/VTM-6.0.

  29. 29.

    Yang H, Shen L, Dong X, Ding Q, An P , Jiang G (2020) “Low complexity CTU partition structure decision and fast intra mode decision for versatile video coding,” in IEEE Trans. Circuits Syst. Video Technol., vol. 30, no. 6, pp. 1668-1682

  30. 30.

    Zhang Q, Li N, Huang L, Gan Y (2014) Effective early termination algorithm for depth map intra coding in 3D-HEVC. Electron Lett 50(14):994–996

    Article  Google Scholar 

  31. 31.

    Zhang Q, Wu Q, Wang X, Gan Y (2014) Early SKIP mode decision for three-dimensional high efficiency video coding using spatial and interview correlations. J Electron Imag 23(5):053017–1–053017-8

    Google Scholar 

  32. 32.

    Zhang Q, Chen M, Huang X, Li N, Gan Y (2015) Low-complexity depth map compression in HEVC-based 3D video coding. Eurasip J Image Video Process 2015(1):1–14

    Article  Google Scholar 

  33. 33.

    Zhang Q, Wang X, Huang X, Su R, Gan Y (2015) Fast mode decision algorithm for 3D-HEVC encoding optimization based on depth information. Digital Signal Process 44(9):37–46

    Article  Google Scholar 

  34. 34.

    Zhang Q, Wang Y, Huang L, Jiang B (2020) Fast CU Partition and Intra Mode Decision Method for H.266/VVC. In: IEEE Access, vol 8, pp 117539–117550

    Google Scholar 

  35. 35.

    Zhang Q, Wang Y, Huang L, Jiang B, Wang X (2020) Fast CU partition decision for H.266/VVC based on the improved DAG-SVM classifier model. Multimedia Systems

Download references

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China No. 61771432, 61302118, 61773018, 61702464, and 61803346, the Basic Research Projects of Education Department of Henan No. 21zx003, and 20A880004, and the Key Research and Development Program of Henan No. 202102210179.

Author information

Affiliations

  1. College of Computer and Communication Engineering, Zhengzhou University of Light Industry, No. 5 Dongfeng Road, Zhengzhou, 450002, China

    Qiuwen Zhang, Yihan Wang, Bin Jiang, Xiao Wang & Rijian Su

Authors
  1. Qiuwen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yihan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bin Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rijian Su
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qiuwen Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhang, Q., Wang, Y., Jiang, B. et al. Adaptive CU partition and early skip mode detection for H.266/VVC. Multimed Tools Appl (2021). https://doi.org/10.1007/s11042-020-10252-6

Download citation

Keywords

  • H.266/VVC
  • Adaptive CU decision
  • Early skip
  • Intra prediction