Multimedia Tools and Applications

, Volume 72, Issue 2, pp 1803–1821 | Cite as

An efficient mode decision algorithm for H.264/AVC intra prediction

  • Yonghong Kuo
  • Jiefeng Yang
  • Jian Chen


Rate distortion optimization technique is adopted by H.264/AVC to select the best intra and inter prediction modes. It achieves remarkable improvement in compression performance, but the computational complexity of coding increases greatly. In order to reduce the computational complexity as much as possible while guaranteeing the video encoding quality and compression efficiency, this paper proposes a fast mode decision method based on the texture direction information of intra prediction modes and the encoding macroblocks. For intra luminance prediction, the proposed algorithm utilizes the smoothness of the encoding macroblock to select the suitable intra prediction block sizes, and then uses the texture direction difference to filter out low possibility prediction modes. The calculation expressions of texture direction difference can be derived by extracting texture direction features from intra prediction modes. For intra chrominance prediction, the candidate prediction modes are determined by a combination of texture direction difference and the sum of absolute transformed difference, which doesn’t significantly degrade peak-signal-noise-rate or increase bit rate. Based on the processing, the number of rate distortion cost calculations decreases dramatically, which indicates a significant reduction of computation cost for intra prediction. Compared with JM11.0 reference software, the proposed algorithm can cut down about 76.79 % total intra-frame coding time at the expense of only about 0.08 dB peak-signal-noise-rate degradation and 2.07 % bit rate increase. It proves that the proposed algorithm achieves a tradeoff between the rate distortion performance and the computational complexity.


H.264/AVC Intra prediction Texture direction difference Sum of absolute transform difference 


  1. 1.
    Chen CN, Chen JZ, Ouyang K, Xia T, Zhou JL (2013) A hybrid fast mode decision method for H.264/AVC intra prediction. Multimed Tools Appl 62(3):719–731CrossRefGoogle Scholar
  2. 2.
    Huang YH, Ou TS, Chen HH (2010) Fast decision of block size, prediction mode, and intra block for H.264 intra prediction. IEEE Trans Circ Syst Video Technol 20(8):1122–1132CrossRefGoogle Scholar
  3. 3.
    ITU-T, JVT (2003) “Draft ITU-T recommendation and final draft international standard of joint video specification (ITUT Rec.H.264/AVC/ISO/IEC 14496-10 AVC)”, Doc. JVTG050. Joint Video Team (JVT) of ISO/IEC MPEAG and ITUT VCEGGoogle Scholar
  4. 4.
    JVT.264/AVC reference software JM11.0. Downloaded from Accessed Aug 2012
  5. 5.
    Lim K, Kim S, Lee J, Pak D, Lee S (2012) Fast block size and mode decision algorithm for intra prediction in H.264/AVC.IEEE. Trans Consum Electron 58(2):654–660CrossRefGoogle Scholar
  6. 6.
    Lin YK, Chang TS (2005) Fast block type decision algorithm for intra prediction in H.264 FRext. In: Proceedings of IEEE international conference on image processing, Genoa, Italy, pp 585–588Google Scholar
  7. 7.
    Moon JM, Kim JH (2010) A new low-complexity integer distortion estimation method for H.264/AVC Encoder. IEEE Trans Circ Syst Video Technol 20(2):207–212CrossRefMathSciNetGoogle Scholar
  8. 8.
    Pan F, Lin X, Rahardja S, Lim KP, Li ZG, Wu D, Wu S (2005) Fast mode decision algorithm for intra-prediction in H.264/AVC video coding. IEEE Trans Circ Syst Video Technol 15(7):813–822CrossRefGoogle Scholar
  9. 9.
    Quan D, Ho YS (2010) Categorization for fast intra prediction mode decision in H.264/AVC. IEEE Trans Consum Electron 56(2):1049–1056CrossRefGoogle Scholar
  10. 10.
    Sarwer MG, Po LR (2007) Fast bit rate estimation for mode decision of H.264/AVC. IEEE Trans Circ Syst Video Technol 17(10):1402–1407CrossRefGoogle Scholar
  11. 11.
    Sarwer MG, Po LR, Wu QMJ (2008) Fast sum of absolute transformed difference based 4 × 4 intra-mode decision of H.264/AVC video coding standard. Signal Process Image Commun 23(8):571–580CrossRefGoogle Scholar
  12. 12.
    Su XQ, Ji L, Li X (2011) A fast and low complexity approach for H.264/AVC intra mode decision. Multimed Tools Appl 52(1):65–76CrossRefGoogle Scholar
  13. 13.
    Tian GF, Zhang TR, Wei XH, Goto S (2008) A block type decision algorithm for H.264/AVC intra prediction based on entropy feature. In: Proceedings of IEEE asia pacific conference on circuits and systems, Macao, China, pp1348–1351Google Scholar
  14. 14.
    Tsai AC, Paul A, Wang JC, Wang JF (2008) Intensity gradient technique for efficient intra-prediction in H.264/AVC. IEEE Trans Circ Syst Video Technol 18(5):694–698CrossRefGoogle Scholar
  15. 15.
    Tu YK, Yang JF, Sun MT (2006) Efficient rate-distortion estimation for H.264/AVC coders. IEEE Trans Circ Syst Video Technol 16(5):600–611CrossRefGoogle Scholar
  16. 16.
    Wang P, Huang H, Tan Z (2012) A fast two-step block type decision algorithm for intra prediction in H.264/AVC high profile. Multimed Tools Appl 60(1):139–160CrossRefMathSciNetGoogle Scholar
  17. 17.
    Wang JC, Wang JF, Yang JF, Chen JT (2007) A fast mode decision algorithm and its VLSI design for H.264/AVC intra-prediction. IEEE Trans Circ Syst Video Technol 17(10):1414–1422CrossRefGoogle Scholar
  18. 18.
    Wiegand T, Sullivan GJ, Bjontegaard G, Luthra A (2003) Overview of the H.264/AVC video coding standard. IEEE Trans Circ Syst Video Technol 13(7):560–576CrossRefGoogle Scholar
  19. 19.
    Zeng HQ, Ma KK, Cai CH (2010) Hierarchical intra mode decision for H.264/AVC. IEEE Trans Circ Syst Video Technol 20(6):907–912CrossRefGoogle Scholar
  20. 20.
    Zhang K, Yuan C, Li Q, Zhong YZ (2007) A fast block type decision method for H.264/AVC intra prediction. In: Proceedings of IEEE international conference on advanced communication technology, Phoenix Park, Korea, pp 673–676Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.School of Telecommunications EngineeringXidian UniversityXi’anChina

Personalised recommendations