Skip to main content

Advertisement

Log in

A novel H.264 rate control algorithm with consideration of visual attention

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In video coding, a well-designed rate control scheme should be concerned with both the objective and subjective quality. However, the existing H.264 rate control algorithms mainly aim at improving the objective quality without considering the human visual system. In this paper, we propose a novel rate control algorithm that takes into account visual attention. In a group of pictures, bits allocated to each frame are related to the local motion attention in it, and more bits are allocated to the frames with strong local motion attention. Similarly, in each frame, more bits are assigned to visually significant macroblocks (MBs), and fewer to visually insignificant MBs. Experiment results show that the proposed algorithm improves the coding quality in frames with strong local motion, and reduces PSNR fluctuation across frames by up to 22.15%. In addition, PSNR in visually important regions is increased by up to 1.45 dB as compared to the standard H.264 rate control scheme that improves the subjective quality. Increased computation complexity of the proposed algorithm is less than 4%, which is negligible.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Adiono T, Isshiki T, Ito K, Ohtsuka T, Li D, Honsawek C, Kunieda H (2000) Face focus coding under H.263+ video coding standard. IEEE Asia-Pacific Conf Circuits Syst 1(No. 1):461–464

    Google Scholar 

  2. Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell PAMI-8:679–698

    Article  Google Scholar 

  3. Chen ZZ, Han JW, Ngan KN (2006) Dynamic bit allocation for multiple video object coding. IEEE Trans Multimed 8(6):1117–1124

    Article  Google Scholar 

  4. Deutsch J, Deutsch D (1963) Attention: some theoretical considerations. Psychol Rev 70:80–90

    Article  Google Scholar 

  5. Ding W, Liu B (1996) Rate control of MPEG video coding and recording by rate-quantization modeling. IEEE Trans Circuits Syst Video Technol 6(1):12–20

    Article  MathSciNet  Google Scholar 

  6. Eleftheriadis A, Jacquin A (1994) Model assisted coding of video teleconferencing sequences at low bit rates. IEEE Int Symp Circuits Syst pp. 177–180

  7. http://iphome.hhi.de/suehring/tml/ for JM 17.2 software, 2010

  8. ISO-IEC JTC1/SC29/WG11 (1993) Coding representation of pictures and audio information, MPEG Phase 2 Test Model 5, Apr

  9. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 10(11):1254–1259

    Article  Google Scholar 

  10. Jo SJ, Lee YH (1996) A statistical feedforward/feedback buffer control for the transmission of digital video signals compressed by DCT-based intrafield coding. IEEE Trans Image Process 5(3):527–529

    Article  Google Scholar 

  11. Juan M, Chi MC, Hsu CT, Chen JW (2003) ROI video coding based on H.263+ with robust skin-color detection technique. IEEE Trans Consum Electron 49(No. 3):724–730

    Article  Google Scholar 

  12. Kim WJ, Yi JW, Kim SD (1999) A bit allocation method based on picture activity for still image coding. IEEE Trans Image Process 8(7):974–977

    Article  Google Scholar 

  13. Koch C, Ullman S (1985) Shift in selective visual attention: toward the underlying neural circuitry. Human Neurobiol 4(4):219–227

    Google Scholar 

  14. Li ZG, Pan F, Lim KP, Feng GN (2003) Adaptive basic unit layer rate control for JVT. JVT-G012, 7Th meeting, Pattaya, Thailand, 7–14, Mar

  15. Ma YF, Hua XS, Lu L, Zhang HJ (2005) A generic framework of user attention model and its application in video summarization. IEEE Trans Multimed 7(5):907–919

    Article  Google Scholar 

  16. Park N, Choi K, Kim J (1995) Adaptive cosine transform coding in human visual model for data broadcasting channel. Asia Pacific Conference Communications (APCC), pp. 658–662

  17. Ramos MG, Hemami SS (1996) Edge-adaptive JPEG image compression. SPIE Vis Commun Image Process pp. 1082–1093

  18. Rapantzikos K, Tsapatsoulis N, Auvrithis Y, Kollias S (2007) Bottom-up spatiotemporal visual attention model for video analysis. IET Image Process 1(2):237–248

    Article  Google Scholar 

  19. Ribas-Corbera J, Lei SM (1999) Rate control in DCT video coding for low-delay communications. IEEE Trans Circuits Syst Video Technol 9(1):172–185

    Article  Google Scholar 

  20. Shen L, Liu Z, Zhang Z, Shi X (2009) Frame-level bit allocation based on incremental PID algorithm and frame complexity estimation. J Vis Commun Image Represent 20(1):28–34

    Article  Google Scholar 

  21. Shen L, Liu Z, Zhang Z, Yan H (2009) Motion attention based frame-level bit allocation scheme for H.264. ACM International Conference on Internet Multimedia Computing and Service, pp. 175–180, Nov

  22. Song HJ, Jay Kuo CC (2004) A region-based H.263+ codec and its rate control for low VAR video. IEEE Trans Multimed 6(3):489–500

    Article  Google Scholar 

  23. Su YP, Sun MT, Hsu V (2005) Global motion estimation from coarsely sampled motion vector field and the applications. IEEE Trans Circuits Syst Video Technol 15(No. 2):232–242

    Article  Google Scholar 

  24. Sun Y, Ahmad I (2004) A robust and adaptive rate control algorithm for object-based video coding. IEEE Trans Circuits Syst Video Technol 14(10):1167–1182

    Article  Google Scholar 

  25. Sun Y, Ahmad I, Li DD, Zhang YQ (2006) Region-based rate control and bit allocation for wireless video transmission. IEEE Trans Multimed 8(1):1–10

    Article  MATH  Google Scholar 

  26. Tang CW (2007) Spatiotemporal visual considerations for video coding. IEEE Trans Multimed 9(2):231–238

    Article  Google Scholar 

  27. Tang CW, Chen CH, Tsai CJ (2006) Visual sensitivity guided bit allocation for video coding. IEEE Trans Multimed 8(1):11–18

    Article  Google Scholar 

  28. Tian L, Sun Y, Sun SX (2009) Frame complexity prediction for H.264/AVC rate control. IEEE International conference on Multimedia and Expo, pp. 45–48, July

  29. Yang X, Lin W, Lu Z, Lin X, Rahardja S, Ong EP, Yao SS (2005) Rate control for videophone using local perceptual cues. IEEE Trans Circuits Syst Video Technol 15(4):496–507

    Article  Google Scholar 

  30. Yang L, Zhang L, Ma S, Zhao D (2009) A ROI quality adjustable rate control scheme for low bitrate video coding. Picture Coding Symposium, Chicago, USA, May.06-08

  31. Zhu Z, Wang Y, Bai Y, Jiang G (2010) On optimizing H.264/AVC rate control by improving R-D model and incorporating HVS characteristics. EURASIP J Adv Signal Process. doi:10.1155/2010/830605

Download references

Acknowledgment

The authors are grateful to Ms Xiuya Yuan for her assistance in improving the language usage. This work is sponsored by Shanghai Rising-Star Program (11QA1402400) and Innovation Program of Shanghai Municipal Education Commission (No.10YZ09), and is supported by the National Natural Science Foundation of China under grant No. 60832003, 60902085 and 61171084.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Liquan Shen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shen, L., Liu, Z. & Zhang, Z. A novel H.264 rate control algorithm with consideration of visual attention. Multimed Tools Appl 63, 709–727 (2013). https://doi.org/10.1007/s11042-011-0893-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-011-0893-z

Keywords

Navigation