Multimedia Tools and Applications

, Volume 75, Issue 16, pp 9903–9925 | Cite as

Video quality enhancement based on visual attention model and multi-level exposure correction

  • Guo-Shiang Lin
  • Xian-Wei Ji


Due to unfavorable environmental conditions such as lack of lighting, poor visual quality in images and videos may make intelligent image/video systems unstable. This means that visual quality enhancement plays an important role in image/video processing, computer vision, and pattern recognition. In this paper, we propose a video quality enhancement scheme based on visual attention model and multi-level exposure correction. To this end, the proposed scheme is composed of four parts: pre-processing, visual attention model generation, multi-level exposure correction, and temporal filtering. To extract more visual cues for visual attention model generation, a pre-processing is used to modify each frame. After preprocessing, facial and non-facial cues are measured to generate visual attention maps of each frame. On the basis of visual attention maps, a multi-level exposure correction algorithm is utilized to adjust the exposure level of each frame and then create several intermediate results. After fusing intermediate results, a synthesized image with good visual quality can be obtained. To avoid flicker effect, a temporal filter is exploited to make the variance of the exposure level small in the temporal domain. To evaluate the performance of the proposed scheme, some images/videos captured by mobile phones and digital cameras are tested. The experimental results show that the proposed scheme can effectively deal with the images/videos with low and high exposure levels. The results also demonstrate that the proposed scheme outperforms some existing methods in terms of visual quality.


Visual attention model Exposure correction Image fusion 



This research was supported by the Ministry of Science and Technology, Taiwan, under the grant of MOST 103-2221-E-212-004-MY2.


  1. 1.
    Aditi M, Irani S (2005) Contrast enhancement of images using human contrast sensitivity. Comput Vision Pattern Recognit 377–382. doi: 10.1145/1140491.1140506
  2. 2.
    Bhukhanwala SA, Ramabadram TV (1994) Automated global enhancement of digitized photographs. IEEE Trans Consum Electron 40(1) doi: 10.1109/30.273657
  3. 3.
    Borji A, Itti L (2013) State of the art in visual attention modeling. IEEE Trans Pattern Anal Mach Intell 35(1):185–207MathSciNetCrossRefGoogle Scholar
  4. 4.
    Chi MC, Chen MJ, Yeh CH (2008) Region-of-Interest video coding based on rate and distortion variations for H.263+. Signal Process Image Commun 23:127–142CrossRefGoogle Scholar
  5. 5.
    Chi MC, Yeh CH, Chen MJ (2009) Robust region-of-interest determination based on user attention model through visual rhythm analysis. IEEE Trans Circ Syst Video Technol 19(8):1025–1038Google Scholar
  6. 6.
    Dorai C, Venkatesh S (2002) Media computing: computational media aesthetics. Kluwer, Norwell. doi: 10.1109/93.959093 CrossRefGoogle Scholar
  7. 7.
    Engel S, Zhang X, Wandell B (1997) Colour tuning in human visual cortex measured with functional magnetic resonance imaging. Nature 388(6,637):68–71CrossRefGoogle Scholar
  8. 8.
    Hsu CT, Yeh CH, Chen CY, Chen MJ (2009) Arbitrary frame rate transcoding through temporal and spatial complexity. IEEE Trans Broadcast 55(4):767–775CrossRefGoogle Scholar
  9. 9.
    Huang SC, Cheng FC, Chiu YS (2013) Efficient contrast enhancement using adaptive gamma correction with weighting distribution. IEEE Trans Image Process 22(3)Google Scholar
  10. 10.
    Huang DY, Lin CJ, Dai SH (2014) Face recognition using the diagonal relative gradient method in a low illumination environment. J Inf Hiding Multimedia Signal Process 5(2):310–323Google Scholar
  11. 11.
    Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Patten Anal Mach Intell 20(11):1254–1259. doi: 10.1109/34.730558 CrossRefGoogle Scholar
  12. 12.
    Jiang X, Yao H, Zhang S, Lu X, Zeng W (2013) Night video enhancement using improved dark channel prior. In Proc. IEEE Int Conf Image Process 553–557. doi: 10.1109/ICIP.2013.6738114
  13. 13.
    Knutsson H (1989) Representing local structure using tensors. In 6th Scandinavian Conf. Image Analysis, pp. 244–254.Google Scholar
  14. 14.
    Krishna Kishore P, Chinna Rao B, Francis PM (2012) ARM based mobile phone- embedded real-time remote video surveillance system with network camera. Int J Emerg Technol Adv Eng 2(8):2250–2459Google Scholar
  15. 15.
    Lin GS, Chang MK, Chang YJ (2011) Gender recognition based on multi-model information fusion. The Asia-pacific signal and information processing association annual summit and conference. doi: 10.1109/ICMLC.2008.4620379
  16. 16.
    Lin G-S, Chen C-Y, Kuo C-T, Lie W-N (2014) A computing framework of adaptive support-window multi-lateral filter for image and depth processing. IEEE Trans Broadcast 60(3):452–463. doi: 10.1109/TBC.2014.2330391 CrossRefGoogle Scholar
  17. 17.
    Lin G-S, Huang H-Y, Chen W-C, Yeh C-Y, Lie W-N (2012) A stereoscopic video conversion scheme based on spatio-temporal analysis of MPEG videos. EURASIP J Adv Signal ProcessGoogle Scholar
  18. 18.
    Messina G, Castorina A, Battiato S, Bosco A (2003) Image quality improvement by adaptive exposure correction techniques. Proc ICME 2003. 549–552. doi: 10.1109/ICME.2003.1220976
  19. 19.
    Pinson MH, Wolf S (2004) A new standardized method for objectively measuring video quality. IEEE Trans Broadcast 50(3):312–322. doi: 10.1109/TBC.2004.834028 CrossRefGoogle Scholar
  20. 20.
    Shi C, Yu K, Li J, Li S (2004) Automatic image quality improvement for videoconferencing. IEEE Int Conf Acoust Speech Signal Process 3:701–704. doi: 10.1109/ICASSP.2004.1326641 Google Scholar
  21. 21.
    Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. Proc IEEE Int Conf Comput Vision Pattern Recognit 1:511–518. doi: 10.1109/CVPR.2001.990517 Google Scholar
  22. 22.
    Wu JC-H, Lin G-S, Hsu H-T, Liao Y-P, Liu K-C, Lie W-N (2013) Quality enhancement based on retinex and pseudo-HDR synthesis algorithms for endoscopic images. In Proc IEEE Int Conf Visual Commun Image Process 1–5. doi: 10.1109/VCIP.2013.6706375
  23. 23.
    Wu HR, Rao KR (2006) Digital video image quality and perceptual coding. CRC press, Taylor & Francis GroupGoogle Scholar
  24. 24.
    Xiao L, Zheng Y (2010) The Implementation of remote digital video Monitoring technology in the construction projects. Int Conf Mech Autom Control Eng (MACE). 1569–1572. doi: 10.1109/MACE.2010.5535992
  25. 25.
    Yang K-C, Guest CC, Das PK (2006) Human visual attention map for compressed video. In Proc. IEEE Int Symp Multimedia 525-532. doi: 10.1109/ISM.2006.86
  26. 26.
    Yang X, Lin W, Lu Z, Lin X, Rahardja S, Ong E, Yao S (2005) Rate control for videophone using local perceptual cues. IEEE Trans Circ Syst Video Technol. doi: 10.1109/TCSVT.2005.844458 Google Scholar
  27. 27.
    Yeh CH, Chen SM, Chern SJ (2008) Content-aware video transcoding via visual attention model analysis. Proc IEEE Int Conf Intell Inf Hiding Multimedia Signal Process 429–432Google Scholar
  28. 28.
    Yeh CH, Lin CY, Muchtar K, Knag LW (2014) Real-time background modeling based on a multi-level texture description. Inf Sci 269:106–127MathSciNetCrossRefGoogle Scholar
  29. 29.
    Zettl H, Sight S (1990) Motion: applied media aesthetics. Wadsworth, BelmontGoogle Scholar
  30. 30.
    Zhai Y, Shah M (2006) Visual attention detection in video sequences using spatiotemporal cues. In Proc. The 14th ACM international conference on multimediaGoogle Scholar
  31. 31.
    Zhao Q (2011) Research on family and shops real-time status of 3G wireless remote monitoring system. J Softw 6(5):814–818. doi: 10.4304/jsw.6.5.814-818 Google Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Department of Computer Science and Information EngineeringDa-Yeh UniversityDacunTaiwan

Personalised recommendations