Advertisement

From Night to Day: GANs Based Low Quality Image Enhancement

  • Yingying Meng
  • Deqiang Kong
  • Zhenfeng ZhuEmail author
  • Yao Zhao
Article
  • 69 Downloads

Abstract

Nighttime images have lower contrast and higher noise than their daytime counterparts of the same scene. The aim of nighttime image enhancement is to improve the visual quality of nighttime images, so that they are visually as close as possible to daytime images. This problem is still challenging because of the deteriorated conditions of illumination lack and uneven lighting. In this paper, we propose a generative adversarial networks (GANs) based framework for nighttime image enhancement. To take advantage of GANs’ powerful ability of generating image from real data distribution, we make the established network well constrained by combining several loss functions including adversarial loss, perceptual loss, and total variation loss. Particularly, a pre-trained network is applied to leverage the perceptual loss which is beneficial to generate high-quality images. Meanwhile, for tackling the light-at-night effect, we present a fusion network in which the dark channel prior based illumination compensation is employed for the training of generator network. Experimental results have demonstrated the effectiveness of the proposed nighttime image enhancement network.

Keywords

Nighttime image Image enhancement Generative adversarial networks 

Notes

Acknowledgements

The authors would like to thank the reviewers for their valuable suggestions. This work was in part supported by the National Natural Science Foundation of China (Nos. 61572068, 61532005) and in part by the Fundamental Research Funds for the Central Universities under Grant No. 2018JBZ001.

References

  1. 1.
    Ketcham DJ (1976) Real-time image enhancement techniques. OSA Image Process 74(2):120–125CrossRefGoogle Scholar
  2. 2.
    Rahman Z, Jobson DJ, Woodell GA (2002) Multi-scale retinex for color image enhancement. In: International conference on image processing, pp 1003–1006Google Scholar
  3. 3.
    Jobson DJ, Rahman Z, Woodell GA (1997) A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Trans Image Process 6(7):965–976CrossRefGoogle Scholar
  4. 4.
    He K, Sun J, Tang X (2009) Single image haze removal using dark channel prior. In: IEEE conference on computer vision and pattern recognition, pp 1956–1963Google Scholar
  5. 5.
    Dong X, Wang G, Pang Y, Li W, Wen J, Meng W, Lu Y (2011) Fast efficient algorithm for enhancement of low lighting video. In: IEEE international conference on multimedia and expo, pp 1–6Google Scholar
  6. 6.
    Jiang X, Yao H, Zhang S, Lu X, Zeng W (2013) Night video enhancement using improved dark channel prior. In: IEEE international conference on image processing, pp 553–557Google Scholar
  7. 7.
    Gong Y, Lee Y, Nguyen TQ (2016) Nighttime image enhancement applying dark channel prior to raw data from camera. In: International SoC design conference, pp 173–174Google Scholar
  8. 8.
    Iizuka S, Simo-Serra E, Ishikawa H (2017) Globally and locally consistent image completion. ACM Trans Graph 36(4):107:1–107:14CrossRefGoogle Scholar
  9. 9.
    Yeh RA, Chen C, Lim T, Hasegawa-Johnson M, Do MN (2017) Semantic image inpainting with perceptual and contextual losses. In: IEEE conference on computer vision and pattern recognitionGoogle Scholar
  10. 10.
    Taigman Y, Polyak A, Wolf L (2017) Unsupervised cross-domain image generation. In: International conference on learning representationsGoogle Scholar
  11. 11.
    Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, Aitken AP, Tejani A, Totz J, Wang Z, Shi W (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: IEEE conference on computer vision and pattern recognition, pp 105–114Google Scholar
  12. 12.
    He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition, pp 770–778Google Scholar
  13. 13.
    Zhu J, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE international conference on computer vision, pp 2242–2251Google Scholar
  14. 14.
    Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Conference on neural information processing systems, pp 2672–2680Google Scholar
  15. 15.
    Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. In: International conference on machine learning, pp 214–223Google Scholar
  16. 16.
    Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville AC (2017) Improved training of Wasserstein GANs. In: Conference on neural information processing systems, pp 5769–5779Google Scholar
  17. 17.
    Liu M, Breuel T, Kautz J (2017) Unsupervised image-to-image translation networks. In: Conference on neural information processing systems, pp 700–708Google Scholar
  18. 18.
    Huang R, Zhang S, Li T, He R (2017) Beyond face rotation: global and local perception GAN for photorealistic and identity preserving frontal view synthesis. In: IEEE international conference on computer vision, pp 2458–2467Google Scholar
  19. 19.
    Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. In: International conference on machine learning, vol 30, p 1Google Scholar
  20. 20.
    Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning, pp 448–456Google Scholar
  21. 21.
    Radford A, Metz L, Chintala S (2016) Unsupervised representation learning with deep convolutional generative adversarial networks. In: International conference on learning representationsGoogle Scholar
  22. 22.
    Johnson J, Alahi A, Li F (2016) Perceptual losses for real-time style transfer and super-resolution. In: European conference on computer vision, pp 694–711Google Scholar
  23. 23.
    Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International conference on learning representationsGoogle Scholar
  24. 24.
    Deng J, Dong W, Socher R, Li L, Li K, Li F (2009) ImageNet: a large-scale hierarchical image database. In: IEEE conference on computer vision and pattern recognition, pp 248–255Google Scholar
  25. 25.
    Rudin LI, Osher S, Fatemi E (1992) Nonlinear total variation based noise removal algorithms. Phys D Nonlinear Phenom 60:259–268MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Strong D, Chan T (2003) Edge-preserving and scale-dependent properties of total variation regularization. Inverse Probl 19(6):165–187MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Milford M, Wyeth GF (2012) SeqSLAM: visual route-based navigation for sunny summer days and stormy winter nights. In: IEEE international conference on robotics and automation, pp 1643–1649Google Scholar
  28. 28.
    Bermejo S, Cabestany J (2001) Oriented principal component analysis for large margin classifiers. Neural Netw 14(10):1447–1461CrossRefzbMATHGoogle Scholar
  29. 29.
    Isola P, Zhu J, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: IEEE conference on computer vision and pattern recognition, pp 5967–5976Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Yingying Meng
    • 1
    • 2
  • Deqiang Kong
    • 1
    • 2
    • 3
  • Zhenfeng Zhu
    • 1
    • 2
    Email author
  • Yao Zhao
    • 1
    • 2
  1. 1.Institute of Information ScienceBeijing Jiaotong UniversityBeijingChina
  2. 2.Beijing Key Laboratory of Advanced Information Science and Network TechnologyBeijingChina
  3. 3.MicrosoftBeijingChina

Personalised recommendations