Skip to main content
SpringerLink
Log in

Robust artifact-free high dynamic range imaging of dynamic scenes

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The irradiance range of the real-world scene is often beyond the capability of digital cameras. Therefore, High Dynamic Range (HDR) images can be generated by fusing images with different exposure of the same scene. However, moving objects pose the most severe problem in the HDR imaging, leading to the annoying ghost artifacts in the fused image. In this paper, we present a novel HDR technique to address the moving objects problem. Since the input low dynamic range (LDR) images captured by a camera act as static linear related backgrounds with moving objects during each individual exposures, we formulate the detection of foreground moving objects as a rank minimization problem. Meanwhile, in order to eliminate the image blurring caused by background slightly change of LDR images, we further rectify the background by employing the irradiances alignment. Experiments on image sequences show that the proposed algorithm performs significant gains in synthesized HDR image quality compare to state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Bogoni L (2000) Extending dynamic range of monochrome and color images through fusion. In: Proceedings of the IEEE International Conference on Pattern Recognition, pp 3007

  2. Boykov Y, Veksler O, Zabih R (2001) Fast approximate energy minimization via graph cuts. IEEE Trans Pattern Anal Mach Intell 1(11):1222–1239

    Article  Google Scholar 

  3. Cai JF, Candès EJ, Shen Z (2010) A singular value thresholding algorithm for matrix completion. Siam J Optim 20(4):1956–1982

    Article  MathSciNet  Google Scholar 

  4. Candès EJ, Recht B (2008) Exact matrix completion via convex optimization. Found Comput Math 9(6):717–772

    Article  MathSciNet  Google Scholar 

  5. Ce L, Deqing S (2013) On bayesian adaptive video super resolution. IEEE Trans Pattern Anal Mach Intell 36(2):346–360

    Google Scholar 

  6. Cui J, Liu Y, Xu Y, Zhao H, Zha H (2013) Tracking generic human motion via fusion of low- and high-dimensional approaches. IEEE Trans Syst Man Cybern: Syst 43(4):996–1002

    Article  Google Scholar 

  7. Debevec PE, Malik J (1997) In: Recovering High Dynamic Range Radiance Maps from Photographs, pp 369–378

  8. Erik R, Greg W, Pattanaik SN, Debevec P (2005) High dynamic range imaging, acquisition, display, and image-based lighting. Morgan Kauffman, Burlington

    Google Scholar 

  9. Gallo O, Gelfand N, Chen W-C, Tico M, Pulli K (2009) Artifact-free high dynamic range imaging. In: Proceedings of the IEEE International Conference on Computational Photography, pp 1–7

  10. Gong D, Tan M, Zhang Y, Hengel AVD, Shi Q (2016) Blind image deconvolution by automatic gradient activation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 1827–1836

  11. Gong D, Yang J, Liu L, Zhang Y, Reid I, Shen C, Hengel AVD, Shi Q (2016) From motion blur to motion flow: a deep learning solution for removing heterogeneous motion blur. In: IEEE Conference on computer vision and pattern recognition

  12. Gong D, Tan M, Zhang Y, Hengel AVD, Shi Q (2017) Self-paced kernel estimation for robust blind image deblurring. In: IEEE International Conference on Computer Vision, pp 1670–1679

  13. Gong D, Tan M, Zhang Y, van den Hengel A, Shi Q (2017) Mpgl: an efficient matching pursuit method for generalized lasso. In: AAAI

  14. Grosch T (2006) Fast and robust high dynamic range image generation with camera and object movement. In: Proceedings of the IEEE International Conference of Vision, Modeling and Visualization

  15. Hu J, Gallo O, Pulli K, Xiaobai S (2013) Hdr deghosting: How to deal with saturation?. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp 1163–1170

  16. Jacobs K, Loscos C, Ward G (2008) Automatic high dynamic range image generation of dynamic environments. IEEE Comput Graph Appl 28(2):84–93

    Article  Google Scholar 

  17. Jinno T, Okuda M (2008) Motion blur free hdr image acquisition using multiple exposures. In: IEEE International Conference on Image Processing, pp 1304–1307

  18. Kang SB, Uyttendaele M, Winder S, Szeliski R (2003) High dynamic range video. ACM Trans Graph 22(3):319–325

    Article  Google Scholar 

  19. Lee C, Li Y, Monga V (2014) Ghost-free high dynamic range imaging via rank minimization. IEEE Signal Process Lett 21(9):1045–1049

    Article  Google Scholar 

  20. Liu Y, Zhang X, Cui J, Wu C, Aghajan H, Zha H (2010) Visual analysis of child-adult interactive behaviors in video sequences. In: International Conference on Virtual Systems and Multimedia, pp 26–33

  21. Liu Y, Cui J, Zhao H, Zha H (2012) Fusion of low-and high-dimensional approaches by trackers sampling for generic human motion tracking. In: International Conference on Pattern Recognition, pp 898–901

  22. Liu L, Cheng L, Liu Y, Jia Y, Rosenblum DS (2016) Recognizing complex activities by a probabilistic interval-based model. In: Thirtieth AAAI Conference on Artificial Intelligence, pp 1266–1272

  23. Liu Y, Nie L, Han L, Zhang L, Rosenblum DS Action2activity: Recognizing complex activities from sensor data, International Joint Conference on Artificial Intelligence (IJCAI)

  24. Liu Y, Nie L, Liu L, Rosenblum DS (2016) From action to activity: Sensor-based activity recognition. Neurocomputing 181:108–115

    Article  Google Scholar 

  25. Liu Y, Zhang L, Nie L, Yan Y, Rosenblum DS Fortune teller: Predicting your career path, Thirtieth AAAI Conference on Artificial Intelligence

  26. Liu Y, Zheng Y, Liang Y, Liu S, Rosenblum DS (2016) Urban water quality prediction based on multi-task multi-view learning. In: Proceedings of the 25th International Joint Conference on Artificial Intelligence

  27. Liu Y, Liang Y, Liu S, Rosenblum DS, Zheng Y Predicting urban water quality with ubiquitous data, CoRR

  28. Lu Y, Wei Y, Liu L, Zhong J, Sun L, Liu Y (2017) Towards unsupervised physical activity recognition using smartphone accelerometers. Multimed Tools Appl 76(8):10701–10719

    Article  Google Scholar 

  29. Mertens T, Kautz J, Reeth FV (2007) Exposure fusion. In: 2007. PG ’07. 15th Pacific Conference on Computer Graphics and Applications, pp 382–390

  30. Nayar SK, Mitsunaga T (2000) High dynamic range imaging: Spatially varying pixel exposures. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp 472–479

  31. Oh T-H, Lee J-Y, Kweon IS (2013) High dynamic range imaging by a rank-1 constraint. In: Proceedings of the IEEE International Conference on Image Processing, pp 790–794

  32. Oh T-H, Lee J-Y, Tai Y-W, So KI (2015) Robust high dynamic range imaging by rank minimization. IEEE Trans Pattern Anal Mach Intell 37(6):1219–1232

    Article  Google Scholar 

  33. Sen P, Kalantari NK, Yaesoubi M, Darabi S, Goldman DB, Shechtman E (2012) Robust patch-based hdr reconstruction of dynamic scenes. ACM Trans Graph 31(6):1–11

    Article  Google Scholar 

  34. Srikantha A, Sidibé D (2012) Ghost detection and removal for high dynamic range images: Recent advances. Image Commun 27(6):650–662

    Google Scholar 

  35. Yan Q, Sun J, Li H, Zhu Y, Zhang Y (2017) High dynamic range imaging by sparse representation. Neurocomputing 269(6):160–169

    Article  Google Scholar 

  36. YongSeok H, KyoungMu L, SangUk L, Youngsu M, Joonhyuk C (2011) Ghost-free high dynamic range imaging. In: Proceedings of the IEEE International Conference on Asian Conference on Computer Vision, pp 486–500

  37. Zhang W, Cham W-K (2012) Gradient-directed multiexposure composition. IEEE Trans Image Process 21(4):2318–2323

    Article  MathSciNet  Google Scholar 

  38. Zhou X, Yang C, Yu W (2013) Moving object detection by detecting contiguous outliers in the low-rank representation. IEEE Trans Pattern Anal Mach Intell 35(3):597–610

    Article  Google Scholar 

  39. Zimmer H, Bruhn A, Weickert J (2011) Freehand HDR imaging of moving scenes with simultaneous resolution enhancement. Comput Graph Forum 30(2):795–825

    Article  Google Scholar 

Download references

Acknowledgments

The work is supported by grants NSF of China (61231016, 61301193, 61303123, 61301192), Natural Science Basis research Plan in Shaanxi Province of China (No. 2013JQ8032), Chang Jiang Scholars Program of China (100017GH030150, 15GH0301). Yanning Zhang has helped with acquisition of funding, technical editing of the manuscript and served as scientific advisors. Yu Zhu has helped with writing assistance, technical editing and language editing of the manuscript. Jinqiu Sun has helped with general supervision of our research group and language editing of the manuscript.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Northwest Polytechnical University, Xi’an, Shaanxi Province, China

    Qingsen Yan, Yu Zhu & Yanning Zhang

Authors
  1. Qingsen Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanning Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yu Zhu.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yan, Q., Zhu, Y. & Zhang, Y. Robust artifact-free high dynamic range imaging of dynamic scenes. Multimed Tools Appl 78, 11487–11505 (2019). https://doi.org/10.1007/s11042-018-6625-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6625-x

Keywords

Search

Navigation