Joint structural similarity and entropy estimation for coded-exposure image restoration

Article
  • 33 Downloads

Abstract

We address the image deblurring using coded exposure which can keep image content that may be lost by a traditional shutter. In the restoration of a coded exposure image, the automatic estimation of smear length is the key problem. Because the coded exposure image does not lose high frequency information of the image, the structural similarity compared with the original image is retained. In this paper, we propose a joint coarse to fine estimation method. By comparing structural similarity between the coded-exposure image and its restored image, the smear length can be roughly estimated first. And then the entropy of the restored image is further computed within a small range of the previously estimated smear length. An image that is restored with the wrong smear length will be far from the structure of the coded image that will have high entropy and low structure similarity with the coded exposure image.

Keywords

Coded-exposure Smear length Image restoration Structural similarity Entropy estimation 

References

  1. 1.
    Agrawal A, Xu Y (2009) Coded exposure deblurring: optimized codes for PSF estimation and invertibility. In: Paper presented at the IEEE conference on computer vision and pattern recognition, Miami, FL, USA, 20–25 June 2009Google Scholar
  2. 2.
    Cho S, Lee S (2009) Fast motion deblurring. In: Paper presented at the acm transactions on graphics, Yokohama, Japan, 16–19 December 2009Google Scholar
  3. 3.
    Dai S, Wu Y (2008) Motion from blur. In: Paper presented at the IEEE conference on computer vision and pattern recognition, Anchorage, AK, USA, 23–28 June 2008Google Scholar
  4. 4.
    Ding G, Guo Y, Zhou J, Gao Y (2016) Large-scale cross-modality search via collective matrix factorization hashing. IEEE Trans Image Process 25(11):5427–5440.  https://doi.org/10.1109/TIP.2016.2607421 MathSciNetCrossRefGoogle Scholar
  5. 5.
    Ding G, Zhou J, Guo Y, Lin Z, Zhao S, Han J (2017) Large-scale image retrieval with sparse embedded hashing. Neurocomputing 257:24–36.  https://doi.org/10.1016/j.neucom.2017.01.055 CrossRefGoogle Scholar
  6. 6.
    Ding Y, McCloskey S, Yu J (2010) Analysis of motion blur with a flutter shutter camera for non-linear motion, vol 6311. In: European conference on computer vision. Springer, Berlin, DOI  https://doi.org/10.1007/978-3-642-15549-9-2, (to appear in print)
  7. 7.
    Don ML, Fu C, Arce GR (2017) Compressive imaging via a rotating coded aperture. Appl Opt 56(3):B142–B153.  https://doi.org/10.1364/AO.56.00B142 CrossRefGoogle Scholar
  8. 8.
    Feng W, Zhang F, Qu X, Zheng S (2016) Per-pixel coded exposure for high-speed and high-resolution imaging using a digital micromirror device camera. Sensors 16(3):331–346.  https://doi.org/10.3390/s16030331 CrossRefGoogle Scholar
  9. 9.
    Feng W, Zhang F, Wang W, Xing W, Qu X (2017) Digital micromirror device camera with per-pixel coded exposure for high dynamic range imaging. Appl Opt 56(13):3831–3840.  https://doi.org/10.1364/AO.56.003831 CrossRefGoogle Scholar
  10. 10.
    Guo Y, Ding G, Han J (2017) Robust quantization for general similarity search. IEEE Trans Image Process 27(2):949–963.  https://doi.org/10.1109/TIP.2017.2766445 MathSciNetCrossRefGoogle Scholar
  11. 11.
    Guo Y, Ding G, Han J, Gao Y (2017) Zero-Shot Learning with transferred samples. IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society 26(7):3277–3290.  https://doi.org/10.1109/TIP.2017.2696747 MathSciNetCrossRefGoogle Scholar
  12. 12.
    Guo Y, Ding G, Liu L, Han J, Shao L (2017) Learning to hash with optimized anchor embedding for scalable retrieval. IEEE Trans Image Process 26 (3):1344–1354.  https://doi.org/10.1109/TIP.2017.2652730 MathSciNetCrossRefGoogle Scholar
  13. 13.
    Harshavardhan S, Gupta S, Venkatesh KS (2014) Flutter shutter based motion deblurring in complex scenes. In: Paper presented at the India conference, Mumbai, India, 13–15 Dec. 2013Google Scholar
  14. 14.
    Hitomi Y, Gu J, Gupta M, Mitsunaga T, Nayar SK (2011) Video from a single coded exposure photograph using a learned over-complete dictionary. In: Paper presented at the IEEE international conference on computer vision, Barcelona, Spain, 6–13 nov. 2011Google Scholar
  15. 15.
    Holloway J, Sankaranarayanan AC, Veeraraghavan A, Tambe S (2013) Flutter shutter video camera for compressive sensing of videos. In: Paper presented at the IEEE international conference on computational photography, Seattle, WA, USA, 28–29 April 2012Google Scholar
  16. 16.
    Huang K, Liang H, Ren W, Zhang J (2013) Motion blur identification using image statistics for coded exposure photography. Springer, New YorkCrossRefGoogle Scholar
  17. 17.
    Jeon HG, Lee JY, Han Y, Kim SJ, Kweon IS (2013) Fluttering pattern generation using modified legendre sequence for coded exposure imaging. In: Paper presented at the IEEE international conference on computer vision, Sydney, NSW, Australia, 1–8 Dec. 2013Google Scholar
  18. 18.
    Jeon HG, Lee JY, Han Y, Kim SJ (2016) Complementary sets of shutter sequences for motion deblurring. In: Paper presented at the IEEE international conference on computer vision, Santiago, Chile, 7–13 Dec. 2015Google Scholar
  19. 19.
    Jeon HG, Lee JY, Han Y, Kim SJ, Kweon IS (2017) Generating fluttering patterns with low autocorrelation for coded exposure imaging. Int J Comput Vis 123(2):269–286.  https://doi.org/10.1007/s11263-016-0976-4 CrossRefGoogle Scholar
  20. 20.
    Jeon HG, Lee JY, Han Y, Kim SJ, Kweon IS (2017) Multi-image deblurring using complementary sets of fluttering patterns. IEEE Trans Image Process 26(5):2311–2326.  https://doi.org/10.1109/TIP.2017.2675202 MathSciNetCrossRefGoogle Scholar
  21. 21.
    Liu D, Gu J, Hitomi Y, Gupta M, Mitsunaga T, Nayar SK (2014) Efficient space-time sampling with pixel-wise coded exposure for high-speed imaging. IEEE Trans Pattern Anal Mach Intell 36(2):248–260.  https://doi.org/10.1109/TPAMI.2013.129 CrossRefGoogle Scholar
  22. 22.
    Mccloskey S, Ding Y, Yu J (2012) Design and estimation of coded exposure point spread functions. IEEE Trans Pattern Anal Mach Intell 34(10):2071–2077.  https://doi.org/10.1109/TPAMI.2012.108 CrossRefGoogle Scholar
  23. 23.
    Michaelides EE (2008) Entropy, order and disorder. Open Thermodynamics Journal 2(1):7–11.  https://doi.org/10.2174/1874396X00802010007 CrossRefGoogle Scholar
  24. 24.
    Raskar R, Agrawal A, Tumblin J (2006) Coded exposure photography: motion deblurring using fluttered shutter. In: Paper presented at the ACM SIGGRAPH, Boston, Massachusetts, July 30–August 03, 2006Google Scholar
  25. 25.
    Shan Q, Jia J, Agarwala A (2008) High-quality motion deblurring from a single image. In: Paper presented at the acm transactions on graphics, Los Angeles, California, 11–15 August 2008Google Scholar
  26. 26.
    Silva EA, Agaian SS (2007) Quantifying image similarity using measure of enhancement by entropy. Proc SPIE-Int Soc Opt Eng 6579.  https://doi.org/10.1117/12.720087
  27. 27.
    Tendero Y, Osher S (2016) On a mathematical theory of coded exposure. Research in the Mathematical Sciences 3(1):4–42.  https://doi.org/10.1186/s40687-015-0051-8 MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society 13(4):600–612.  https://doi.org/10.1109/TIP.2003.819861 CrossRefGoogle Scholar
  29. 29.
    Wang Z, Simoncelli EP, Bovik AC (2004) Multiscale structural similarity for image quality assessment. In: Paper presented at the conference record of the thirty-seventh asilomar conference on signals, systems and computers Pacific Grove, CA, USA, 9–12 Nov. 2003Google Scholar
  30. 30.
    Zhang J, Suo Y, Zhao C, Tran TD (2015) CMOS Implementation of pixel-wise coded exposure imaging for insect-based sensor node. In: Paper presented at the biomedical circuits and systems conference, Atlanta, GA, USA, 22–24 Oct. 2015Google Scholar
  31. 31.
    Zhang J, Xiong T, Tran T, Chin S, Etiennecummings R (2016) Compact all-CMOS spatiotemporal compressive sensing video camera with pixel-wise coded exposure. Opt Express 24(8):9013–9024.  https://doi.org/10.1364/OE.24.009013 CrossRefGoogle Scholar
  32. 32.
    Zhou C, Nayar S (2009) What are good apertures for defocus deblurring?. In: Paper presented at the IEEE international conference on computational photography, San Francisco, CA, USA, 16–17 April 2009Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Information and Communication EngineeringDalian University of TechnologyDalianChina
  2. 2.School of Information EngineeringDalian Ocean UniversityDalianChina

Personalised recommendations