Advertisement

Journal of Mathematical Imaging and Vision

, Volume 61, Issue 4, pp 515–533 | Cite as

Automatic Flare Spot Artifact Detection and Removal in Photographs

  • Patricia VitoriaEmail author
  • Coloma Ballester
Article

Abstract

Flare spot is one type of flare artifact caused by a number of conditions, frequently provoked by one or more high-luminance sources within or close to the camera field of view. When light rays coming from a high-luminance source reach the front element of a camera, it can produce intra-reflections within camera elements that emerge at the film plane forming non-image information or flare on the captured image. Even though preventive mechanisms are used, artifacts can appear. In this paper, we propose a robust computational method to automatically detect and remove flare spot artifacts. Our contribution is threefold: firstly, we propose a characterization which is based on intrinsic properties that a flare spot is likely to satisfy; secondly, we define a new confidence measure able to select flare spots among the candidates; and, finally, a method to accurately determine the flare region is given. Then, the detected artifacts are removed by using exemplar-based inpainting. We show that our algorithm achieves top-tier quantitative and qualitative performance.

Keywords

Flare artifacts Flare spot Mathematical morphology Inpainting Confidence measure 

Notes

Acknowledgements

The authors acknowledge partial support by MINECO/FEDER UE project, with reference TIN2015-70410-C2-1-R, and by H2020-MSCA-RISE-2017 project with reference 777826 NoMADS.

References

  1. 1.
    Agarwal, S., Furukawa, Y., Snavely, N., Simon, I., Curless, B., Seitz, S.M., Szeliski, R.: Building rome in a day. Commun. ACM 54(10), 105–112 (2011)CrossRefGoogle Scholar
  2. 2.
    Aljabar, P., Heckemann, R.A., Hammers, A., Hajnal, J.V., Rueckert, D.: Multi-atlas based segmentation of brain images: atlas selection and its effect on accuracy. Neuroimage 46(3), 726–738 (2009)CrossRefGoogle Scholar
  3. 3.
    Arias, P., Facciolo, G., Caselles, V., Sapiro, G.: A variational framework for exemplar-based image inpainting. Int. J. Comput. Vis. 93, 319–347 (2011).  https://doi.org/10.1007/s11263-010-0418-7 MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Bay, H., Fasel, B., Van Gool, L.: Interactive museum guide: fast and robust recognition of museum objects. In: Proc. Int. Workshop on Mobile Vision (2006)Google Scholar
  5. 5.
    Boynton, P.A., Kelley, E.F.: Liquid-filled camera for the measurement of high-contrast images. In: Cockpit Displays X, vol. 5080, pp. 370–379. International Society for Optics and Photonics (2003)Google Scholar
  6. 6.
    Chabert, F.: Automated lens flare removal. Technical report, Stanford University, Department of Electrical Engineering (2015)Google Scholar
  7. 7.
    Criminisi, A., Pérez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004)CrossRefGoogle Scholar
  8. 8.
    Demanet, L., Song, B., Chan, T.: Image inpainting by correspondence maps: a deterministic approach. Appl. Comput. Math. 1100(217–50), 99 (2003)Google Scholar
  9. 9.
    Dice, L.R.: Measures of the amount of ecologic association between species. Ecology 26(3), 297–302 (1945)CrossRefGoogle Scholar
  10. 10.
    Evans, E. D.: An analysis and reduction of flare light in optical systems. PhD thesis, The Ohio State University (1988)Google Scholar
  11. 11.
    Fedorov, V., Facciolo, G., Arias, P.: Variational framework for non-local inpainting. Image Process. On Line 5, 362–386 (2015)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Fergus, R., Perona, P., Zisserman, A.: Object class recognition by unsupervised scale-invariant learning. In: 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings, vol. 2, pp. II–II. IEEE (2003)Google Scholar
  13. 13.
    Gu, J., Ramamoorthi, R., Belhumeur, P., Nayar, S.: Removing image artifacts due to dirty camera lenses and thin occluders. ACM Trans. Graphics 28, 144 (2009)CrossRefGoogle Scholar
  14. 14.
    Hara, T., Saito, H., Kanade, T.: Removal of glare caused by water droplets. In: CVMP’09. Conference for Visual Media Production, pp 144–151. IEEE (2009)Google Scholar
  15. 15.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003)zbMATHGoogle Scholar
  16. 16.
    Kawai, N., Sato, T., Yokoya, N.: Image inpainting considering brightness change and spatial locality of textures and its evaluation. In: Pacific-Rim Symposium on Image and Video Technology, pp 271–282. Springer (2009)Google Scholar
  17. 17.
    Kojima, T., Matsumaru, T., Banno, M.: Computer-simulation analysis of ghost images in photographic objectives. In: 1980 International Lens Design Conference, vol. 237, pp. 504–512. International Society for Optics and Photonics (1980)Google Scholar
  18. 18.
    Kondo, H., Chiba, Y., Yoshida, T.: Veiling glare in photographic systems. Opt. Eng. 21(2), 212343 (1982)CrossRefGoogle Scholar
  19. 19.
    Lindeberg, T.: Scale-space theory: a basic tool for analyzing structures at different scales. J. Appl. Stat. 21(1–2), 225–270 (1994)CrossRefGoogle Scholar
  20. 20.
    Lindeberg, T.: Feature detection with automatic scale selection. Int. J. Comput. Vis. 30(2), 79–116 (1998)CrossRefGoogle Scholar
  21. 21.
    Lowe, D.G.: Object recognition from local scale-invariant features. In: The proceedings of the seventh IEEE international conference on Computer vision, vol. 2, pp. 1150–1157. IEEE (1999)Google Scholar
  22. 22.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  23. 23.
    Macleod, H.A.: Thin-Film Optical Filters. CRC Press, Boca Raton (2010)CrossRefGoogle Scholar
  24. 24.
    Martin, S.: Glare characteristics of lenses. Opt. Acta 19, 499–513 (1972)CrossRefGoogle Scholar
  25. 25.
    Matsuda, S., Nitoh, T.: Flare as applied to photographic lenses. Appl. Opt. 11(8), 1850–1856 (1972)CrossRefGoogle Scholar
  26. 26.
    Mikolajczyk, K.: Detection of local features invariant to affines transformations. PhD Thesis, Institut National Polytechnique de Grenoble-INPG (2002)Google Scholar
  27. 27.
    Nussberger, A., Grabner, H., Van Gool, L.: Robust aerial object tracking from an airborne platform. IEEE Aerosp. Electron. Syst. Mag. 31(7), 38–46 (2016)CrossRefGoogle Scholar
  28. 28.
    Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., Toyama, K.: Digital photography with flash and no-flash image pairs. In: ACM Trans. Graphics (TOG), vol. 23, pp. 664–672. ACM (2004)Google Scholar
  29. 29.
    Prokopetc, K., Bartoli, A.: Slim (slit lamp image mosaicing): handling reflection artifacts. Int. J. Comput. Assist. Radiol. Surg. 12, 1–10 (2017)CrossRefGoogle Scholar
  30. 30.
    Psotny, D.: Removing lens flare from digital photographs. Technical Report, Charles Univ. Prague (2009)Google Scholar
  31. 31.
    Raskar, R., Agrawal, A., Wilson, C.A., Veeraraghavan, A.: Glare aware photography: 4d ray sampling for reducing glare effects of camera lenses. ACM Trans. Graphics 27(3), 56 (2008)CrossRefGoogle Scholar
  32. 32.
    Ray, S. F.: Applied photographic optics: lenses and optical systems for photography, film, video, electronic and digital imaging, (2002)Google Scholar
  33. 33.
    Rey-Otero, I., Delbracio, M.: Anatomy of the sift method. Image Process. On Line 4, 370–396 (2014).  https://doi.org/10.5201/ipol.2014.82 CrossRefGoogle Scholar
  34. 34.
    Snavely, N., Seitz, S.M., Szeliski, R.: Photo tourism: exploring photo collections in 3d. ACM Trans. Graphics 25, 835–846 (2006)CrossRefGoogle Scholar
  35. 35.
    Steenstrup, S., Munk, O.: Automatic apparatus for measuring veiling-glare distribution. Opt. Acta 27, 939–947 (1980)CrossRefGoogle Scholar
  36. 36.
    Talvala, E.-V., Adams, A., Horowitz, M., Levoy, M.: Veiling glare in high dynamic range imaging. ACM Trans. Graphics 26, 37 (2007)CrossRefGoogle Scholar
  37. 37.
    Tan, R. T., Ikeuchi, K.: Separating reflection components of textured surfaces using a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(2):178–193 (2005). ISSN 0162-8828.  https://doi.org/10.1109/TPAMI.2005.36
  38. 38.
    Tuytelaars, T., Mikolajczyk, K., et al.: Local invariant feature detectors: a survey. Found. Trends® Comput. Graphics Vis. 3(3), 177–280 (2008)CrossRefGoogle Scholar
  39. 39.
    Von Gioi, R.G., Monasse, P., Morel, J.-M., Tang, Z.: Towards high-precision lens distortion correction. In: 2010 17th IEEE International Conference on Image Processing, pp 4237–4240. IEEE (2010)Google Scholar
  40. 40.
    Wels, M., Zheng, Y., Carneiro, G., Huber, M., Hornegger, J., Comaniciu, D.: Fast and robust 3-d mri brain structure segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp 575–583. Springer (2009)Google Scholar
  41. 41.
    Wexler, Y., Shechtman, E., Irani, M.: Space-time completion of video. IEEE Trans. Pattern Anal. Mach. Intell. 29(3), 463–476 (2007)CrossRefGoogle Scholar
  42. 42.
    Wu, T.-P., Tang, C.-K.: A bayesian approach for shadow extraction from a single image. In: ICCV 2005. Tenth IEEE International Conference on Computer Vision, 2005, vol. 1, pp. 480–487. IEEE (2005)Google Scholar
  43. 43.
    Zhang, J., Marszałek, M., Lazebnik, S., Schmid, C.: Local features and kernels for classification of texture and object categories: a comprehensive study. Int. J. Comput. Vis. 73(2), 213–238 (2007)CrossRefGoogle Scholar
  44. 44.
    Zinemanas, P., Arias, P., Haro, G., Gomez, E.: Visual music transcription of clarinet video recordings trained with audio-based labelled data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 463–470 (2017)Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Universitat Pompeu FabraBarcelonaSpain

Personalised recommendations