Abstract
Occupying dead pixels, removing uninterested objects and shadows are often desired in the applications of an UAV to extract the natural and man-made feature boundaries. Image inpainting provides a mean to reconstruct the image. The basic idea behind inpainting methods is to naturally fill in absent or lacking portion of an image by using information from the surrounding area. Applications of this technique include the rebuilding of imperfect photographs and films, elimination of superimposed text, removal/replacement of unwanted objects, redeye correction, image coding. This paper reviews various image inpainting methods like PDE based image inpainting, wavelet-based inpainting, structural inpainting, exemplar-based image inpainting and textural inpainting with their variations. Image inpainting can also be used indirectly in squeezing image where some percentage of the original image is transmitted, and the whole image can be reconstructed on the other end using a pre-trained neural network. The critical reviews of each of these traditional methods along with the latest CNN based techniques are compared and suitability of these techniques for examining or repairing the UAV image is analyzed. In this paper, some of the existing quality assessment metrics like PSNR, MSE, ASVS, BorSal etc.related to image inpainting are also discussed.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ratliff BM, Tyo JS, Boger JK, Black WT, Bowers DL, Fetrow MP (2007) Dead pixel replacement in lwir microgrid polarimeters. Opt Express 15(12):7596–7609
Guillemot C, Le Meur O (2014) Image inpainting: overview and recent advances. IEEE Signal Process Mag 31(1):127–144
Efros AA, Leung TK (1999) Texture synthesis by non-parametric sampling. In: Proceedings of the 7th IEEE international conference on computer vision, vol 2. IEEE, pp 1033–1038
Wei LY, Levoy M (2000) Fast texture synthesis using tree-structured vector quantization. In: Proceedings of the 27th annual conference on computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co, pp 479–488
Bugeau A, BertalmÃo M, Caselles V, Sapiro G (2010) A comprehensive framework for image inpainting. IEEE Trans Image Process 19(10):2634–2645
Criminisi A, Pérez P, Toyama K (2004) Region filling and object removal by exemplar based image inpainting. IEEE Trans Image Process 13(9):1200–1212
Drori I, Cohen-Or D, Yeshurun H (2003) Fragment-based image completion. In: ACM Transactions on graphics (TOG), vol 22. ACM, pp 303–312
Bentley JL (1975) Multidimensional binary search trees used for associative searching. Commun ACM 18(9):509–517
Bertalmio M, Vese L, Sapiro G, Osher S (2003) Simultaneous structure and texture image inpainting. IEEE Trans Image Process 12(8):882–889
Starck JL, Elad M, Donoho DL (2005) Image decomposition via the combination of sparse representations and a variational approach. IEEE Trans Image Process 14(10):1570–1582
Komodakis N (2006) Image completion using global optimization. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06), vol 1. IEEE, pp 442–452
Sun J, Yuan L, Jia J, Shum HY (2005) Image completion with structure propagation. In: ACM transactions on graphics (ToG), vol 24. ACM, pp 861–868
Fadili JM, Starck JL, Elad M, Donoho DL (2009) Mcalab: Reproducible research in signal and image decomposition and inpainting. Comput Sci Eng 1:44–63
Xu Z, Sun J (2010) Image inpainting by patch propagation using patch sparsity. IEEE Trans Image Process 19(5):1153–1165
Ardis PA, Brown CM, Singhal A (2010) Inpainting quality assessment. J Electron Imaging 19(1):011002
Gupta K, Kazi S, Kong T (2016) Deeppaint: a tool for image inpainting. Google Scholar
Oncu AI, Deger F, Hardeberg JY (2012) Evaluation of digital inpainting quality in the context of artwork restoration. In: European conference on computer vision. Springer, pp 561–570
Venkatesh MV, Sen-ching SC (2010) Eye tracking based perceptual image inpainting quality analysis. In: 2010 IEEE international conference on image processing. IEEE, pp 1109–1112
Schmidt U, Gao Q, Roth S (2010) A generative perspective on mrfs in low-level vision. In: 2010 IEEE computer society conference on computer vision and pattern recognition. IEEE, pp 1751–1758
Liu J, Musialski P, Wonka P, Ye J (2013) Tensor completion for estimating missing values in visual data. IEEE Trans Pattern Anal Mach Intell 35(1):208–220
Richard MMOBB, Chang MYS (2001) Fast digital image inpainting. In: Appeared in the proceedings of the international conference on visualization, imaging and image processing (VIIP 2001), Marbella, Spain, pp 106–107
Bertalmio M, Sapiro G, Caselles V, Ballester C (2000) Image inpainting. In: Proceedings of the 27th annual conference on computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co, pp 417–424
Telea A (2004) An image inpainting technique based on the fast marching method. J Graph Tools 9(1):23–34
Tschumperlé D (2006) Fast anisotropic smoothing of multi-valued images using curvature-preserving pde’s. Int J Comput Vis 68(1):65–82
Rudin LI, Osher S, Fatemi E (1992) Nonlinear total variation based noise removal algorithms. Physica D 60(1–4):259–268
Chan TF, Shen J (2001) Nontexture inpainting by curvature-driven diffusions. J Vis Commun Image Represent 12(4):436–449
Shen J, Kang SH, Chan TF (2003) Euler’s elastica and curvature-based inpainting. SIAM J Appl Math 63(2):564–592
Ashikhmin M (2001) Synthesizing natural textures. In: Proceedings of the 2001 symposium on interactive 3D graphics, Citeseer, pp 217–226
Liang L, Liu C, Xu YQ, Guo B, Shum HY (2001) Real-time texture synthesis by patch-based sampling. ACM Trans Graph (ToG) 20(3):127–150
Barnes C, Shechtman E, Goldman DB, Finkelstein A (2010) The generalized patchmatch correspondence algorithm. In: European conference on computer vision. Springer, pp 29–43
Efros AA, Freeman WT (2001) Image quilting for texture synthesis and transfer. In: Proceedings of the 28th annual conference on computer graphics and interactive techniques. ACM, pp 341–346
Barnes C, Shechtman E, Goldman DB, Finkelstein A (2010) Supplementary material for the generalized patchmatch correspondence algorithm. Retrieved from on Sep 9, 6
Bertalmio M, Bertozzi AL, Sapiro G (2001) Navier-stokes, fluid dynamics, and image and video inpainting. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. CVPR 2001, vol 1. IEEE, pp I–I
Elad M, Starck JL, Querre P, Donoho DL (2005) Simultaneous cartoon and texture image inpainting using morphological component analysis (mca). Appl Comput Harmon Anal 19(3):340–358
Aujol JF, Ladjal S, Masnou S (2010) Exemplar-based inpainting from a variational point of view. SIAM J Math Anal 42(3):1246–1285
Cheng Q, Shen H, Zhang L, Li P (2014) Inpainting for remotely sensed images with a multichannel nonlocal total variation model. IEEE Trans Geosci Remote Sens 52(1):175–187
Nalawade VV, Ruikar SD Image inpainting using wavelet transform. Int J Adv Eng Technol E-ISSN, 0976–3945
Shen H, Zhang L (2009) A map-based algorithm for destriping and inpainting of remotely sensed images. IEEE Trans Geosci Remote Sens 47(5):1492–1502
Cai N, Su Z, Lin Z, Wang H, Yang Z, Ling BWK (2017) Blind inpainting using the fully convolutional neural network. Vis Comput 33(2):249–261
Xie J, Xu L, Chen E (2012) Image denoising and inpainting with deep neural networks. In: Advances in neural information processing systems. pp 341–349
Hays J, Efros AA (2008) Scene completion using millions of photographs. Commun ACM 51(10):87–94
Dang TT, Beghdadi A, Larabi MC (2013) Visual coherence metric for evaluation of color image restoration. In: 2013 colour and visual computing symposium (CVCS). IEEE, pp 1–6
Ardis PA, Singhal A (2009) Visual salience metrics for image inpainting. In: Visual communications and image processing 2009, vol 7257. W. International Society for Optics and Photonics, p 72571
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Kadian, G., Khadanga, G. (2020). Review of Inpainting Techniques for UAV Images. In: Jain, K., Khoshelham, K., Zhu, X., Tiwari, A. (eds) Proceedings of UASG 2019. UASG 2019. Lecture Notes in Civil Engineering, vol 51. Springer, Cham. https://doi.org/10.1007/978-3-030-37393-1_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-37393-1_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-37392-4
Online ISBN: 978-3-030-37393-1
eBook Packages: EngineeringEngineering (R0)