Abstract
A nonlocal variational formulation for interpolating a sparsely sampled image is introduced in this paper. The proposed variational formulation, originally motivated by image inpainting problems, encourages the transfer of information between similar image patches, following the paradigm of exemplar-based methods. Contrary to the classical inpainting problem, no complete patches are available from the sparse image samples, and the patch similarity criterion has to be redefined as here proposed. Initial experimental results with the proposed framework, at very low sampling densities, are very encouraging. We also explore some departures from the variational setting, showing a remarkable ability to recover textures at low sampling densities.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bertalmío, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proc. of SIGGRAPH (2000)
Bertalmío, M., Bertozzi, A., Sapiro, G.: Navier-stokes, fluid-dynamics and image and video inpainting. In: Proc. of the IEEE Conf. on CVPR (2001)
Masnou, S., Morel, J.M.: Level lines based disocclusion. In: Proc. of IEEE ICIP (1998)
Efros, A., Leung, T.: Texture synthesis by non-parametric sampling. In: Proc. of the IEEE Conf. on CVPR, pp. 1033–1038 (1999)
Levin, A., Zomet, A., Weiss, Y.: Learning how to inpaint from global image statistics. In: Proc. of ICCV (2003)
Criminisi, A., Pérez, P., Toyama, K.: Region filling and object removal by exemplar-based inpainting. IEEE Trans. on IP 13(9), 1200–1212 (2004)
Sun, J., Yuan, L., Jia, J., Shum, H.Y.: Image completion with structure propagation. In: Proc. of SIGGRAPH (2005)
Bertalmío, M., Vese, L., Sapiro, G., Osher, S.: Simultaneous structure and texture inpainting. IEEE Trans. on Image Processing 12(8), 882–889 (2003)
Cao, F., Gousseau, Y., Masnou, S., Pérez, P.: Geometrically-guided exemplar-based inpainting (submitted, 2008)
Gröchenig, K., Strohmer, T.: Numerical and theoretical aspects of non-uniform sampling of band-limited images. In: Marvasti, F. (ed.) Theory and Practice of Nonuniform Sampling. Kluwer/Plenum (2000)
Chan, T., Shen, J.H.: Mathematical models for local nontexture inpaintings. SIAM J. App. Math. 62(3), 1019–1043 (2001)
Arigovindan, M., Suhling, M., Hunziker, P., Unser, M.: Variational image reconstruction from arbitrarily spaced samples: a fast multiresolution spline solution. IEEE Trans. on IP 14(4), 450–460 (2005)
Shepard, D.: A two-dimensional interpolation function for irregularly-spaced data. In: Proc. of 23rd ACM national conf., pp. 517–524. ACM Press, New York (1968)
Candes, E.J., Wakin, M.B.: An introduction to compressive sampling. IEEE Signal Processing Magazine 25(2), 21–30 (2008)
Aharon, M., Elad, M., Bruckstein, A.: The K-SVD: An algorithm for designing of overcomplete dictionaries for sparse representatio. IEEE Trans. on Signal Processing 54(11), 4311–4322 (2006)
Mairal, J., Sapiro, G., Elad, M.: Learning multiscale sparse representations for image and video restoration. SIAM MMS 7(1), 214–241 (2008)
Elad, M., Starck, J., Querre, P., Donoho, D.: Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA). Applied and Computational Harmonic Analysis 19(3), 340–358 (2005)
Hays, J., Efros, A.: Scene completion using millions of photographs (2008)
Peyré, G., Bougleux, S., Cohen, L.: Non-local regularization of inverse problems. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 57–68. Springer, Heidelberg (2008)
Lezoray, O., Elmoataz, A., Bougleux, S.: Graph regularization for color image processing. Comput. Vis. Image Underst. 107(1-2), 38–55 (2007)
Gilboa, G., Osher, S.: Nonlocal linear image regularization and supervised segmentation. SIAM Mult. Mod. and Sim. 6(2), 595–630 (2007)
Belhachmi, Z., Bucur, D., Burgeth, B., Weickert, J.: How to choose interpolation data in images. Technical Report No. 205, Department of Mathematics, Saarland University, Saarbrücken, Germany (2008)
Arias, P., Caselles, V., Sappiro, G.: A variational framework for non-local image inpainting. In: Proc. of EMMCVPR. Springer, Heidelberg (2009)
Buades, A., Coll, B., Morel, J., Sbert, C.: Self similarity driven color demosaicing. IEEE TIP 18(6), 1192–1202 (2009)
Mairal, J., Bach, F., Ponce, J., Sapiro, G., Zisserman, A.: Non-local sparse models for image restoration. In: Proc. of ICCV (2009)
Protter, M., Elad, M., Takeda, H., Milanfar, P.: Generalizing the non-local-means to super-resolution reconstruction. IEEE Trans. on IP 18(1), 36–51 (2009)
Bartesaghi, A., Sprechmann, P., Liu, J., Randall, G., Sapiro, G., Subramaniam, S.: Classification and 3d averaging with missing wedge correction in biological electron tomography? Journal of Structural Biology 162(3), 436–450 (2008)
Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. CoRR abs/0805.4471 (2008)
Awate, S., Whitaker, R.: Unsupervised, information-theoretic, adaptive image filtering for image restoration. IEEE Trans. on PAMI 28(3), 364–376 (2006)
Brox, T., Kleinschmidt, O., Cremers, D.: Efficient nonlocal means for denoising of textural patterns. IEEE Trans. on IP 17(7), 1057–1092 (2008)
Kervrann, C., Boulanger, J.: Optimal spatial adaptation for patch-based image denoising. IEEE Trans. on IP 15(10), 2866–2878 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Facciolo, G., Arias, P., Caselles, V., Sapiro, G. (2009). Exemplar-Based Interpolation of Sparsely Sampled Images. In: Cremers, D., Boykov, Y., Blake, A., Schmidt, F.R. (eds) Energy Minimization Methods in Computer Vision and Pattern Recognition. EMMCVPR 2009. Lecture Notes in Computer Science, vol 5681. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03641-5_25
Download citation
DOI: https://doi.org/10.1007/978-3-642-03641-5_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03640-8
Online ISBN: 978-3-642-03641-5
eBook Packages: Computer ScienceComputer Science (R0)