Non-Local Kernel Regression for Image and Video Restoration

  • Haichao Zhang
  • Jianchao Yang
  • Yanning Zhang
  • Thomas S. Huang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6313)


This paper presents a non-local kernel regression (NL-KR) method for image and video restoration tasks, which exploits both the non-local self-similarity and local structural regularity in natural images. The non-local self-similarity is based on the observation that image patches tend to repeat themselves in natural images and videos; and the local structural regularity reveals that image patches have regular structures where accurate estimation of pixel values via regression is possible. Explicitly unifying both properties, the proposed non-local kernel regression framework is robust and applicable to various image and video restoration tasks. In this work, we are specifically interested in applying the NL-KR model to image and video super-resolution (SR) reconstruction. Extensive experimental results on both single images and realistic video sequences demonstrate the superiority of the proposed framework for SR tasks over previous works both qualitatively and quantitatively.


Near Neighbor Image Patch Kernel Regression Bicubic Interpolation Similar Patch 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Supplementary material

978-3-642-15558-1_41_MOESM1_ESM.rar (18.9 mb)
Electronic Supplementary Material (19,397 KB)


  1. 1.
    Tschumperle, D.: PDE’s Based Regularization of Multivalued Images and Applications. PhD thesis (2002)Google Scholar
  2. 2.
    Elad, M., Aharon, M.: Image denoising via learned dictionaries and sparse representation. In: CVPR, pp. 17–22 (2006)Google Scholar
  3. 3.
    Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution as sparse representation of raw image patches. In: CVPR (2008)Google Scholar
  4. 4.
    Chang, H., Yeung, D.Y., Xiong, Y.: Super-resolution through neighbor embedding. In: CVPR (2004)Google Scholar
  5. 5.
    Li, X.: Video processing via implicit and mixture motion models. IEEE Trans. on Circuits and Systems for Video Technology 17, 953–963 (2007)CrossRefGoogle Scholar
  6. 6.
    Takeda, H., Farsiu, S., Milanfar, P.: Kernel regression for image processing and reconstruction. IEEE TIP 16, 349–366 (2007)MathSciNetGoogle Scholar
  7. 7.
    Tomasi, C.: Bilateral filtering for gray and color images. In: ICCV, pp. 839–846 (1998)Google Scholar
  8. 8.
    Buades, A., Coll, B.: A non-local algorithm for image denoising. In: CVPR (2005)Google Scholar
  9. 9.
    Protter, M., Elad, M., Takeda, H., Milanfar, P.: Generalizing the non-local-means to super-resolution reconstruction. IEEE TIP, 36–51 (2009)Google Scholar
  10. 10.
    Glasner, D., Bagon, S., Irani, M.: Super-resolution from a single image. In: ICCV (2009)Google Scholar
  11. 11.
    Peyre, G., Bougleux, S., Cohen, L.: Non-local regularization of inverse problems. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 57–68. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  12. 12.
    Efros, A., Leung, T.: Texture synthesis by non-parametric sampling. In: ICCV, pp. 1033–1038 (1999)Google Scholar
  13. 13.
    Wong, A., Orchard, J.: A nonlocal-means approach to exemplar-based inpainting. In: ICIP (2002)Google Scholar
  14. 14.
    Protter, M., Elad, M.: Super resolution with probabilistic motion estimation. IEEE TIP, 1899–1904 (2009)Google Scholar
  15. 15.
    Chatterjee, P., Milanfar, P.: A generalization of non-local means via kernel regression. In: SPIE Conf. on Computational Imaging (2008)Google Scholar
  16. 16.
    Takeda, H., Milanfar, P., Protter, M., Elad, M.: Super-resolution without explicit subpixel motion estimation. IEEE TIP (2009)Google Scholar
  17. 17.
    Mairal, J., Bach, F., Ponce, J., Sapiro, G., Zisserman, A.: Non-local sparse models for image restoration. In: ICCV (2009)Google Scholar
  18. 18.
  19. 19.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Member, S., Simoncelli, E.P., Member, S.: Image quality assessment: From error visibility to structural similarity. IEEE TIP 13, 600–612 (2004)Google Scholar
  20. 20.
    Fattal, R.: Image upsampling via imposed edge statistics. In: SIGGRAPH (2007)Google Scholar
  21. 21.
    Kim, K.I., Kwon, Y.: Example-based learning for single-image super-resolution and jpeg artifact removal. Technical report (2008)Google Scholar
  22. 22.
    Danielyan, A., Foi, A., Katkovnik, V., Egiazarian, K.: Image and video super-resolution via spatially adaptive block-matching filtering. In: Int. Workshop on Local and Non-Local Approx. in Image Process (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Haichao Zhang
    • 1
    • 2
  • Jianchao Yang
    • 2
  • Yanning Zhang
    • 1
  • Thomas S. Huang
    • 2
  1. 1.School of Computer ScienceNorthwestern Polytechnical UniversityXi’anChina
  2. 2.Beckman InstituteUniversity of Illinois at Urbana-ChampaignUSA

Personalised recommendations