Video Super Resolution Using Non-Local Means with Adaptive Decaying Factor and Searching Window

  • Yawei LiEmail author
  • Xiaofeng Li
  • Cui Yao
  • Zhizhong Fu
  • Xiuxia Yin
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10116)


Power consumption, transmission bandwidth, and spatial-temporal resolution tradeoff are among the most important factors that affect video processing on mobile devices. Scalable video coding (SVC) provides a possible solution to overcome these problems. Every video in SVC format consists of high-resolution (HR) frames and low-resolution (LR) frames. For a better viewing experience, super resolution (SR) is introduced to refine the LR frames. Non-local means (NLM) based SR has a promising prospect. Since NLM originates from image denoising, the fixed parameters of NLM is not fit for SR tasks. A fixed decaying factor tends to blur the details in flat regions and a fixed searching window results in mismatches among pixels. Thus, we propose two adaptive parameters to address these problems. We generalize key features that affect SR methods’ applicability of implementation on hardware and show NLM is fit for hardware implementation. The experimental results validate the proposed algorithm.


Video Sequence Image Denoising Scalable Video Code Decay Factor Super Resolution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was supported by the Natural Science Foundation of China (61671126).


  1. 1.
    Healy, D., Mitchell, O.R.: Digital video bandwidth compression using block truncation coding. IEEE Trans. Commun. 29, 1809–1817 (1982)CrossRefGoogle Scholar
  2. 2.
    Banerjee, S.: Low-power content-based video acquisition for super-resolution enhancement. IEEE Trans. Multimedia 11, 455–464 (2009)CrossRefGoogle Scholar
  3. 3.
    Shen, M., Xue, P.: Low-power video acquisition with super-resolution reconstruction for mobile devices. IEEE Trans. Consum. Electron. 56, 2520–2528 (2010)CrossRefGoogle Scholar
  4. 4.
    Ben-Ezra, M., Nayar, S.K.: Motion deblurring using hybrid imaging. In: Proceedings of the CVPR, pp. 657–664 (2003)Google Scholar
  5. 5.
    Tai, Y., Du, H., Brown, M., Lin, S.: Image/video deblurring using a hybrid camera. In: Proceedings of the CVPR, pp. 1–8 (2008)Google Scholar
  6. 6.
    Tai, Y., Du, H., Brown, M., Lin, S.: Correction of spatially varying image and video motion blur using a hybrid camera. IEEE Trans. Pattern Anal. Mach. Intell 32, 1012–1028 (2010)CrossRefGoogle Scholar
  7. 7.
    Brandi, F., de Queiroz, R., Mukherjee, D.: Super-resolution of video using key frames and motion estimation. In: Proceedings of the ICIP, pp. 321–324 (2010)Google Scholar
  8. 8.
    Zhang, L., Wu, X.: An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Trans. Image Process. 15, 2226–2238 (2006)CrossRefGoogle Scholar
  9. 9.
    Allebach, J.P., Wong, P.W.: Edge-directed interpolation. In: Proceedings of the ICIP, pp. 707–710 (1996)Google Scholar
  10. 10.
    Li, X., Orchard, M.T.: New edge-directed interpolation. IEEE Trans. Image Process. 10, 1521–1527 (2001)CrossRefGoogle Scholar
  11. 11.
    Wang, Q., Ward, R.K.: A new orientation-adaptive interpolation method. IEEE Trans. Image Process. 16, 889–900 (2007)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Tsai, R.Y., Huang, T.S.: Multiple frame restoration and registration. In: Advances in Computer Vision and Image Processing, pp. 317–339 (1984)Google Scholar
  13. 13.
    Capel, D., Zisserman, A.: Computer vision applied to super-resolution. IEEE Sig. Process. Mag. 20, 75–86 (2003)CrossRefGoogle Scholar
  14. 14.
    Park, S.C., Park, M.K., Kang, M.G.: Super-resolution image reconstruction: a technical overview. IEEE Sig. Process. Mag. 20, 21–36 (2003)CrossRefGoogle Scholar
  15. 15.
    Farsiu, S., Robinson, M.D., Elad, M., Milanfar, P.: Fast and robust multiframe super resolution. IEEE Trans. Image Process. 13, 1327–1344 (2004)CrossRefGoogle Scholar
  16. 16.
    Fu, Z., Li, Z., Ding, L., Nguyen, T.: Translation invariance-based super resolution method for mixed resolution multiview video. In: Proceedings of the ICIP, pp. 5457–5461 (2014)Google Scholar
  17. 17.
    Ancuti, C., Ancuti, C.O., Bekaert, P.: Video super-resolution using high quality photographs. In: Proceedings of the ICASSP, pp. 862–865 (2010)Google Scholar
  18. 18.
    Capel, D., Zisserman, A.: Super-resolution from multiple views using learnt image models. In: Proceedings of the CVPR, pp. 627–634 (2001)Google Scholar
  19. 19.
    Freeman, W.T., Jones, T.R., Pasztor, E.C.: Example-based super-resolution. IEEE Comput. Graph. Appl. 22, 56–65 (2002)CrossRefGoogle Scholar
  20. 20.
    Chang, H., Yeung, D.Y., Xiong, Y.: Super-resolution through neighbor embedding. In: Proceedings of the CVPR, pp. 275–282 (2004)Google Scholar
  21. 21.
    Buades, A., Coll, B., Morel, J.M.: Nonlocal image and movie denoising. Int. J. Comput. Vis. 76, 123–139 (2008)CrossRefGoogle Scholar
  22. 22.
    Protter, M., Elad, M., Takeda, H., Milanfar, P.: Generalizing the nonlocal-means to super-resolution reconstruction. IEEE Trans. Image Process 18, 36–51 (2009)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Basavaraja, S.V., Bopardikar, A.S., Velusamy, S.: Detail warping based video super-resolution using image guides. In: Proceedings of the ICIP, pp. 2009–2012 (2010)Google Scholar
  24. 24.
    Lengyel, R., Soroushmehr, S.M.R., Shirani, S.: Multi-view video super-resolution for hybrid cameras using modified NLM and adaptive thresholding. In: Proceedings of the ICIP, pp. 5437–5441 (2014)Google Scholar
  25. 25.
    Wong, A., Fieguth, P., Clausi, D.: A perceptually adaptive approach to image denoising using anisotropic non-local means. In: Proceedings of the ICIP, pp. 537–540 (2008)Google Scholar
  26. 26.
    Amanatiadis, A., Andreadis, I., Konstantinidis, K.: Design and implementation of a fuzzy area-based image-scaling technique. IEEE Trans. Instrum. Meas. 57, 1504–1513 (2008)CrossRefGoogle Scholar
  27. 27.
    Lopez, S., Callico, G.M., Tobajas, F., Lopez, J.F., Sarmiento, R.: A novel real-time DSP-based video super-resolution system. IEEE Trans. Consum. Electron. 55, 2264–2270 (2009)CrossRefGoogle Scholar
  28. 28.
    Singhal, N., Park, I.K., Cho, S.: Implementation and optimization of image processing algorithms on handheld gpu. In: Proceedings of the ICIP, pp. 4481–4484 (2010)Google Scholar
  29. 29.
    Szydzik, T., Callico, G.M., Nunez, A.: Efficient FPGA implementation of a high-quality super-resolution algorithm with real-time performance. IEEE Trans. Consum. Electron. 57, 664–672 (2011)CrossRefGoogle Scholar
  30. 30.
    Wang, C., Chan, S.C.: A new bandwidth adaptive non-local kernel regression algorithm for image/video restoration and its GPU realization. In: Proceedings of the ISCAS, pp. 1388–1391 (2013)Google Scholar
  31. 31.
    Duchon, C.E.: Lanczos filtering in one and two dimensions. J. Appl. Meteorol. 18, 1016–1022 (1979)CrossRefGoogle Scholar
  32. 32.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Yawei Li
    • 1
    Email author
  • Xiaofeng Li
    • 1
  • Cui Yao
    • 1
  • Zhizhong Fu
    • 1
  • Xiuxia Yin
    • 1
  1. 1.University of Electronic Science and Technology of ChinaChengduChina

Personalised recommendations