Multi-frame image super-resolution reconstruction via low-rank fusion combined with sparse coding

  • Xuan ZhuEmail author
  • Peng Jin
  • XianXian Wang
  • Na Ai


The sparse coding method has been successfully applied to multi-frame super-resolution in recent years. In this paper, we propose a new multi-frame super-resolution framework which combines low-rank fusion with sparse coding to improve the performance of multi-frame super-resolution. The proposed method gets the high-resolution image by a three-stage process. First, a fused low-resolution image is obtained from multi-frame image by the method of registration and low-rank fusion. Then, we use the jointly training method to train a pair of learning dictionaries which have good adaptive ability. Finally, we use the learning dictionaries combined with sparse coding theory to realize super-resolution reconstruction of the fused low-resolution image. As the experiment results show, this method can recover the lost high frequency information, and has good robustness.


Super-resolution reconstruction Multi-frame image Low-rank fusion Sparse coding 



  1. 1.
    Dai Q, Yoo S, Kappeler A, Katsaggelos AK (2016) Sparse representation based multiple frame video super-resolution. IEEE Trans Image Proc (99):1–1Google Scholar
  2. 2.
    Dong W, Zhang L, Shi G, Li X (2013) Nonlocally centralized sparse representation for image restoration. IEEE Trans Image Proc: Publ IEEE Sign Proc Soc 22(4):1620MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Gao X, Zhang K, Tao D, Li X (2012) Image super-resolution with sparse neighbor embedding. IEEE Trans Image Proc Publ IEEE Sign Proc Soc 21(7):3194–3205MathSciNetzbMATHGoogle Scholar
  4. 4.
    Gu S, Xie Q, Meng D, Zuo W, Feng X, Zhang L (2017) Weighted nuclear norm minimization and its applications to low level vision. Int J Comput Vis 121(2):183–208CrossRefGoogle Scholar
  5. 5.
    Irani M, Peleg S (1991) Improving resolution by image registration. Cvgip Graph Models Image Proc 53(91):231–239CrossRefGoogle Scholar
  6. 6.
    Jakhetiya V, Kumar A, Tiwari AK (2010) Image interpolation by adaptive 2-D autoregressive modeling. 7546Google Scholar
  7. 7.
    Kato T, Hino H, Murata N (2015) Multi-frame image super resolution based on sparse coding. Neural Netw 66(C):64–78CrossRefGoogle Scholar
  8. 8.
    Kato T, Hino H, Murata N (2017) Double sparsity for multi-frame super resolution. Neurocomputing 240:115–126CrossRefGoogle Scholar
  9. 9.
    Lazarov AD (2001) Iterative MMSE method and recurrent Kalman procedure for ISAR image reconstruction. Aerospace Electron Syst IEEE Trans 37(4):1432–1441CrossRefGoogle Scholar
  10. 10.
    Li Z, Wang L, Yu T, Hu BL (2014) Image super-resolution via low-rank representation. Appl Mech Mater 568-570:652–655CrossRefGoogle Scholar
  11. 11.
    Park SC, Park MK, Kang MG (2003) Super-resolution image reconstruction: a technical overview. Sign Proc Mag IEEE 20(4):21–36CrossRefGoogle Scholar
  12. 12.
    Schultz RR, Stevenson RL (1994) A Bayesian approach to image expansion for improved definition. Image Proc IEEE Tran 3(3):233–242CrossRefGoogle Scholar
  13. 13.
    Stark H, Oskoui P (1989) High-resolution image recovery from image-plane arrays, using convex projections. J Opt Soc Am Opt Image Sci 6(11):1715–1726CrossRefGoogle Scholar
  14. 14.
    Timofte R, De V, Gool LV (2014) Anchored Neighborhood Regression for Fast Example-Based Super-Resolution. In: IEEE International Conference on Computer Vision 1920–1927Google Scholar
  15. 15.
    Wang H-p, Li-li Z, Jie Z (2010) Region-based Bicubic image interpolation algorithm. Comput Eng 36(19):216–218Google Scholar
  16. 16.
    Wright J, Ganesh A, Rao S, Ma Y (2009) Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices. Advances in Neural Information Processing Systems 87 (4):20:23–20:56Google Scholar
  17. 17.
    Yan HX, Liu YJ (2013) Research of adaptive gradient projection algorithm on remote sensing image reconstruction. Adv Mater Res 765-767(765–767):572–575CrossRefGoogle Scholar
  18. 18.
    Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Proc Publ IEEE Sign Proc Soc 19(11):2861–2873MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Yang B, Huang J, Yang Y (2015) Semantic Description and Information Retrieval Research of Surveillance Video in Smart Transportation SystemGoogle Scholar
  20. 20.
    Zajchowski R, Martin J (2015) Analytic image reconstruction from partial data for a single-scan cone-beam CT with scatter correction. Med Phys 42(11):6625–6640CrossRefGoogle Scholar
  21. 21.
    Zhang L, Wu X (2006) An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Trans Image Process 15(15):2226–2238CrossRefGoogle Scholar
  22. 22.
    Zhu X, Li B, Tao J (2015) Jiang B Super-resolution image reconstruction via patch haar wavelet feature extraction combined with sparse coding. In: IEEE International Conference on Information and AutomationGoogle Scholar
  23. 23.
    Zhu X, Wang X, Wang J, Jin P, Liu L, Mei D (2017) Image Super-Resolution Based on Sparse Representation via Direction and Edge Dictionaries. Mathematical Problems in Engineering (2017–6-28) 2017 (5):1–11Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Information Science and Technology of Northwest UniversityXi’anPeople’s Republic of China

Personalised recommendations