MMSR: A Multi-model Super Resolution Framework

  • Ninghui Yuan
  • Zhihao Zhu
  • Xinzhou Wu
  • Li ShenEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11783)


Single image super-resolution (SISR), as an important image processing method, has received great attentions from both industry and academia. Currently, most super-resolution image reconstruction approaches are based on the deep-learning techniques and they usually focus on the design and optimization of different network models. But they usually ignore the differences among image texture features and use the same model to train all the input images, which greatly influence the training efficiency. In this paper, we try to build a framework to improve the training efficiency through specifying an appropriate model for each type of images according to their texture characteristics, and we propose MMSR, a multi-model super resolution framework. In this framework, all input images are classified by an approach called TVAT (Total Variance above the Threshold). Experimental results indicate that our MMSR framework brings a 66.7% performance speedup on average without influencing the accuracy of the results HR images. Moreover, MMSR framework exhibits good scalability.


Super resolution Multi-model General framework Classification 


  1. 1.
    Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Cham (2014). Scholar
  2. 2.
    Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 391–407. Springer, Cham (2016). Scholar
  3. 3.
    Shi, W., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: CVPR (2016)Google Scholar
  4. 4.
    Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) 3(6), 8 (2016)Google Scholar
  5. 5.
    Mao, X.-J., Shen, C., Yang, Y.-B.: Image restoration using convolutional auto-encoders with symmetric skip connections. In: The Annual Conference on Neural Information Processing Systems (NIPS), August 2016Google Scholar
  6. 6.
    Tai, Y., Yang, J., Liu, X.: Image super-resolution via deep recursive residual network. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)Google Scholar
  7. 7.
    Tong, T., Li, G., Liu, X., Gao, Q.: Image super-resolution using dense skip connections. In: IEEE International Conference on Computer Vision (ICCV) (2017)Google Scholar
  8. 8.
    Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)Google Scholar
  9. 9.
    Wang, X., Yu, K., Dong, C., Change Loy, C.: Recovering realistic texture in image super-resolution by deep spatial feature transform. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)Google Scholar
  10. 10.
    The Pirm Challenge on Perceptual Super Resolution.
  11. 11.
    Blau, Y., Michaeli, T.: The Perception-distortion tradeoff. In: ECCV 2018Google Scholar
  12. 12.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P: Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRefGoogle Scholar
  13. 13.
    Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems (NIPS), pp. 2672–2680, March 2014Google Scholar
  14. 14.
    Yegulalp, S.: Facebook brings GPU-powered machine learning to Python. InfoWorld, 19 January 2017Google Scholar
  15. 15.
    Lorica, B.: Why AI and machine learning researchers are beginning to embrace PyTorch. O’Reilly Media, 3 August 2017Google Scholar
  16. 16.
    Ketkar, N.: Deep Learning with Python, pp. 195–208. Apress, Berkeley (2017)CrossRefGoogle Scholar

Copyright information

© IFIP International Federation for Information Processing 2019

Authors and Affiliations

  1. 1.School of ComputerNational University of Defense TechnologyChangshaChina

Personalised recommendations