Skip to main content
SpringerLink
Log in
Book cover

International Conference on Multimedia Modeling

MMM 2020: MultiMedia Modeling pp 148–160Cite as

A Compact Deep Neural Network for Single Image Super-Resolution

  • Conference paper
  • First Online:

  • 2306 Accesses

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11962))

Abstract

Convolutional neural network (CNN) has recently been applied into single image super-resolution (SISR) task. But the applied CNN models are increasingly cumbersome which will cause heavy memory and computational burden when deploying in realistic applications. Besides, existing CNNs for SISR have trouble in handling different scales information with same kernel size. In this paper, we propose a compact deep neural network (CDNN) to (1) reduce the amount of model parameters (2) decrease computational operations and (3) process different scales information. We devise two kinds of channel-wise scoring units (CSU), including adaptive channel-wise scoring unit (ACSU) and constant channel-wise scoring unit (CCSU), which act as judges to score for different channels. With further sparsity regularization imposed on CSUs and ensuing pruning of low-score channels, we can achieve considerable storage saving and computation simplification. In addition, the CDNN contains a dense inception structure, the convolutional kernels of which are in different sizes. This enables the CDNN to cope with different scales information in one natural image. We demonstrate the effectiveness of CSUs, dense inception on benchmarks and the proposed CDNN has superior performance over other methods.

This work was supported in part by the National Natural Science Foundation of China under Grant 61871437 and in part by the Natural Science Foundation of Hubei Province of China under Grant 2019CFA022.

J. Qian—Equal contribution.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Agustsson, E., Timofte, R.: NTIRE 2017 challenge on single image super-resolution: dataset and study. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2017)

    Google Scholar 

  2. Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.L.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: BMVC (2012)

    Google Scholar 

  3. Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10593-2_13

    Chapter  Google Scholar 

  4. Chao, D., Chen, C.L., Tang, X.: Accelerating the super-resolution convolutional neural network. In: CVPR (2016)

    Google Scholar 

  5. Dahl, R., Norouzi, M., Shlens, J.: Pixel recursive super resolution. In: ICCV (2017)

    Google Scholar 

  6. Duchon, C.E.: Lanczos filtering in one and two dimensions. J. Appl. Meteorol. 18(8), 1016–1022 (1979)

    Article  Google Scholar 

  7. Fattal, R.: Image upsampling via imposed edge statistics. ACM Trans. Graph. 26(3), 95 (2007)

    Article  Google Scholar 

  8. Freeman, W.T., Jones, T.R., Pasztor, E.C.: Example-based super-resolution. Comput. Graph. Appl. IEEE 22(2), 56–65 (2002)

    Article  Google Scholar 

  9. Haichao, Z., Yanning, Z., Haisen, L., Huang, T.S.: Generative Bayesian image super resolution with natural image prior. IEEE Trans. Image Process. 21(9), 4054–4067 (2012)

    Article  MathSciNet  Google Scholar 

  10. Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. In: Advances in Neural Information Processing Systems, pp. 1135–1143 (2015)

    Google Scholar 

  11. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)

    Google Scholar 

  12. Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)

  13. Hu, X., Mu, H., Zhang, X., Wang, Z., Tan, T., Sun, J.: Meta-SR: a magnification-arbitrary network for super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1575–1584 (2019)

    Google Scholar 

  14. Huang, G., Liu, S., Laurens, V.D.M., Weinberger, K.Q.: CondenseNet: an efficient DenseNet using learned group convolutions (2017)

    Google Scholar 

  15. Huang, G., Liu, Z., Weinberger, K.Q.: Densely connected convolutional networks. CoRR abs/1608.06993 (2016). http://arxiv.org/abs/1608.06993

  16. Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: CVPR (2015)

    Google Scholar 

  17. Jianchao, Y., John, W., Thomas, H., Yi, M.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)

    Article  MathSciNet  Google Scholar 

  18. Jun, X.U., Liu, H., Yin, Y.: Medical image super-resolution reconstruction method based on non-local autoregressive learning. Pattern Recog. Artif. Intell. 30(8), 747–753 (2017)

    Google Scholar 

  19. Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: CVPR (2016)

    Google Scholar 

  20. Kwang In, K., Younghee, K.: Single-image super-resolution using sparse regression and natural image prior. IEEE Trans. Pattern Anal. Mach. Intell. 32(6), 1127 (2010)

    Article  Google Scholar 

  21. Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: ICCV (2017)

    Google Scholar 

  22. Li, S., He, F., Du, B., Zhang, L., Xu, Y., Tao, D.: Fast spatio-temporal residual network for video super-resolution. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019

    Google Scholar 

  23. Li, T., Xu, M., Yang, R., Tao, X.: A DenseNet based approach for multi-frame in-loop filter in HEVC. In: 2019 Data Compression Conference (DCC), pp. 270–279. IEEE (2019)

    Google Scholar 

  24. Li, X., Orchard, M.T.: New edge-directed interpolation. IEEE Trans. Image Process. 10(10), 1521–1527 (2001)

    Article  Google Scholar 

  25. Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M.: Enhanced deep residual networks for single image super-resolution. In: Computer Vision and Pattern Recognition Workshops (CVPRW) (2017)

    Google Scholar 

  26. Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., Zhang, C.: Learning efficient convolutional networks through network slimming (2017)

    Google Scholar 

  27. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV 2001 (2001)

    Google Scholar 

  28. Rastegari, M., Ordonez, V., Redmon, J., Farhadi, A.: XNOR-Net: ImageNet classification using binary convolutional neural networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 525–542. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_32

    Chapter  Google Scholar 

  29. Schmidt, M., Fung, G., Rosales, R.: Fast optimization methods for L1 regularization: a comparative study and two new approaches. In: Kok, J.N., Koronacki, J., Mantaras, R.L., Matwin, S., Mladenič, D., Skowron, A. (eds.) ECML 2007. LNCS (LNAI), vol. 4701, pp. 286–297. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74958-5_28

    Chapter  Google Scholar 

  30. Szegedy, C., et al.: Going deeper with convolutions. CoRR abs/1409.4842 (2014). http://arxiv.org/abs/1409.4842

  31. Tong, T., Li, G., Liu, X., Gao, Q.: Image super-resolution using dense skip connections. In: ICCV (2017)

    Google Scholar 

  32. Wang, X., Hou, C., Pu, L., Hou, Y.: A depth estimating method from a single image using FoE CRF. Multimedia Tools Appl. 74(21), 9491–9506 (2015)

    Article  Google Scholar 

  33. Yang, J., Wright, J., Huang, T.S., Yi, M.: Image super-resolution as sparse representation of raw image patches. In: CVPR (2008)

    Google Scholar 

  34. Yang, P., Zhang, N., Zhang, S., Yu, L., Zhang, J., Shen, X.: Content popularity prediction towards location-aware mobile edge caching. IEEE Trans. Multimed. 21(4), 915–929 (2019). https://doi.org/10.1109/TMM.2018.2870521

    Article  Google Scholar 

  35. Ye, J., Lu, X., Lin, Z., Wang, J.Z.: Rethinking the smaller-norm-less-informative assumption in channel pruning of convolution layers. arXiv preprint arXiv:1802.00124 (2018)

  36. Ying, T., Jian, Y., Liu, X., Xu, C.: MemNet: a persistent memory network for image restoration. In: ICCV (2017)

    Google Scholar 

  37. Yu, R., et al.: NISP: pruning networks using neuron importance score propagation (2017)

    Google Scholar 

  38. Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Boissonnat, J.-D., et al. (eds.) Curves and Surfaces 2010. LNCS, vol. 6920, pp. 711–730. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-27413-8_47

    Chapter  Google Scholar 

  39. Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: CVPR, June 2018

    Google Scholar 

  40. Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2472–2481 (2018)

    Google Scholar 

  41. Zou, W.W.W., Yuen, P.C.: Very low resolution face recognition problem. IEEE Trans. Image Process. Publ. IEEE Sig. Process. Soc. 21(1), 327–40 (2012)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Huazhong University of Science and Technology, Wuhan, China

    Xiaoyu Xu, Jian Qian, Li Yu, Shengju Yu,  HaoTao & Ran Zhu

Authors
  1. Xiaoyu Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jian Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Li Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shengju Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. HaoTao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ran Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li Yu .

Editor information

Editors and Affiliations

  1. Korea Advanced Institute of Science and Technology, Daejeon, Korea (Republic of)

    Yong Man Ro

  2. National Chiao Tung University, Hsinchu, Taiwan

    Wen-Huang Cheng

  3. Korea Advanced Institute of Science and Technology, Daejeon, Korea (Republic of)

    Junmo Kim

  4. National Cheng Kung University, Tainan City, Taiwan

    Wei-Ta Chu

  5. Tsinghua University, Beijing, China

    Peng Cui

  6. Korea Advanced Institute of Science and Technology, Daejeon, Korea (Republic of)

    Jung-Woo Choi

  7. National Tsing Hua University, Hsinchu, Taiwan

    Min-Chun Hu

  8. Ghent University, Ghent, Belgium

    Wesley De Neve

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 284 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xu, X., Qian, J., Yu, L., Yu, S., HaoTao, Zhu, R. (2020). A Compact Deep Neural Network for Single Image Super-Resolution. In: Ro, Y., et al. MultiMedia Modeling. MMM 2020. Lecture Notes in Computer Science(), vol 11962. Springer, Cham. https://doi.org/10.1007/978-3-030-37734-2_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-37734-2_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-37733-5

  • Online ISBN: 978-3-030-37734-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation