Advertisement

Deep Network Cascade for Image Super-resolution

  • Zhen Cui
  • Hong Chang
  • Shiguang Shan
  • Bineng Zhong
  • Xilin Chen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8693)

Abstract

In this paper, we propose a new model called deep network cascade (DNC) to gradually upscale low-resolution images layer by layer, each layer with a small scale factor. DNC is a cascade of multiple stacked collaborative local auto-encoders. In each layer of the cascade, non-local self-similarity search is first performed to enhance high-frequency texture details of the partitioned patches in the input image. The enhanced image patches are then input into a collaborative local auto-encoder (CLA) to suppress the noises as well as collaborate the compatibility of the overlapping patches. By closing the loop on non-local self-similarity search and CLA in a cascade layer, we can refine the super-resolution result, which is further fed into next layer until the required image scale. Experiments on image super-resolution demonstrate that the proposed DNC can gradually upscale a low-resolution image with the increase of network layers and achieve more promising results in visual quality as well as quantitative performance.

Keywords

Super-resolution Auto-encoder Deep learning 

References

  1. 1.
    Baker, S., Kanade, T.: Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(9), 1167–1183 (2002)CrossRefGoogle Scholar
  2. 2.
    Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H.: Greedy layer-wise training of deep networks. In: Advances in Neural Information Processing Systems (NIPS), vol. 19, p. 153 (2007)Google Scholar
  3. 3.
    Buades, A., Coll, B., Morel, J.-M.: A non-local algorithm for image denoising. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2005)Google Scholar
  4. 4.
    Chang, H., Yeung, D.-Y., Xiong, Y.: Super-resolution through neighbor embedding. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, p. I–275 (2004)Google Scholar
  5. 5.
    Dong, W., Zhang, L., Shi, G.: Centralized sparse representation for image restoration. In: IEEE International Conference on Computer Vision (ICCV), pp. 1259–1266 (2011)Google Scholar
  6. 6.
    Elad, M., Aharon, M.: Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image Processing 15(12), 3736–3745 (2006)CrossRefMathSciNetGoogle Scholar
  7. 7.
    Farsiu, S., Robinson, M.D., Elad, M., Milanfar, P.: Fast and robust multiframe super resolution. IEEE Transactions on Image Processing 13(10), 1327–1344 (2004)CrossRefGoogle Scholar
  8. 8.
    Freedman, G., Fattal, R.: Image and video upscaling from local self-examples. ACM Transactions on Graphics 30(2), 12 (2011)CrossRefGoogle Scholar
  9. 9.
    Freeman, W.T., Jones, T.R., Pasztor, E.C.: Example-based super-resolution. IEEE Computer Graphics and Applications 22(2), 56–65 (2002)CrossRefGoogle Scholar
  10. 10.
    Glasner, D., Bagon, S., Irani, M.: Super-resolution from a single image. In: IEEE International Conference on Computer Vision (ICCV), pp. 349–356 (2009)Google Scholar
  11. 11.
    Gunturk, B.K., Batur, A.U., Altunbasak, Y., Hayes III, M.H., Mersereau, R.M.: Eigenface-domain super-resolution for face recognition. IEEE Transactions on Image Processing 12(5), 597–606 (2003)CrossRefGoogle Scholar
  12. 12.
    Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Computation 18(7), 1527–1554 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  13. 13.
    Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  14. 14.
    Irani, M., Peleg, S.: Improving resolution by image registration. Graphical Models and Image Processing 53(3), 231–239 (1991)CrossRefGoogle Scholar
  15. 15.
    Kim, K.I., Kwon, Y.: Single-image super-resolution using sparse regression and natural image prior. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(6), 1127–1133 (2010)CrossRefMathSciNetGoogle Scholar
  16. 16.
    Le, Q.V., Ngiam, J., Coates, A., Lahiri, A., Prochnow, B., Ng, A.Y.: On optimization methods for deep learning. In: International Conference on Machine Learning, ICML (2011)Google Scholar
  17. 17.
    Lee, H., Battle, A., Raina, R., Ng, A.Y.: Efficient sparse coding algorithms. In: Advances in neural information processing systems (NIPS), vol. 19, p. 801 (2007)Google Scholar
  18. 18.
    Lee, H., Ekanadham, C., Ng, A.: Sparse deep belief net model for visual area v2. In: Advances in neural information processing systems (NIPS), vol. 20, pp. 873–880 (2008)Google Scholar
  19. 19.
    Lee, H., Grosse, R., Ranganath, R., Ng, A.Y.: Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: International Conference on Machine learning (ICML), pp. 609–616 (2009)Google Scholar
  20. 20.
    Lin, Z., He, J., Tang, X., Tang, C.-K.: Limits of learning-based superresolution algorithms. International Journal of Computer Vision 80(3), 406–420 (2008)CrossRefGoogle Scholar
  21. 21.
    Lin, Z., Shum, H.-Y.: Fundamental limits of reconstruction-based superresolution algorithms under local translation. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(1), 83–97 (2004)CrossRefGoogle Scholar
  22. 22.
    Lu, X., Yuan, H., Yan, P., Yuan, Y., Li, X.: Geometry constrained sparse coding for single image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1648–1655 (2012)Google Scholar
  23. 23.
    Ranzato, M., Boureau, Y.L., LeCun, Y.: Sparse feature learning for deep belief networks. In: Advances in neural information processing systems (NIPS), vol. 20, pp. 1185–1192 (2007)Google Scholar
  24. 24.
    Ngiam, J., Koh, P.W., Chen, Z., Bhaskar, S., Ng, A.Y.: Sparse filtering. In: Advances in Neural Information Processing Systems (NIPS), vol. 24, pp. 1125–1133 (2011)Google Scholar
  25. 25.
    Nguyen, K., Sridharan, S., Denman, S., Fookes, C.: Feature-domain super-resolution framework for gabor-based face and iris recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2642–2649 (2012)Google Scholar
  26. 26.
    Park, S.C., Park, M.K., Kang, M.G.: Super-resolution image reconstruction: a technical overview. Signal Processing Magazine 20(3), 21–36 (2003)CrossRefGoogle Scholar
  27. 27.
    Tipping, M.E., Bishop, C.M.: Bayesian image super-resolution. In: Advances in Neural Information Processing Systems (NIPS), vol. 15, pp. 1279–1286 (2002)Google Scholar
  28. 28.
    Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.-A.: Extracting and composing robust features with denoising autoencoders. In: International Conference on Machine learning (ICML), pp. 1096–1103. ACM (2008)Google Scholar
  29. 29.
    Wang, W., Cui, Z., Chang, H., Shan, S., Chen, X.: Deeply coupled auto-encoder networks for cross-view classification. arXiv preprint arXiv:1402.2031 (2014)Google Scholar
  30. 30.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error measurement to structural similarity. IEEE Transactions on Image Processing 13(4), 600–612 (2004)CrossRefGoogle Scholar
  31. 31.
    Xie, J., Xu, L., Chen, E.: Image denoising and inpainting with deep neural networks. In: Advances in Neural Information Processing Systems (NIPS), pp. 350–358 (2012)Google Scholar
  32. 32.
    Yang, J., Lin, Z., Cohen, S.: Fast image super-resolution based on in-place example regression. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2013)Google Scholar
  33. 33.
    Yang, J., Wright, J., Huang, T., Ma, Y.: Image super-resolution as sparse representation of raw image patches. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8 (2008)Google Scholar
  34. 34.
    Zhang, K., Gao, X., Tao, D., Li, X.: Multi-scale dictionary for single image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1114–1121 (2012)Google Scholar
  35. 35.
    Zhang, L., Wu, X.: An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Transactions on Image Processing 15(8), 2226–2238 (2006)CrossRefGoogle Scholar
  36. 36.
    Zontak, M., Irani, M.: Internal statistics of a single natural image. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2011)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Zhen Cui
    • 1
    • 2
  • Hong Chang
    • 1
  • Shiguang Shan
    • 1
  • Bineng Zhong
    • 2
  • Xilin Chen
    • 1
  1. 1.Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS)Institute of Computing Technology, CASBeijingChina
  2. 2.School of Computer Science and TechnologyHuaqiao UniversityXiamenChina

Personalised recommendations