Contextual-Field Supported Iterative Representation for Face Hallucination

  • Kangli Zeng
  • Tao LuEmail author
  • Xiaolin LiEmail author
  • Yanduo Zhang
  • Li Peng
  • Shenming Qu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11336)


Face hallucination is a special super-resolution (SR) algorithm that enhances the resolution and quality of low-resolution (LR) facial image. For reconstructing finer high frequency information which are missing in image degradation, learning-based face SR methods rely on accurate prior information from training samples. In this paper, we propose a contextual-field supported iterative representation algorithm for face hallucination to discovery accurate prior. Different from traditional local-patch based methods, we use contextual-field supported sampling to replace local receptive field patch sampling for enriching prior information. Then, two weighted matrices are introduced to constrain reconstruction-errors term and representation-coefficients term simultaneously, one matrix ameliorates the heteroscedasticity of real data and the other one improves the stability of solution. Finally, we use iterative representation learning to iteratively update the supported dictionary pairs and their representation-coefficients to refine accurate high-frequency information. The experimental results show that the proposed approach outperforms some state-of-the-art face hallucination methods over FERET and CMU-MIT face databases using both subjective and objective evaluation indexes.


Face hallucination Iterative representation Contextual information Dictionary learning 


  1. 1.
    Baker, S., Kanade, T.: Hallucinating faces. In: Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition 2000, p. 83 (2002)Google Scholar
  2. 2.
    Chang, H., Yeung, D., Xiong, Y.: Super-resolution through neighbor embedding. Proc. Comput. Vis. Pattern Recogn. 1, I-275–I-282 (2004)Google Scholar
  3. 3.
    Cui, Z., Chang, H., Shan, S., Zhong, B., Chen, X.: Deep network cascade for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 49–64. Springer, Cham (2014). Scholar
  4. 4.
    Dong, C., Chen, C.L., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)CrossRefGoogle Scholar
  5. 5.
    Jiang, J., Hu, R., Han, Z., Lu, T., Huang, K.: Position-patch based face hallucination via locality-constrained representation. In: IEEE International Conference on Multimedia and Expo, pp. 212–217 (2012)Google Scholar
  6. 6.
    Jiang, J., Hu, R., Wang, Z., Han, Z., Ma, J.: Facial image hallucination through coupled-layer neighbor embedding. IEEE Trans. Circuits Syst. Video Technol. 26(9), 1674–1684 (2016)CrossRefGoogle Scholar
  7. 7.
    Jiang, J., Yu, Y., Tang, S., Ma, J., Qi, G.J., Aizawa, A.: Context-patch based face hallucination via thresholding locality-constrained representation and reproducing learning. In: IEEE International Conference on Multimedia and Expo, pp. 469–474 (2017)Google Scholar
  8. 8.
    Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks, pp. 1646–1654 (2015)Google Scholar
  9. 9.
    Liu, C., Shum, H.Y., Freeman, W.T.: Face Hallucination: Theory and Practice. Kluwer Academic Publishers, Dordrecht (2007)CrossRefGoogle Scholar
  10. 10.
    Lu, T., Guan, Y., Chen, D., Xiong, Z., He, W.: Low-rank constrained collaborative representation for robust face recognition. In: IEEE International Workshop on Multimedia Signal Processing, pp. 1–7 (2017)Google Scholar
  11. 11.
    Lu, T., Xiong, Z., Zhang, Y., Wang, B., Lu, T.: Robust face super-resolution via locality-constrained low-rank representation. IEEE Access 5(99), 13103–13117 (2017)CrossRefGoogle Scholar
  12. 12.
    Ma, X., Huang, H., Wang, S., Qi, C.: A simple approach to multiview face hallucination. IEEE Signal Process. Lett. 17(6), 579–582 (2010)CrossRefGoogle Scholar
  13. 13.
    Ma, X., Zhang, J., Qi, C.: Hallucinating face by position-patch. Pattern Recogn. 43(6), 2224–2236 (2010)CrossRefGoogle Scholar
  14. 14.
    Phillips, P., Moon, H., Rizvi, S., Rauss, P.: The feret evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 22(10), 1090–1104 (2000)CrossRefGoogle Scholar
  15. 15.
    Romano, Y., Elad, M.: Con-patch: when a patch meets its context. IEEE Trans. Image Process. Publ. IEEE Signal Process. Soc. 25(9), 3967–3978 (2016)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Romano, Y., Isidoro, J., Milanfar, P.: RAISR: rapid and accurate image super resolution. IEEE Trans. Comput. Imaging 3(1), 110–125 (2017). Scholar
  17. 17.
    Rowleys, H.: Neural network-based face detection. IEEE Trans. Pattern Anal. Mach. Intell. 20(1), 23–38 (1998)CrossRefGoogle Scholar
  18. 18.
    Shi, J., Liu, X., Zong, Y., Qi, C., Zhao, G.: Hallucinating face image by regularization models in high-resolution feature space. IEEE Trans. Image Process. PP(99), 1 (2018)Google Scholar
  19. 19.
    Timofte, R., De, V., Gool, L.V.: Anchored neighborhood regression for fast example-based super-resolution. In: IEEE International Conference on Computer Vision, pp. 1920–1927 (2013)Google Scholar
  20. 20.
    Timofte, R., De Smet, V., Van Gool, L.: A+: adjusted anchored neighborhood regression for fast super-resolution. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9006, pp. 111–126. Springer, Cham (2015). Scholar
  21. 21.
    Wang, Z., Hu, R., Wang, S., Jiang, J.: Face hallucination via weighted adaptive sparse regularization. IEEE Trans. Circuits Syst. Video Technol. 24(5), 802–813 (2014)CrossRefGoogle Scholar
  22. 22.
    Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process 13(4), 600–612 (2004)CrossRefGoogle Scholar
  23. 23.
    Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Yang, Z., He, P.: Non-local diffusion weighted image super-resolution using collaborative joint information. Exp. Ther. Med. 15(1), 217–225 (2018)Google Scholar
  25. 25.
    Zhang, Y., et al.: Collaborative representation cascade for single-image super-resolution. IEEE Trans. Syst. Man Cybern. Syst. PP(99), 1–16 (2017)Google Scholar
  26. 26.
    Zhang, Y., Zhang, Y., Zhang, J., Wang, H., Dai, Q.: Single image super-resolution via iterative collaborative representation. In: Ho, Y.-S., Sang, J., Ro, Y.M., Kim, J., Wu, F. (eds.) PCM 2015. LNCS, vol. 9315, pp. 63–73. Springer, Cham (2015). Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Hubei Key Laboratory of Intelligent Robot, School of Computer Science and EngineeringWuhan Institute of TechnologyWuhanChina
  2. 2.School of Computer Science and TechnologyHuazhong University of Science and TechnologyWuhanChina
  3. 3.School of SoftwareHe’nan UniversityKaifengChina

Personalised recommendations