Efficient and Effective Face Frontalization for Face Recognition in the Wild

  • Nelson Méndez
  • Luis A. Bouza
  • Leonardo Chang
  • Heydi Méndez-Vázquez
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10657)


Face image alignment is one of the most important steps in a face recognition system, being directly linked to its accuracy. In this work we propose a method for face frontalization based on the use of 3D models obtained from 2D images. We first extend the 3D Generic Elastic Model method in order to make it suitable for real applications, and once we have the 3D dense model of a face image, we obtain its frontal projection, introducing a new method for the synthesis of occluded regions. We evaluate the proposal by frontalizing the face images on LFW database and compare it with other frontalization techniques using different face recognition methods. We show that the proposed method allows to effectively align the images in an efficient way.


Face frontalization 3D face modeling Face recognition 


  1. 1.
    DLib C++ library kernel description. Accessed 02 May 2017
  2. 2.
    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. Techical Report SIGGRAPH (1999)Google Scholar
  3. 3.
    Deng, W., Hu, J., Wu, Z., Guo, J.: Lighting-aware face frontalization for unconstrained face recognition. Pattern Recogn. 68, 260–271 (2017)CrossRefGoogle Scholar
  4. 4.
    Ding, C., Tao, D.: A comprehensive survey on pose-invariant face recognition. ACM Trans. Intell. Syst. Technol. (TIST) 7(3), 37 (2016)Google Scholar
  5. 5.
    Ding, L., Ding, X., Fang, C.: Continuous pose normalization for pose-robust face recognition. IEEE Sig. Process. Lett. 19(11), 721–724 (2012)CrossRefGoogle Scholar
  6. 6.
    Hassner, T., Harel, S., Paz, E., Enbar, R.: Effective face frontalization in unconstrained images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4295–4304 (2015)Google Scholar
  7. 7.
    Heo, J.: 3D generic elastic models for 2D pose synthesis and face recognition. Technical Report (2009)Google Scholar
  8. 8.
    Heo, J., Savvides, M.: 3-D generic elastic models for fast and texture preserving 2-D novel pose synthesis. Trans. Inf. Forensics Secur. 7(2), 563–576 (2012)CrossRefGoogle Scholar
  9. 9.
    Heo, J., Savvides, M.: Gender and ethnicity specific generic elastic models from a single 2D image for novel 2D pose face synthesis and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 34(12), 2341–2350 (2012)CrossRefGoogle Scholar
  10. 10.
    Huang, G., Mattar, M., Lee, H., Learned-Miller, E.G.: Learning to align from scratch. In: NIPS, pp. 764–772 (2012)Google Scholar
  11. 11.
    Huang, G.B., Jain, V., Learned-Miller, E.: Unsupervised joint alignment of complex images. In: ICCV, pp. 1–8. IEEE (2007)Google Scholar
  12. 12.
    Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical Report 07–49, University of Massachusetts, Amherst (2007)Google Scholar
  13. 13.
    Jacobson, A.: Algorithms and interfaces for real-time deformation of 2D and 3D shapes. Ph.D. Thesis, ETH Zurich (2013)Google Scholar
  14. 14.
    Jo, J., Choi, H., Kim, I.J., Kim, J.: Single-view-based 3D facial reconstruction method robust against pose variations. Pattern Recogn. 48(1), 73–85 (2015)CrossRefGoogle Scholar
  15. 15.
    Moeini, A., Moeini, H., Faez, K.: Pose-invariant facial expression recognition based on 3D face reconstruction and synthesis from a single 2D image. In: ICPR, pp. 1746–1751 (2014)Google Scholar
  16. 16.
    Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: BMVC, vol. 1, p. 6 (2015)Google Scholar
  17. 17.
    Prabhu, U., Heo, J., Savvides, M.: Unconstrained pose-invariant face recognition using 3D generic elastic models. Trans. Pattern Anal. Mach. Intell. 33(10), 1952–1961 (2011)CrossRefGoogle Scholar
  18. 18.
    Sagonas, C., Panagakis, Y., Zafeiriou, S., Pantic, M.: Robust statistical face frontalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3871–3879 (2015)Google Scholar
  19. 19.
    Simonyan, K., Parkhi, O.M., Vedaldi, A., Zisserman, A.: Fisher vector faces in the wild. In: BMVC, vol. 2, p. 4 (2013)Google Scholar
  20. 20.
    Wolf, L., Hassner, T., Taigman, Y.: Effective unconstrained face recognition by combining multiple descriptors and learned background statistics. IEEE Trans. Pattern Anal. Mach. Intell. 33(10), 1978–1990 (2011)CrossRefGoogle Scholar
  21. 21.
    Yin, X., Yu, X., Sohn, K., Liu, X., Chandraker, M.: Towards large-pose face frontalization in the wild. arXiv preprint arXiv:1704.06244 (2017)

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Nelson Méndez
    • 1
  • Luis A. Bouza
    • 1
  • Leonardo Chang
    • 1
  • Heydi Méndez-Vázquez
    • 1
  1. 1.Advanced Technologies Application CenterHavanaCuba

Personalised recommendations