Skip to main content
SpringerLink
Log in

Face Reconstruction from Profile to Frontal Evaluation of Face Recognition

  • Chapter
  • First Online:
Data Management and Analysis

Part of the book series: Studies in Big Data ((SBD,volume 65))

  • 1038 Accesses

Abstract

One of the main challenges in face recognition is handling extreme variation of poses which may be faced for images collected in labs and in the wild. Recognizing faces in profile view has been shown to perform poorly compared to using frontal view of faces. Indeed, previous approaches failed to capture distinct features of a profile face compared to a frontal one. Approaches to enhance face recognition on profile faces have been recently proposed following two different trends. One trend depends on training a neural network model with big multi-view face datasets to learn features of faces by handling all poses. The second trend generates a frontal face image (face reconstruction) from any given face pose and applies feature extraction and face recognition on the generated face instead of profile faces. Recent methods for face reconstruction use generative adversarial networks (GAN) learning model to train two competing neural networks to generate authentic frontal view of any pose preserving person’s identity. For the work described in this paper, we trained a feature extraction neural network model to learn representation of any face pose which is then compared with each other using Euclidean distance. We also used two recent face reconstruction techniques to generate frontal faces. We evaluated the performance of using the generated frontal faces against the posed counterparts. In the conducted experiments, we used three face datasets that contain several challenges for face recognition having faces in a variety of poses and in the wild.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ahonen, T., Hadid, A., & Pietikainen, M. (2006). Face description with local binary patterns: Application to face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(12), 2037–2041.

    Article  Google Scholar 

  2. Turk, M. A., & Pentland, A. P. (1991). Face recognition using eigenfaces. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1991. Proceedings CVPR’91 (pp. 586–591). Piscataway: IEEE.

    Google Scholar 

  3. Huang, G. B., Ramesh, M., Berg, T., & Learned-Miller, E. (2007). Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical report, Technical Report 07-49, University of Massachusetts, Amherst.

    Google Scholar 

  4. Cao, Z., Yin, Q., Tang, X., & Sun, J. (2010). Face recognition with learning-based descriptor. In 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2707–2714). Piscataway: IEEE.

    Chapter  Google Scholar 

  5. Schroff, F., Kalenichenko, D., & Philbin, J. (2015). Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 815–823).

    Google Scholar 

  6. Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014). Deepface: Closing the gap to human-level performance in face verification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1701–1708).

    Google Scholar 

  7. Sun, Y., Wang, X., & Tang, X. (2014). Deep learning face representation from predicting 10,000 classes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1891–1898).

    Google Scholar 

  8. Sun, Y., Liang, D., Wang, X., & Tang, X. (2015). Deepid3: Face recognition with very deep neural networks. Preprint arXiv:1502.00873.

    Google Scholar 

  9. Kumar, N., Berg, A. C., Belhumeur, P. N., & Nayar, S. K. (2009). Attribute and simile classifiers for face verification. In 2009 IEEE 12th International Conference on Computer Vision (pp. 365–372). Piscataway: IEEE.

    Chapter  Google Scholar 

  10. Sengupta, S., Chen, J.-C., Castillo, C., Patel, V. M., Chellappa, R., & Jacobs, D. W. (2016). Frontal to profile face verification in the wild. In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV) (pp. 1–9). Piscataway: IEEE.

    Google Scholar 

  11. Hassner, T., Harel, S., Paz, E., & Enbar, R. (2015). Effective face frontalization in unconstrained images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4295–4304).

    Google Scholar 

  12. Yim, J., Jung, H., Yoo, B., Choi, C., Park, D., & Kim, J. (2015). Rotating your face using multi-task deep neural network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 676–684).

    Google Scholar 

  13. Tran, L., Yin, X., & Liu, X. (2017). Disentangled representation learning GAN for pose-invariant face recognition. In CVPR (Vol. 3, p. 7).

    Google Scholar 

  14. Huang, R., Zhang, S., Li, T., He, R., et al. (2017). Beyond face rotation: Global and local perception GAN for photorealistic and identity preserving frontal view synthesis. Preprint arXiv:1704.04086.

    Google Scholar 

  15. Yi, D., Lei, Z., Liao, S., & Li, S. Z. (2014). Learning face representation from scratch. Preprint arXiv:1411.7923.

    Google Scholar 

  16. Thomaz, C. E. (2012). Fei face database. http://fei.edu.br/~cet/facedatabase.html. Accessed 2 October 2012.

    Google Scholar 

  17. Phillips, P. J., Moon, H., Rizvi, S. A., & Rauss, P. J. (2000). The feret evaluation methodology for face-recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(10), 1090–1104.

    Article  Google Scholar 

  18. Klare, B. F., Klein, B., Taborsky, E., Blanton, A., Cheney, J., Allen, K., et al. (2015). Pushing the frontiers of unconstrained face detection and recognition: IARPA Janus Benchmark A. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1931–1939).

    Google Scholar 

  19. Masi, I., Rawls, S., Medioni, G., & Natarajan, P. (2016). Pose-aware face recognition in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4838–4846).

    Google Scholar 

  20. Huang, G., Mattar, M., Lee, H., & Learned-Miller, E. G. (2012). Learning to align from scratch. In Advances in Neural Information Processing Systems (pp. 764–772).

    Google Scholar 

  21. Wolf, L., Hassner, T., & Taigman, Y. (2009). Similarity scores based on background samples. In Asian Conference on Computer Vision (pp. 88–97). Berlin: Springer.

    Google Scholar 

  22. Zhu, X., Lei, Z., Yan, J., Yi, D., & Li, S. Z. (2015). High-fidelity pose and expression normalization for face recognition in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 787–796).

    Google Scholar 

  23. Zhu, Z., Luo, P., Wang, X., & Tang, X. (2014). Multi-view perceptron: A deep model for learning face identity and view representations. In Advances in Neural Information Processing Systems (pp. 217–225).

    Google Scholar 

  24. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., et al. Generative adversarial nets. In Advances in Neural Information Processing Systems (pp. 2672–2680).

    Google Scholar 

  25. Denton, E. L., Chintala, S., Fergus, R., et al. (2015). Deep generative image models using a Laplacian pyramid of adversarial networks. In Advances in Neural Information Processing Systems (pp. 1486–1494).

    Google Scholar 

  26. Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., et al. (2016). Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint.

    Google Scholar 

  27. Yu, X., & Porikli, F. (2016). Ultra-resolving face images by discriminative generative networks. In European Conference on Computer Vision (pp. 318–333). Berlin: Springer.

    Google Scholar 

  28. Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B., & Lee, H. (2016). Generative adversarial text to image synthesis. Preprint arXiv:1605.05396.

    Google Scholar 

  29. Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017). Image-to-image translation with conditional adversarial networks. arXiv preprint.

    Google Scholar 

  30. Wu, X., He, R., Sun, Z., & Tan, T. (2015). A light CNN for deep face representation with noisy labels. Preprint arXiv:1511.02683.

    Google Scholar 

  31. Gross, R., Matthews, I., Cohn, J., Kanade, T., & Baker, S. (2010). Multi-pie. Image and Vision Computing, 28(5), 807–813.

    Article  Google Scholar 

  32. Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001. CVPR 2001 (Vol. 1, pp. I–I). Piscataway: IEEE.

    Google Scholar 

  33. Dalal, N., & Triggs, B. (2005). Histograms of oriented gradients for human detection. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005 (Vol. 1, pp. 886–893). Piscataway: IEEE.

    Google Scholar 

  34. Wu, B., Ai, H., Huang, C., & Lao, S. (2004). Fast rotation invariant multi-view face detection based on real Adaboost. In Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings (pp. 79–84). Piscataway: IEEE.

    Google Scholar 

  35. Huang, C., Ai, H., Li, Y., & Lao, S. (2005). Vector boosting for rotation invariant multi-view face detection. In Tenth IEEE International Conference on Computer Vision, 2005. ICCV 2005 (Vol. 1, pp. 446–453). Piscataway: IEEE.

    Google Scholar 

  36. Jones, M., & Viola, P. (2003). Fast multi-view face detection. Mitsubishi Electric Research Lab TR-20003-96, 3, 14.

    Google Scholar 

  37. Zhang, C., & Zhang, Z. (2014). Improving multiview face detection with multi-task deep convolutional neural networks. In 2014 IEEE Winter Conference on Applications of Computer Vision (WACV) (pp. 1036–1041). Piscataway: IEEE.

    Google Scholar 

  38. Farfade, S. S., Saberian, M. J., & Li, L.-J. (2015). Multi-view face detection using deep convolutional neural networks. In Proceedings of the 5th ACM on International Conference on Multimedia Retrieval (pp. 643–650). New York: ACM.

    Chapter  Google Scholar 

  39. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (pp. 1097–1105).

    Google Scholar 

  40. Jain, V., & Learned-Miller, E. G. (2010). Fddb: A benchmark for face detection in unconstrained settings. UMass Amherst Technical Report.

    Google Scholar 

  41. Li, H., Lin, Z., Shen, X., Brandt, J., & Hua, G. (2015). A convolutional neural network cascade for face detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 5325–5334).

    Google Scholar 

  42. Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (pp. 580–587).

    Google Scholar 

  43. Uijlings, J. R. R., Van De Sande, K. E. A., Gevers, T., & Smeulders, A. W. M. (2013). Selective search for object recognition. International Journal of Computer Vision, 104(2), 154–171.

    Article  Google Scholar 

  44. Zitnick, C. L., & Dollár, P. (2014). Edge boxes: Locating object proposals from edges. In European Conference on Computer Vision (pp. 391–405). Berlin: Springer.

    Google Scholar 

  45. Girshick, R. (2015). Fast R-CNN. In Proceedings of the IEEE International Conference on Computer Vision (pp. 1440–1448).

    Google Scholar 

  46. Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster R-CNN: Towards real-time object detection with region proposal networks. In Advances in Neural Information Processing Systems (pp. 91–99).

    Google Scholar 

  47. Jiang, H., & Learned-Miller, E. (2016). Face detection with the faster R-CNN. Preprint arXiv:1606.03473.

    Google Scholar 

  48. Yang, S., Luo, P., Loy, C.-C., & Tang, X. (2016). Wider face: A face detection benchmark. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 5525–5533).

    Google Scholar 

  49. Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., et al. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. Preprint arXiv:1704.04861.

    Google Scholar 

  50. Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.

    Article  Google Scholar 

  51. Bay, H., Tuytelaars, T., & Van Gool, L. (2006). Surf: Speeded up robust features. Computer Vision-ECCV 2006 (pp. 404–417).

    Google Scholar 

  52. Geng, C., & Jiang, X. (2009). Face recognition using sift features. In 2009 16th IEEE International Conference on Image Processing (ICIP) (pp. 3313–3316). Piscataway: IEEE.

    Chapter  Google Scholar 

  53. Bicego, M., Lagorio, A., Grosso, E., & Tistarelli, M. (2006). On the use of sift features for face authentication. In Conference on Computer Vision and Pattern Recognition Workshop, 2006. CVPRW’06 (pp. 35–35). Piscataway: IEEE.

    Google Scholar 

  54. Du, G., Su, F., & Cai, A. (2009). Face recognition using surf features. In Sixth International Symposium on Multispectral Image Processing and Pattern Recognition (pp. 749628–749628). International Society for Optics and Photonics.

    Google Scholar 

  55. Chan, C.-H., Kittler, J., & Messer, K. (2007). Multi-scale local binary pattern histograms for face recognition. In Advances in Biometrics (pp. 809–818).

    Google Scholar 

  56. Sharif Razavian, A., Azizpour, H., Sullivan, J., & Carlsson, S. (2014). Cnn features off-the-shelf: An astounding baseline for recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 806–813).

    Google Scholar 

  57. Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. A. (2017). Inception-v4, inception-resnet and the impact of residual connections on learning. In AAAI (Vol. 4, p. 12).

    Google Scholar 

  58. Guo, Y., Zhang, L., Hu, Y., He, X., & Gao, J. (2016). MS-Celeb-1M: A dataset and benchmark for large scale face recognition. In European Conference on Computer Vision. Berlin: Springer.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Calgary, Calgary, AB, Canada

    Salim Afra & Reda Alhajj

  2. Department of Computer Engineering, Istanbul Medipol University, Istanbul, Turkey

    Reda Alhajj

Authors
  1. Salim Afra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Reda Alhajj
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Salim Afra .

Editor information

Editors and Affiliations

  1. Department of Computer Science, University of Calgary, Department of Computer Engineering Istanbul Medipol University Istanbul, Turkey, Calgary, AB, Canada

    Reda Alhajj

  2. Department of Electrical and Computer Engineering, University of Calgary, Calgary, AB, Canada

    Mohammad Moshirpour

  3. Department of Electrical and Computer Engineering, University of Calgary, Calgary, AB, Canada

    Behrouz Far

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Afra, S., Alhajj, R. (2020). Face Reconstruction from Profile to Frontal Evaluation of Face Recognition. In: Alhajj, R., Moshirpour, M., Far, B. (eds) Data Management and Analysis. Studies in Big Data, vol 65. Springer, Cham. https://doi.org/10.1007/978-3-030-32587-9_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-32587-9_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-32586-2

  • Online ISBN: 978-3-030-32587-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation