Face Synthesis for Eyeglass-Robust Face Recognition

  • Jianzhu Guo
  • Xiangyu ZhuEmail author
  • Zhen Lei
  • Stan Z. Li
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10996)


In the application of face recognition, eyeglasses could significantly degrade the recognition accuracy. A feasible method is to collect large-scale face images with eyeglasses for training deep learning methods. However, it is difficult to collect the images with and without glasses of the same identity, so that it is difficult to optimize the intra-variations caused by eyeglasses. In this paper, we propose to address this problem in a virtual synthesis manner. The high-fidelity face images with eyeglasses are synthesized based on 3D face model and 3D eyeglasses. Models based on deep learning methods are then trained on the synthesized eyeglass face dataset, achieving better performance than previous ones. Experiments on the real face database validate the effectiveness of our synthesized data for improving eyeglass face recognition performance.


Face recognition 3D eyeglass fitting Face image synthesis 



This work was supported by the Chinese National Natural Science Foundation Projects #61473291, #61572536, #61572501, #61573356, the National Key Research and Development Plan (Grant No. 2016YFC0801002), and AuthenMetric R&D Funds.


  1. 1.
    Taigman, Y., Yang, M., Ranzato, M.A., Wolf, L.: Deepface: closing the gap to human-level performance in face verification. In: CVPR (2014)Google Scholar
  2. 2.
    Sun, Y., Wang, X., Tang, X.: Deep learning face representation from predicting 10,000 classes. In: CVPR (2014)Google Scholar
  3. 3.
    Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: a unified embedding for face recognition and clustering. In: CVPR (2015)Google Scholar
  4. 4.
    Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., Song, L.: Sphereface: deep hypersphere embedding for face recognition. In: CVPR (2017)Google Scholar
  5. 5.
    Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical report (2007)Google Scholar
  6. 6.
    Wolf, L., Hassner, T., Maoz, I.: Face recognition in unconstrained videos with matched background similarity. In: CVPR (2011)Google Scholar
  7. 7.
    Kemelmacher-Shlizerman, I., Seitz, S.M., Miller, D., Brossard, E.: The megaface benchmark: 1 million faces for recognition at scale. In: CVPR (2016)Google Scholar
  8. 8.
    Saito, Y., Kenmochi, Y., Kotani, K.: Estimation of eyeglassless facial images using principal component analysis. In: ICIP (1999)Google Scholar
  9. 9.
    Wu, C., Liu, C., Shum, H.Y., Xy, Y.Q., Zhang, Z.: Automatic eyeglasses removal from face images. TPAMI 26, 322–336 (2004)CrossRefGoogle Scholar
  10. 10.
    Du, C., Su, G.: Eyeglasses removal from facial images. PR 26, 2215–2220 (2005)Google Scholar
  11. 11.
    Park, J.S., Oh, Y.H., Ahn, S.C., Lee, S.W.: Glasses removal from facial image using recursive error compensation. TPAMI 27, 805–811 (2005)CrossRefGoogle Scholar
  12. 12.
    Yuan, X., Tang, D., Liu, Y., Ling, Q., Fang, L.: From 2D to 3D. TCSVT 27 (2017)Google Scholar
  13. 13.
    Zhang, Q., Guo, Y., Laffont, P.Y., Martin, T., Gross, M.: A virtual try-on system for prescription eyeglasses. IEEE COMPUT GRAPH 37, 84–93 (2017)CrossRefGoogle Scholar
  14. 14.
    Niswar, A., Khan, I.R., Farbiz, F.: Virtual try-on of eyeglasses using 3D model of the head. In: VRCAI (2011)Google Scholar
  15. 15.
    Su, H., Qi, C.R., Li, Y., Guibas, L.J.: Render for CNN: Viewpoint estimation in images using CNNs trained with rendered 3d model views. In: ICCV (2015)Google Scholar
  16. 16.
    Massa, F., Russell, B.C., Aubry, M.: Deep exemplar 2D–3D detection by adapting from real to rendered views. In: CVPR (2016)Google Scholar
  17. 17.
    Stark, M., Goesele, M., Schiele, B.: Back to the future: learning shape models from 3D CAD data. In: BMVC (2010)Google Scholar
  18. 18.
    Liebelt, J., Schmid, C.: Multi-view object class detection with a 3D geometric model. In: CVPR (2010)Google Scholar
  19. 19.
    Zhu, X., Lei, Z., Yan, J., Yi, D., Li, S.Z.: High-fidelity pose and expression normalization for face recognition in the wild. In: CVPR (2016)Google Scholar
  20. 20.
    Wu, X., He, R., Sun, Z., Tan, T.: A light CNN for deep face representation with noisy labels. arXiv preprint arXiv:1511.02683 (2015)
  21. 21.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CVPR (2016)Google Scholar
  22. 22.
    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: CVPR (2009)Google Scholar
  23. 23.
    Sun, Y., Chen, Y., Wang, X., Tang, X.: Deep learning face representation by joint identification-verification. In: NIPS (2014)Google Scholar
  24. 24.
    Jia, Y., Shelhamer, E., Donahue, J., et al.: Caffe: convolutional architecture for fast feature embedding. In: ACM Multimedia (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Jianzhu Guo
    • 1
    • 2
  • Xiangyu Zhu
    • 1
    • 2
    Email author
  • Zhen Lei
    • 1
    • 2
  • Stan Z. Li
    • 1
    • 2
  1. 1.CBSR&NLPRInstitute of Automation, Chinese Academy of SciencesBeijingChina
  2. 2.University of Chinese Academy of SciencesBeijingChina

Personalised recommendations