Advertisement

Parallel-Structure-based Transfer Learning for Deep NIR-to-VIS Face Recognition

  • Yufei Wang
  • Yali Li
  • Shengjin WangEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11901)

Abstract

This paper considers a heterogeneous face recognition problem, i.e., matching near-infrared (NIR) to visible (VIS) face images. The significant domain gap between the NIR and VIS modalities poses great challenges to accurate face recognition. To overcome the domain gap problem, previous works usually adopted a series structure to transfer high-level features. This paper proposes a Parallel-Structure-based Transfer learning method (PST), which fully utilizes multi-scale feature map information. Specifically, PST consists of two parallel streams of network, i.e., a source stream (S-stream) and a transfer stream (T-stream). S-stream is pre-trained on a large-scale VIS database, and its parameters are fixed. It preserves the discriminative ability learned from the large-scale source dataset. T-stream absorbs multi-scale feature maps from S-stream and transfers the NIR and VIS face embeddings to a unique feature space, which is agnostic to the input image modality. The proposed PST method achieves state-of-the-art performance on CASIA NIR-VIS 2.0 Database, the largest near-infrared face database.

Keywords

Heterogeneous face recognition Near-infrared Transfer learning Multi-scale 

References

  1. 1.
    Cao, Q., Shen, L., Xie, W., Parkhi, O.M., Zisserman, A.: VGGFace2: a dataset for recognising faces across pose and age. In: 2018 13th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2018), pp. 67–74. IEEE (2018)Google Scholar
  2. 2.
    Deng, J., Guo, J., Zafeiriou, S.: ArcFace: additive angular margin loss for deep face recognition. arXiv preprint arXiv:1801.07698 (2018)
  3. 3.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  4. 4.
    He, R., Wu, X., Sun, Z., Tan, T.: Learning invariant deep representation for NIR-VIS face recognition (2017)Google Scholar
  5. 5.
    He, R., Wu, X., Sun, Z., Tan, T.: Wasserstein CNN: learning invariant features for NIR-VIS face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 41, 1761–1773 (2018)CrossRefGoogle Scholar
  6. 6.
    Kemelmacher-Shlizerman, I., Seitz, S.M., Miller, D., Brossard, E.: The MegaFace benchmark: 1 million faces for recognition at scale. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4873–4882 (2016)Google Scholar
  7. 7.
    Klare, B.F., Jain, A.K.: Heterogeneous face recognition using kernel prototype similarities. IEEE Trans. Pattern Anal. Mach. Intell. 35(6), 1410–1422 (2013)CrossRefGoogle Scholar
  8. 8.
    Li, S., Yi, D., Lei, Z., Liao, S.: The CASIA NIR-VIS 2.0 face database. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 348–353 (2013)Google Scholar
  9. 9.
    Li, Y., Wang, S., Tian, Q., Ding, X.: Feature representation for statistical-learning-based object detection: a review. Pattern Recogn. 48(11), 3542–3559 (2015)CrossRefGoogle Scholar
  10. 10.
    Li, Y., Wang, S., Tian, Q., Ding, X.: A survey of recent advances in visual feature detection. Neurocomputing 149, 736–751 (2015)CrossRefGoogle Scholar
  11. 11.
    Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., Song, L.: SphereFace: deep hypersphere embedding for face recognition Google Scholar
  12. 12.
    Liu, X., Song, L., Wu, X., Tan, T.: Transferring deep representation for NIR-VIS heterogeneous face recognition. In: 2016 International Conference on Biometrics (ICB), pp. 1–8. IEEE (2016)Google Scholar
  13. 13.
    Paszke, A., et al.: Automatic differentiation in pytorch (2017)Google Scholar
  14. 14.
    Reale, C., Nasrabadi, N.M., Kwon, H., Chellappa, R.: Seeing the forest from the trees: a holistic approach to near-infrared heterogeneous face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 54–62 (2016)Google Scholar
  15. 15.
    Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)Google Scholar
  16. 16.
    Socolinsky, D.A., Selinger, A., Neuheisel, J.D.: Face recognition with visible and thermal infrared imagery. Comput. Vis. Image Underst. 91(1–2), 72–114 (2003)CrossRefGoogle Scholar
  17. 17.
    Song, L., Zhang, M., Wu, X., He, R.: Adversarial discriminative heterogeneous face recognition. arXiv preprint arXiv:1709.03675 (2017)
  18. 18.
    Wang, F., Cheng, J., Liu, W., Liu, H.: Additive margin softmax for face verification. IEEE Signal Process. Lett. 25(7), 926–930 (2018)CrossRefGoogle Scholar
  19. 19.
    Wang, F., Xiang, X., Cheng, J., Yuille, A.L.: NormFace: L2 hypersphere embedding for face verification. In: Proceedings of the 2017 ACM on Multimedia Conference, pp. 1041–1049. ACM (2017)Google Scholar
  20. 20.
    Wang, H., Wang, Y., Zhou, Z., Ji, X., Liu, W.: CosFace: large margin cosine loss for deep face recognitionGoogle Scholar
  21. 21.
    Wu, X., Huang, H., Patel, V.M., He, R., Sun, Z.: Disentangled variational representation for heterogeneous face recognition. arXiv preprint arXiv:1809.01936 (2018)
  22. 22.
    Wu, X., Song, L., He, R., Tan, T.: Coupled deep learning for heterogeneous face recognition. arXiv preprint arXiv:1704.02450 (2017)
  23. 23.
    Yi, D., Lei, Z., Liao, S., Li, S.Z.: Learning face representation from scratch. arXiv preprint arXiv:1411.7923 (2014)
  24. 24.
    Zhang, Y., Zhang, Y., Yang, Q.: Parameter transfer unit for deep neural networks. arXiv preprint arXiv:1804.08613 (2018)

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Beijing National Research Center for Information Science and Technology, Department of Electronic EngineeringTsinghua UniversityBeijingChina

Personalised recommendations