Face Verification Between ID Document Photos and Partial Occluded Spot Photos

  • Yunfei Zhao
  • Shikui WeiEmail author
  • Xiang Jiang
  • Tao Ruan
  • Yao Zhao
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11902)


ID-spot face verification is an important problem in face verification area, which aims to identify whether the spotted face is the same to the ID photo. Although some face verification systems have been deployed in many application scenarios, most of them are used in a constrained environment and many key problems need to be addressed furthermore. In this paper, we focus on a challenging ID-spot face verification task, in which the spot photo is partially occluded. Toward this end, a two-stream network is employed to learn more discriminative feature for distinguishing different ID-Spot face pairs. In addition, to suppress the negative effect of background and occlusion, a global weight pooling method is proposed, which makes the available face area more significant than the background and occlusion. The experimental results show that the proposed method obtains 10% improvements on FAR@0.01 compared with previous schemes.


Face verification Occluded face ID document photo versus spot photo 



This work was supported in part by National Key Research and Development of China (No. 2017YFC1703503), National Natural Science Foundation of China (No. 61532005, No. 61572065), Program of China Scholarships Council (No. 201807095006), Fundamental Research Funds for the Central Universities (No. 2018JBZ001).


  1. 1.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015)Google Scholar
  2. 2.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  3. 3.
    Yi, D., Lei, Z., Liao, S., Li, S.Z.: Learning face representation from scratch. Technical report arXiv:1411.7923 (2014)
  4. 4.
    Guo, Y., Zhang, L., Hu, Y., He, X., Gao, J.: MS-Celeb-1M: a dataset and benchmark for large-scale face recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 87–102. Springer, Cham (2016). Scholar
  5. 5.
    Wen, Y., Zhang, K., Li, Z., Qiao, Y.: A discriminative feature learning approach for deep face recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 499–515. Springer, Cham (2016). Scholar
  6. 6.
    Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., Song, L.: Sphereface: deep hypersphere embedding for face recognition. In: CVPR (2017)Google Scholar
  7. 7.
    Wang, H., Wang, Y., Zhou, Z., Ji, X., Gong, D., Zhou, J., et al.: CosFace: large margin cosine loss for deep face recognition. arXiv:1801.0941 (2018)
  8. 8.
    Deng, J., Guo, J., Zafeiriou, S.: ArcFace: additive angular margin loss for deep face recognition. arXiv:1801.07698 (2018)
  9. 9.
    Wang, F., Liu, W., Liu, H., Cheng, J.: Additive margin softmax for face verification. arXiv:1801.05599 (2018)
  10. 10.
    Sun, Y., Chen, Y., Wang, X., Tang, X.: Deep learning face representation by joint identification-verification. In: NIPS (2014)Google Scholar
  11. 11.
    Sun, Y., Wang, X., Tang, X.: Deeply learned face representations are sparse, selective, and robust. In: Computer Vision and Pattern Recognition, pp. 2892–2900 (2015)Google Scholar
  12. 12.
    Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: CVPR (2015)Google Scholar
  13. 13.
    Tang, X., Wang, X.: Face photo recognition using sketch. In: ICIP (2002)Google Scholar
  14. 14.
    Liu, Q., Tang, X., Jin, H., Lu, H., Ma, S.: A nonlinear approach for face sketch synthesis and recognition. In: CVPR (2005)Google Scholar
  15. 15.
    Liao, S., Yi, D., Lei, Z., Qin, R., Li, S.Z.: Heterogeneous face recognition from local structures of normalized appearance. In: Tistarelli, M., Nixon, M.S. (eds.) ICB 2009. LNCS, vol. 5558, pp. 209–218. Springer, Heidelberg (2009). Scholar
  16. 16.
    Klare, B.F., Jain, A.K.: Heterogeneous face recognition using kernel prototype similarities. IEEE Trans. PAMI 35(6), 1410–1422 (2013)CrossRefGoogle Scholar
  17. 17.
    Zhu, X., et al.: Large-scale bisample learning on id vs. spot face recognition. arXiv:1806.03018 (2018)
  18. 18.
    Shi, Y., Jain, A.K.: DocFace: matching ID document photos to selfies. arXiv:1805.02283 (2018)
  19. 19.
    Shi, Y., Jain, A.K.: DocFace+: ID document to selfie matching. arXiv:1809.05620 (2018)
  20. 20.
    Zhao, F., Feng, J., Zhao, J., Yang, W., Yan, S.: Robust LSTM-autoencoders for face de-occlusion in the wild. arXiv:1612.08534 (2016)
  21. 21.
    Singh, A., Patil, D., Reddy, M., Omkar, S.: Disguised face identification (DFI) with facial keypoints using spatial fusion convolutional network. In: The IEEE International Conference on Computer Vision (ICCV), pp. 1648–1655 (2017)Google Scholar
  22. 22.
    Trigueros, D.S., Meng, L., Hartnett, M.: Enhancing convolutional neural networks for face recognition with occlusion maps and batch triplet loss. In: Image and Vision Computing (2018)Google Scholar
  23. 23.
    Chen, S., Liu, Y., Gao, X., Han, Z.: MobileFaceNets: efficient CNNs for accurate real-time face verification on mobile devices. arXiv:1804.05737 (2018)
  24. 24.
    Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications. CoRR, abs/1704.04861 (2017)Google Scholar
  25. 25.
    Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetV2: inverted residuals and linear bottlenecks. CoRR, abs/1801.04381 (2018)Google Scholar
  26. 26.
    Zhang, X., Zhou, X., Lin, M., Sun, J.: ShuffleNet: an extremely efficient convolutional neural network for mobile devices. CoRR, abs/1707.01083 (2017)Google Scholar
  27. 27.
  28. 28.
    Huang, G.B., Ramesh, M., Berg, T., et al.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments (2007)Google Scholar
  29. 29.
    Moschoglou, S., Papaioannou, A., Sagonas, C., Deng, J., Kotsia, I., Zafeiriou, S.: AgeDB: the first manually collected in-the-wild age database. In: CVPRW (2017)Google Scholar
  30. 30.
    Sengupta, S., Chen, J.-C., Castillo, C., Patel, V.M., Chellappa, R., Jacobs, D.W.: Frontal to profile face verification in the wild. In: WACV, pp. 1–9 (2016)Google Scholar
  31. 31.
    Zhang, K., Zhang, Z., Li, Z., Qiao, Y.: Joint face detection and alignment using multi-task cascaded convolutional networks. IEEE Signal Proc. Lett. 23(10), 1499–1503 (2016)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Yunfei Zhao
    • 1
  • Shikui Wei
    • 1
    Email author
  • Xiang Jiang
    • 1
  • Tao Ruan
    • 1
  • Yao Zhao
    • 1
  1. 1.School of Computer and Information TechnologyBeijing Jiaotong UniversityBeijingChina

Personalised recommendations