CNNs Under Attack: On the Vulnerability of Deep Neural Networks Based Face Recognition to Image Morphing

  • Lukasz WandzikEmail author
  • Raul Vicente Garcia
  • Gerald Kaeding
  • Xi Chen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10431)


Facial recognition has become a critical constituent of common automatic border control gates. Despite many advances in recent years, face recognition systems remain susceptible to an ever evolving diversity of spoofing attacks. It has recently been shown that high-quality face morphing or splicing can be employed to deceive facial recognition systems in a border control scenario. Moreover, facial morphs can easily be produced by means of open source software and with minimal technical knowledge. The purpose of this work is to quantify the severeness of the problem using a large dataset of morphed face images. We employ a state-of-the-art face recognition algorithm based on deep convolutional neural networks and measure its performance on a dataset of 7260 high-quality facial morphs with varying blending factor. Using the Inception-ResNet-v1 architecture we train a deep neural model on 4 million images to obtain a validation rate of \(99.96\%\) at \(0.04\%\) false acceptance rate (FAR) on the original, unmodified images. The same model fails to repel \(1.13\%\) of all morphing attacks, accepting both the impostor and the document owner. Based on these results, we discuss the observed weaknesses and possible remedies.


Face recognition Biometric spoofing Face morphing Deep learning 



This work has been partially funded by the German Federal Ministry of Education and Research (BMBF) under contract number FKZ: 16KIS 0512.


  1. 1.
    Dutta, A.: Predicting Performance of a Face Recognition System Based on Image Quality. Ph.D. thesis, University of Twente (2015).
  2. 2.
    European Union, E.A.f.t.M.o.O.C.a.t.E.B.: Best Practice Technical Guidelines for Automated Border Control (ABC) Systems. FRONTEX (2015)Google Scholar
  3. 3.
    Ferrara, M., Franco, A., Maltoni, D.: The magic passport. In: 2014 IEEE International Joint Conference on Biometrics (IJCB), pp. 1–7 (2014)Google Scholar
  4. 4.
    Galbally, J., Marcel, S., Fierrez, J.: Biometric antispoofing methods: a survey in face recognition. IEEE Access 2, 1530–1552 (2014)CrossRefGoogle Scholar
  5. 5.
    Guo, Y., Zhang, L., Hu, Y., He, X., Gao, J.: MS-Celeb-1M: a dataset and benchmark for large-scale face recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 87–102. Springer, Cham (2016). doi: 10.1007/978-3-319-46487-9_6 CrossRefGoogle Scholar
  6. 6.
    Hadid, A., Evans, N., Marcel, S., Fierrez, J.: Biometrics systems under spoofing attack: an evaluation methodology and lessons learned. IEEE Signal Process. Mag. 32(5), 20–30 (2015)CrossRefGoogle Scholar
  7. 7.
    Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical report, 07–49, University of Massachusetts, Amherst., October 2007Google Scholar
  8. 8.
    Zuo, J., Wechsler, H., et al.: Adaptive biometric authentication using nonlinear mappings on quality measures and verification scores. In: 2010 17th IEEE International Conference on Image Processing (ICIP), IEEE, Piscataway (2010).
  9. 9.
    Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1867–1874 (2014)Google Scholar
  10. 10.
    King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)Google Scholar
  11. 11.
    Makrushin, A., Neubert, T., Dittmann, J.: Automatic generation and detection of visually faultless facial morphs. In: Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, VISAPP, (VISIGRAPP 2017), vol. 6, pp. 39–50. INSTICC, ScitePress (2017)Google Scholar
  12. 12.
    Nguyen, A., Yosinski, J., Clune, J.: Deep neural networks are easily fooled: high confidence predictions for unrecognizable images. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 427–436 (2015)Google Scholar
  13. 13.
    Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: British Machine Vision Conference (2015)Google Scholar
  14. 14.
    Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Trans. Graph. (TOG) 22, 313–318 (2003). ACMCrossRefGoogle Scholar
  15. 15.
    Raghavendra, R., Raja, K., Busch, C.: Detecting morphed facial images. In: Proceedings of 8th IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS-2016) 6–9 September, Niagra Falls, USA (2016)Google Scholar
  16. 16.
    Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-pie. In: Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. IEEE Computer Society (2008).
  17. 17.
    Sandberg, D.: Tensorflow implementation of the facenet face recognizer (2016).
  18. 18.
    Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: a unified embedding for face recognition and clustering. CoRR abs/1503.03832 (2015).
  19. 19.
    Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.: Inception-v4, inception-resnet and the impact of residual connections on learning (2016).
  20. 20.
    Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., Fergus, R.: Intriguing properties of neural networks (2013).
  21. 21.
    Valente, J., Soatto, S.: Perspective distortion modeling, learning and compensation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 9–16 (2015)Google Scholar
  22. 22.
    Wen, Y., Zhang, K., Li, Z., Qiao, Y.: A discriminative feature learning approach for deep face recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 499–515. Springer, Cham (2016). doi: 10.1007/978-3-319-46478-7_31 Google Scholar
  23. 23.
    Wolberg, G.: Digital Image Warping, vol. 10662. IEEE Computer Society Press, Los Alamitos (1990)Google Scholar
  24. 24.
    Zhang, K., Zhang, Z., Li, Z., Qiao, Y.: Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Sig. Process. Lett. 23(10), 1499–1503 (2016)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Lukasz Wandzik
    • 1
    Email author
  • Raul Vicente Garcia
    • 1
  • Gerald Kaeding
    • 1
  • Xi Chen
    • 1
  1. 1.Fraunhofer Institute for Production Systems and Design Technology IPKBerlinGermany

Personalised recommendations