Measuring the Gender and Ethnicity Bias in Deep Models for Face Recognition

  • Alejandro AcienEmail author
  • Aythami MoralesEmail author
  • Ruben Vera-RodriguezEmail author
  • Ivan BartolomeEmail author
  • Julian FierrezEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11401)


We explore the importance of gender and ethnic attributes in the decision-making of face recognition technologies. Our work is in part motivated by the new European regulation for personal data protection, which forces data controllers to avoid discriminative hazards while managing sensitive data like biometric data. The experiments in this paper are aimed to study what extent sensitive data like gender or ethnic origin attributes are present in the most common face recognition networks. For this, our experiments include two popular pre-trained networks: VGGFace and Resnet50. Both pre-trained models are able to classify gender and ethnicity easily (over 95% of performance) even suppressing 80% of the neurons in their embedding layers. The experimentation is conducted on a publicly available database known as Labeled Faces in the Wild with more than 13000 images of faces with a huge range of poses, ages, races and nationalities.


Face recognition Human attributes Gender Ethnic Discrimination 



This work was funded by the project CogniMetrics (TEC2015-70627-R) and Bio-Guard (Ayudas Fundación BBVA a Equipos de Investigación Científica 2017).


  1. 1.
    Neves, J., Narducci, F., Barra, S., Proença, H.: Biometric recognition in surveillance scenarios: a survey. Artif. Intell. Rev. 46(4), 515–541 (2016)CrossRefGoogle Scholar
  2. 2.
    Gonzalez-Sosa, E., Vera-Rodriguez, R., Fierrez, J., Ortega-Garcia, J.: Exploring facial regions in unconstrained scenarios: experience on ICB-RW. IEEE Intell. Syst. 33(3), 60–63 (2018)Google Scholar
  3. 3.
    Selinger, E., Polonetsky, J., Tene, O.: The Cambridge Handbook of Consumer Privacy, 1st edn. Cambridge University Press, Cambridge (2018)CrossRefGoogle Scholar
  4. 4.
    Ranjan, R., et al.: Deep learning for understanding faces: machines may be just as good, or better, than humans. IEEE Signal Process. Mag. 35, 66–83 (2018)CrossRefGoogle Scholar
  5. 5.
    Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. Proc. Br. Mach. Vis. 1(3), 6 (2015)Google Scholar
  6. 6.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)Google Scholar
  7. 7.
    Schroff, F., Kalenichenko, D., Philbin., J.: FaceNet: a unified embedding for face recognition and clustering. In: Proceedings of CVPR (2015)Google Scholar
  8. 8.
    LFW Homepage. Accessed 15 June 2018
  9. 9.
    Tome, P., Fierrez, J., Vera-Rodriguez, R., Nixon, M.: Soft biometrics and their application in person recognition at a distance. IEEE Trans. Inf. Forensics Secur. 9(3), 464–475 (2014)CrossRefGoogle Scholar
  10. 10.
    Tome, P., Vera-Rodriguez, R., Fierrez, J., Ortega-Garcia, J.: Facial soft biometric features for forensic face recognition. In: Forensic Science International, vol. 257, pp. 171–284, December 2015Google Scholar
  11. 11.
    Dantcheva, A., Elia, P., Ross, A.: What else does your biometric data reveal? A survey on soft biometrics. IEEE Trans. Inf. Forensics Secur. 11(3), 441–467 (2016)CrossRefGoogle Scholar
  12. 12.
    Prabhakar, S., Pankanti, S., Jain, A.K.: Biometric recognition: Security and privacy concerns. IEEE Secur. Priv. Mag. 1(2), 33–42 (2003)CrossRefGoogle Scholar
  13. 13.
    EU 2016/679 (General Data Protection Regulation). Accessed 17 Oct 2018
  14. 14.
    Goodman, B., Flaxman, F.: European Union regulations on algorithmic decision-making and a “right to explanation”. AI Mag. 38(3), 50–57 (2016)CrossRefGoogle Scholar
  15. 15.
    Huang, G.B., Mattar, M., Lee, H., Learned-Miller, E.: Learning to Align from the scratch. In: Advances in Neural Information Processing Systems NIPS (2012)Google Scholar
  16. 16.
    Gonzalez-Sosa, E., Fierrez, J., Vera-Rodriguez, R., Alonso-Fernandez, F.: Facial Soft biometrics for recognition in the wild: recent works, annotation and COTS evaluation. IEEE Trans. Inf. Forensics Secur. 13(7), 2001–2014 (2018)CrossRefGoogle Scholar
  17. 17.
    Cao, Q., Shen, L., Xie, W., M. Parkhi, O., Zisserman, A.: VGGFace2: a dataset for recognising faces across pose and age. arXiv:1710.08092 (2017)
  18. 18.
    Klare, B., et al.: Pushing the frontiers of unconstrained face detection and recognition: IARPA Janus benchmark A. In: CVPR, pp. 1931–1939 (2015)Google Scholar
  19. 19.
    Malli, R.C.: Github Homepage. Accessed 15 June 2018
  20. 20.
    Fierrez, J., Morales, A., Vera-Rodriguez, R., Camacho, D.: Multiple classifiers in biometrics. Part 1: fundamentals and review. Inf. Fusion 44, 57–64 (2018)CrossRefGoogle Scholar
  21. 21.
    Mirjalili, V., Raschka, S., Namboodiri, A., Ross, A.: Semi-adversarial networks: convolutional autoencoders for imparting privacy to face images. In: Proceedings of 11th IAPR International Conference on Biometrics, Australia, February 2018Google Scholar
  22. 22.
    Gomez-Barrero, M., Maiorana, E., Galbally, J., Campisi, P., Fierrez, J.: Multi-biometric template protection based on homomorphic encryption. Pattern Recogn. 67, 149–163 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Biometrics and Data Pattern Analytics (BiDA) Lab, EPSUniversidad Autonoma de MadridMadridSpain

Personalised recommendations