Mitigating Bias in Gender, Age and Ethnicity Classification: A Multi-task Convolution Neural Network Approach

  • Abhijit DasEmail author
  • Antitza Dantcheva
  • Francois Bremond
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11129)


This work explores joint classification of gender, age and race. Specifically, we here propose a Multi-Task Convolution Neural Network (MTCNN) employing joint dynamic loss weight adjustment towards classification of named soft biometrics, as well as towards mitigation of soft biometrics related bias. The proposed algorithm achieves promising results on the UTKFace and the Bias Estimation in Face Analytics (BEFA) datasets and was ranked first in the BEFA Challenge of the European Conference of Computer Vision (ECCV) 2018.


Bias Facial analysis Age Gender and race Soft biometrics Facial attributes 


  1. 1.
    Buolamwini, J., Gebru, T.: Gender shades: intersectional accuracy disparities in commercial gender classification. In: Conference on Fairness, Accountability and Transparency, pp. 77–91 (2018)Google Scholar
  2. 2.
    Zafeiriou, S., Zhang, C., Zhang, Z.: A survey on face detection in the wild: past, present and future. Comput. Vis. Image Underst. 138, 1–24 (2015)CrossRefGoogle Scholar
  3. 3.
    Dantcheva, A., Elia, P., Ross, A.: What else does your biometric data reveal? A survey on soft biometrics. IEEE Trans. Inf. Forensics Secur. 11, 1–26 (2015)Google Scholar
  4. 4.
    Ranjan, R., Sankaranarayanan, S., Castillo, C.D., Chellappa, R.: An all-in-one convolutional neural network for face analysis. In: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), pp. 17–24. IEEE (2017)Google Scholar
  5. 5.
    Dehghan, A., Ortiz, E.G., Shu, G., Masood, S.Z.: Dager: deep age, gender and emotion recognition using convolutional neural network. arXiv preprint arXiv:1702.04280 (2017)
  6. 6.
    Wang, Y., Kosinski, M.: Deep neural networks are more accurate than humans at detecting sexual orientation from facial images. J. Pers. Soc. Psychol. 114(2), 246 (2018)CrossRefGoogle Scholar
  7. 7.
    Wu, X., Zhang, X.: Automated inference on criminality using face images. arXiv preprint arXiv:1611.04135, pp. 4038–4052 (2016)
  8. 8.
    Dantcheva, A., Bremond, F., Bilinski, P.: Show me your face and I will tell you your height, weight and body mass index. In: International Conference on Pattern Recognition (ICPR) (2018)Google Scholar
  9. 9.
    Dantcheva, A., Dugelay, J.: Female facial aesthetics based on soft biometrics and photo-quality. In: IEEE International Conference on Multimedia and Expo (ICME) (2011)Google Scholar
  10. 10.
    Klare, B.F., Burge, M.J., Klontz, J.C., Bruegge, R.W.V., Jain, A.K.: Face recognition performance: role of demographic information. IEEE Trans. Inf. Forensics Secur. 7(6), 1789–1801 (2012)CrossRefGoogle Scholar
  11. 11.
    Ngan, M., Ngan, M., Grother, P.: Face recognition vendor test (FRVT) performance of automated gender classification algorithms. US Department of Commerce, National Institute of Standards and Technology (2015)Google Scholar
  12. 12.
    Farinella, G., Dugelay, J.L.: Demographic classification: do gender and ethnicity affect each other? In: 2012 International Conference on Informatics, Electronics & Vision (ICIEV), pp. 383–390. IEEE (2012)Google Scholar
  13. 13.
    Dwork, C., Hardt, M., Pitassi, T., Reingold, O., Zemel, R.: Fairness through awareness. In: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, pp. 214–226. ACM (2012)Google Scholar
  14. 14.
    Clare Garvie, A.B., Frankle, J.: The perpetual line-up: unregulated police face recognition in America (2016)Google Scholar
  15. 15.
    Cheng, Y.D., O’Toole, A.J., Abdi, H.: Classifying adults’ and children’s faces by sex: computational investigations of subcategorical feature encoding. Cogn. Sci. 25(5), 819–838 (2001)CrossRefGoogle Scholar
  16. 16.
    Dantcheva, A., Bremond, F.: Gender estimation based on smile-dynamics. IEEE Trans. Inf. Forensics Secur. 12(3), 719–729 (2017)CrossRefGoogle Scholar
  17. 17.
    Bilinski, P., Dantcheva, A., Brémond, F.: Can a smile reveal your gender? In: 2016 International Conference of the Biometrics Special Interest Group (BIOSIG), pp. 1–6. IEEE (2016)Google Scholar
  18. 18.
    Ranjan, R., Castillo, C.D., Chellappa, R.: L2-constrained softmax loss for discriminative face verification. arXiv preprint arXiv:1703.09507 (2017)
  19. 19.
    Ryu, H.J., Adam, H., Mitchell, M.: InclusiveFaceNet: improving face attribute detection with race and gender diversity. In: Workshop on Fairness, Accountability, and Transparency in Machine Learning (FAT/ML) (2018)Google Scholar
  20. 20.
    Dwork, C., Immorlica, N., Kalai, A.T., Leiserson, M.D.: Decoupled classifiers for group-fair and efficient machine learning. In: Conference on Fairness, Accountability and Transparency, pp. 119–133 (2018)Google Scholar
  21. 21.
    Han, H., Jain, A.K.: Age, gender and race estimation from unconstrained face images. Department of Computer Science and Engineering, Michigan State University, East Lansing, MI, USA, MSU Technical report (MSU-CSE-14-5) (2014)Google Scholar
  22. 22.
    Klare, B.F., et al.: Pushing the frontiers of unconstrained face detection and recognition: Iarpa Janus benchmark a. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1931–1939 (2015)Google Scholar
  23. 23.
    Levi, G., Hassner, T.: Age and gender classification using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 34–42 (2015)Google Scholar
  24. 24.
    Escalera, S., et al.: Chalearn looking at people and faces of the world: face analysis workshop and challenge 2016. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1–8 (2016)Google Scholar
  25. 25.
    Zhang, Z., Song, Y., Qi, H.: Age progression/regression by conditional adversarial autoencoder. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2017)Google Scholar
  26. 26.
    Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)Google Scholar
  27. 27.
    Yi, D., Lei, Z., Liao, S., Li, S.Z.: Learning face representation from scratch. arXiv preprint arXiv:1411.7923 (2014)
  28. 28.
    Fang, Y., Ma, Z., Zhang, Z., Zhang, X.Y., Bai, X.: Dynamic multi-task learning with convolutional neural networkGoogle Scholar
  29. 29.
    Yin, X., Liu, X.: Multi-task convolutional neural network for pose-invariant face recognition. IEEE Trans. Image Process. 27, 964–975 (2017)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Huang, G.B., Mattar, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. In: Workshop on Faces in ‘Real-Life’ Images: Detection, Alignment, and Recognition (2008)Google Scholar
  31. 31.
    Guo, Y., Zhang, L., Hu, Y., He, X., Gao, J.: MS-Celeb-1M: a dataset and benchmark for large-scale face recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 87–102. Springer, Cham (2016). Scholar
  32. 32.
    Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A.: Inception-v4, inception-resnet and the impact of residual connections on learning (2017)Google Scholar
  33. 33.
    Zhang, K., Zhang, Z., Li, Z., Qiao, Y.: Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process. Lett. 23(10), 1499–1503 (2016)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Abhijit Das
    • 1
    Email author
  • Antitza Dantcheva
    • 1
  • Francois Bremond
    • 1
  1. 1.InriaSophia AntipolisFrance

Personalised recommendations