Skip to main content
SpringerLink
Log in

Real-time demographic profiling from face imagery with Fisher vectors

  • Original paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

In the last decade, demographic profiling from facial imagery has grown in its importance in the computer vision field. For demographic profiling, we usually mean gender, ethnicity, and age identification from face images. In this paper, we propose an efficient and effective profiling framework and we assess the quality of the proposed approach comparing the results obtained by our system with those achieved by other recently published methods on large datasets of facial images with different age, gender, and ethnicity. These results show how a carefully engineered pipeline of efficient image analysis and pattern recognition techniques leads to state-of-the-art results at 20 FPS using a single thread on a 1.6 GHZ i5-2467M processor.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. “Ethnicity: definition of ethnicity”. Oxford Dictionaries. Oxford University Press. Retrieved 28 December 2013.

  2. We call “white” a dataset of points sampled from a probability distribution with \({\varvec{\mu }}={\varvec{0}}\), and \({\varvec{\varSigma }}={\varvec{I}}\) where \({\varvec{I}}\) is the identity matrix.

  3. d is a parameter to be set. Usually a good value is \(d \simeq min(log_2^2 N,D)\)

  4. http://goo.gl/NKVCam

  5. https://www.openu.ac.il/home/hassner/Adience/data.html.

  6. http://chalearnlap.cvc.uab.es/dataset/18/description/.

  7. http://chalearnlap.cvc.uab.es/dataset/19/description/.

  8. A video with showing the output of our algorithm on every subject in this dataset is available as supplementary material.

References

  1. Acharya, K.A., Babu, R.V., Vadhiyar, S.S.: A real-time implementation of sift using gpu. J. Real Time Image Process. 14(2), 267–277 (2018)

    Article  Google Scholar 

  2. Akata, Z., Perronnin, F., Harchaoui, Z., Schmid, C.: Good practice in large-scale learning for image classification. IEEE Trans. Pattern Anal. Mach. Intell. 36(3), 507–520 (2014)

    Article  Google Scholar 

  3. Alexandre, L.A.: Gender recognition: a multiscale decision fusion approach. Pattern Recognit. Lett. 31(11), 1422–1427 (2010)

    Article  Google Scholar 

  4. Antipov, G., Baccouche, M., Berrani, S.-A., Dugelay, J.-L.: Apparent age estimation from face images combining general and children-specialized deep learning models. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 96–104 (2016)

  5. Canziani, A., Paszke, A., Culurciello, E.: An analysis of deep neural network models for practical applications. arXiv preprint arXiv:1605.07678 (2016)

  6. Chang, K.-Y., Chen, C.-S., Hung, Y.-P.: A ranking approach for human ages estimation based on face images. In: Proceedings of ICPR, Aug, pp. 3396–3399 (2010)

  7. Chang, K.-Y., Chen, C.-S., Hung, Y.-P.: Ordinal hyperplanes ranker with cost sensitivities for age estimation. In: Proceedings of CVPR, June, pp. 585–592 (2011)

  8. Cirne, M.V.M., Pedrini, H.: Gender recognition from face images using a geometric descriptor. In: 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 2006–2011. IEEE (2017)

  9. Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–685 (2001)

    Article  Google Scholar 

  10. Demirkus, M., Precup, D., Clark, J., Arbel, T.: Hierarchical spatio-temporal probabilistic graphical model with multiple feature fusion for estimating binary facial attribute classes in real-world face videos. IEEE Trans. Pattern Anal. Mach. Intell. 6(1185), 1203 (2015)

    Google Scholar 

  11. Du, S., Ward, R.: Wavelet-based illumination normalization for face recognition. In: Proceedings of ICPR (2005)

  12. Duan, M., Li, K., Li, K.: An ensemble cnn2elm for age estimation. IEEE Trans. Inf. Forensics Secur. 13(3), 758–772 (2018)

    Article  Google Scholar 

  13. Duan, M., Li, K., Yang, C., Li, K.: A hybrid deep learning cnn-elm for age and gender classification. Neurocomputing 275, 448–461 (2018)

    Article  Google Scholar 

  14. Eidinger, E., Enbar, R., Hassner, T.: Age and gender estimation of unfiltered faces. IEEE Trans. Inf. Forensics Secur. 9(12), 2170–2179 (2014)

    Article  Google Scholar 

  15. Escalera, S., Torres Torres, M., Martinez, B., Baró, X., Jair Escalante, H., Guyon, I., Tzimiropoulos, G., Corneou, C., Oliu, M., Ali Bagheri, M., et al: Chalearn looking at people and faces of the world: face analysis workshop and challenge 2016. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1–8 (2016)

  16. Fourney, A., Laganiere, R.: Constructing face image logs that are both complete and concise. In: Proceedings of CRV, pp. 488–494 (2007)

  17. Geng, X., Yin, C., Zhou, Z.-H.: Facial age estimation by learning from label distributions. IEEE Trans. Pattern Anal. Mach. Intell. 35(10), 2401–2412 (2013)

    Article  Google Scholar 

  18. Guo, G., Mu, G., Fu, Y., Huang, T.S.: Human age estimation using bio-inspired features. In: Proceedings of CVPR, June, pp. 112–119 (2009)

  19. Guo, G., Mu, G.: A study of large-scale ethnicity estimation with gender and age variations. In: Proceedings of CVPRW, pp. 79–86. IEEE (2010)

  20. Guo, G., Mu, G.: Human age estimation: what is the influence across race and gender? In: Proceedings of CVPRW, June, pp. 71–78 (2010)

  21. Guo, G., Mu, G.: Simultaneous dimensionality reduction and human age estimation via kernel partial least squares regression. In: Proceedings of CVPR, June, pp. 657–664 (2011)

  22. Guo, G., Mu, G.: Simultaneous dimensionality reduction and human age estimation via kernel partial least squares regression. In: Proceedings of CVPR, June, pp. 657–664 (2011)

  23. Guo, G., Zhang, C.: A study on cross-population age estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4257–4263 (2014)

  24. Gurpinar, F., Kaya, H., Dibeklioglu, H., Salah, A.: Kernel elm and cnn based facial age estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 80–86 (2016)

  25. Han, H., Otto, C., Jain, A.K.: Age estimation from face images: human vs. machine performance. In: Proceedings of ICB (2013)

  26. Hansen, P.C.: The truncated SVD as a method for regularization. BIT Numer. Math. 27(4), 534–553 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  27. Hassner, T., Harel, S., Paz, E., Enbar, R.: Effective face frontalization in unconstrained images. In: Proceedings of CVPR, June 2015

  28. Hou, L., Yu, C.-P., Samaras, D.: Squared earth movers distance loss for training deep neural networks on ordered-classes. In: Proceedings of NIPS (2017)

  29. Huang, G.-B., Chen, L., Siew, C.K., et al.: Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans. Neural Netw. 17(4), 879–892 (2006)

    Article  Google Scholar 

  30. Huerta, I., Fernández, C., Prati, A.: Facial age estimation through the fusion of texture and local appearance descriptors. In: European Conference on Computer Vision, pp. 667–681. Springer (2014)

  31. Huo, Z., Yang, X., Xing, C., Zhou, Y., Hou, P., Lv, J., Geng, X.: Deep age distribution learning for apparent age estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 17–24 (2016)

  32. Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: Proceedings of CVPR (2014)

  33. King, D.E.: Max-Margin Object Detection. ArXiv e-prints, January (2015)

  34. Lapuschkin, S., Binder, A., Müller, K.-R., Samek, W.: Understanding and comparing deep neural networks for age and gender classification. arXiv preprint arXiv:1708.07689 (2017)

  35. Levi, G., Hassner, T.: Age and gender classification using convolutional neural networks. In: Proceedings of CVPRW, June (2015)

  36. Liu, X., Li, S., Kan, M., Zhang, J., Wu, S., Liu, W., Han, H., Shan, S., Chen, X.: Agenet: Deeply learned regressor and classifier for robust apparent age estimation. In: The IEEE International Conference on Computer Vision (ICCV) Workshops, December 2015

  37. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  38. Malli, R.C., Aygün, M., Ekenel, H.K.: Apparent age estimation using ensemble of deep learning models. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 714–721. IEEE (2016)

  39. Meyers, E., Wolf, L.: Using biologically inspired features for face processing. Int. J. Comput. Vis. 76(1), 93–104 (2008)

    Article  Google Scholar 

  40. Nasrollahi, K., Moeslund, T.B: Face quality assessment system in video sequences. In: Biometrics and Identity Management, pp. 10–18. Springer (2008)

  41. Ng, C.B., Tay, Y.H., Goi, B.-M.: Vision-based human gender recognition: a survey. CoRR, abs/1204.1611 (2012)

  42. Ngan, M., Grother, P.: Face recognition vendor test (frvt) performance of automated age estimation algorithms. NIST Interagency Report, 7995 (2014)

  43. Ngan, M., Grother, P.: Face recognition vendor test (frvt) performance of automated gender classification algorithms. In: Technical Report NIST IR 8052. National Institute of Standards and Technology (2015)

  44. Ni, B., Song, Z., Yan, S.: Web image and video mining towards universal and robust age estimator. IEEE Trans. Multimed. 13(6), 1217–1229 (2011)

    Article  Google Scholar 

  45. Nvidia cublas library for fast gpu-accelerated implementation of the standard basic linear algebra subroutines (blas). https://developer.nvidia.com/cublas. Accessed 18 Sep 2018

  46. Nvidia embedded gpu boards. https://developer.nvidia.com/embedded/develop/hardware. Accessed 18 Sep 2018

  47. Platt, J.C., Cristianini, N., Shawe-taylor, J.: Large margin DAGs for multiclass classification. In: Proceedings of NIPS, pp. 547–553. MIT Press (2000)

  48. Riesenhuber, Maximilian, Poggio, Tomaso: Hierarchical models of object recognition in cortex. Nat. Neurosci. 2(11), 1019–1025 (1999)

    Article  Google Scholar 

  49. Rothe, R., Timofte, R., Van Gool, L.: Dex Deep expectation of apparent age from a single image. In: The IEEE International Conference on Computer Vision (ICCV) Workshops, December 2015

  50. Rozza, A., Lombardi, G., Casiraghi, E.: Novel ipca-based classifiers and their application to spam filtering. In: Proceedings of ISDA, pp. 797–802 (2009)

  51. Rozza, A., Lombardi, G., Rosa, M., Casiraghi, E: O-ipcac and its application to eeg classification. In: Proceedings of WAPA, pp. 4–11 (2010)

  52. Rozza, Alessandro, Lombardi, Gabriele, Casiraghi, Elena, Campadelli, Paola: Novel fisher discriminant classifiers. Pattern Recognit. 45(10), 3725–3737 (2012)

    Article  MATH  Google Scholar 

  53. Seidenari, L., Rozza, A., Del Bimbo, A.: Real-time age estimation from face imagery using fisher vectors. In: Proceedings of ICIAP (2015)

  54. Seidenari, L., Serra, G., Badanov, A.D., Bimbo, A.: Local pyramidal descriptors for image recognition. IEEE Trans. Pattern Anal. Mach. Intell. 36, 1033–1040 (2014)

    Article  Google Scholar 

  55. Simonyan, K., Parkhi, O.M., Vedaldi, A., Zisserman, A.: Fisher Vector Faces in the Wild. In: Proceedings of British Machine Vision Conference (BMVC) (2013)

  56. Snchez, Jorge, Perronnin, Florent, Mensink, Thomas, Verbeek, Jakob: Image classification with the fisher vector: theory and practice. Int. J. Comput. Vis. 105(3), 222–245 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  57. Uricár, M., Timofte, R., Rothe, R., Matas, J., Van Gool, L.: Structured output svm prediction of apparent age, gender and smile from deep features. In: Proceedings CVPRW, pp. 25–33 (2016)

  58. Wang, J.G., Li, J., Yau, W.Y., Sung, E.: Boosting dense SIFT descriptors and shape contexts of face images for gender recognition. In: IEEE Workshop on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 96–102 (2010)

  59. Yang, X., Gao, B.-B., Xing, C., Huo, Z.W., Wei, X.S., Zhou, Y., Wu, J., Geng, X.: Deep label distribution learning for apparent age estimation. In: The IEEE International Conference on Computer Vision (ICCV) Workshops, December 2015

  60. Yun, F., Huang, T.S.: Human age estimation with regression on discriminative aging manifold. IEEE Trans. Multimed. 10(4), 578–584 (2008)

    Article  MathSciNet  Google Scholar 

  61. Zhang, K., Gao, C., Guo, L., Sun, M., Yuan, X., Han, T.X., Zhao, Z., Li, B.: Age group and gender estimation in the wild with deep ror architecture. IEEE Access 5, 22492–22503 (2017)

    Article  Google Scholar 

  62. Zhang, K., Sun, M., Han, X., Yuan, X., Guo, L., Liu, T.: Residual networks of residual networks: multilevel residual networks. IEEE Trans. Circuits Syst. Video Technol. 28, 1303–1314 (2017)

    Article  Google Scholar 

  63. Zhu, Y., Li, Y., Mu, G., Guo, G.: A study on apparent age estimation. In: The IEEE International Conference on Computer Vision (ICCV) Workshops, December 2015

Download references

Acknowledgements

Lorenzo Seidenari is partially supported by “THE SOCIAL MUSEUM AND SMART TOURISM”, MIUR project no. CTN01_00034_23154_SMST.

Author information

Authors and Affiliations

  1. Lastminute.com group, Chiasso, Switzerland

    Lorenzo Seidenari, Alessandro Rozza & Alberto Del Bimbo

Authors
  1. Lorenzo Seidenari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alessandro Rozza
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alberto Del Bimbo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lorenzo Seidenari.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was performed when Dr. Rozza was affiliated to another company.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Seidenari, L., Rozza, A. & Del Bimbo, A. Real-time demographic profiling from face imagery with Fisher vectors. Machine Vision and Applications 30, 359–374 (2019). https://doi.org/10.1007/s00138-018-0991-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-018-0991-2

Keywords

Search

Navigation