Skip to main content
SpringerLink
Log in
Book cover

Computer Vision pp 1–13Cite as

Facial Attribute Recognition: A Survey

  • Living reference work entry
  • First Online:

Definition

We present a survey of attribute recognition research in the computer vision community over the past decade. Most of our attention is given to facial attributes, but attributes of objects, pedestrians, and actions are considered as well.

Background

Facial attributes – human-describable features of faces – were introduced to the computer vision community in 2008, with their first application being image search [1]. Kumar et al. identified a problem with the image search engines of the time, realizing that simple descriptive search terms would not produce expected face image results. Attributes were then used for face recognition and verification as well as, again, for image search and retrieval [2, 3, 4] before attribute recognition itself became the focus of research. Facial attribute recognition is related to the problem of soft biometrics [5], which is focused on identification using these so-called soft traits rather than recognizing them for purely descriptive purposes.

W...

This is a preview of subscription content, log in via an institution.

References

  1. Kumar N, Belhumeur P, Nayar S (2008) Facetracer: a search engine for large collections of images with faces. In: European conference on computer vision. Springer, pp 340–353

    Google Scholar 

  2. Kumar N, Berg A, Belhumeur P, Nayar S (2009) Attribute and simile classifiers for face verification. In: International conference on computer vision. IEEE, pp 365–372

    Google Scholar 

  3. Kumar N, Berg A, Belhumeur PN, Nayar S (2011) Describable visual attributes for face verification and image search. IEEE Trans Pattern Anal Mach Intell 33(10):1962–1977

    Article  Google Scholar 

  4. Siddiquie B, Feris RS, Davis LS (2011) Image ranking and retrieval based on multi-attribute queries. In: CVPR 2011, pp 801–808

    Google Scholar 

  5. Jain AK, Dass SC, Nandakumar K (2004) Can soft biometric traits assist user recognition? In: Biometric technology for human identification. International Society for Optics and Photonics, vol 5404, pp 561–573

    Google Scholar 

  6. Ng CB, Tay YH, Goi B-M (2012) Recognizing human gender in computer vision: a survey. In: Pacific rim international conference on artificial intelligence. Springer, pp 335–346

    Google Scholar 

  7. Ramanathan N, Chellappa R, Biswas S (2009) Computational methods for modeling facial aging: a survey. J Vis Lang Comput 20(3):131–144

    Article  Google Scholar 

  8. Fu Y, Guo G, Huang TS (2010) Age synthesis and estimation via faces: a survey. IEEE Trans Pattern Anal Mach Intell 32(11):1955–1976

    Article  Google Scholar 

  9. Golomb BA, Lawrence DT, Sejnowski TJ (1990) Sexnet: a neural network identifies sex from human faces. In: NIPS, vol 1, p 2

    Google Scholar 

  10. Cottrell GW, Metcalfe J (1991) EMPATH: face, emotion, and gender recognition using holons. In: Advances in neural information processing systems, pp 564–571

    Google Scholar 

  11. Gao W, Ai H (2009) Face gender classification on consumer images in a multiethnic environment. In: International conference on biometrics. Springer, pp 169–178

    Google Scholar 

  12. BenAbdelkader C, Griffin P (2005) A local region-based approach to gender classification from face images. In: 2005 IEEE computer society conference on computer vision and pattern recognition-workshops, CVPR Workshops. IEEE, p 52

    Google Scholar 

  13. Guo G, Dyer CR, Fu Y, Huang TS (2009) Is gender recognition affected by age. In: 2009 IEEE 12th international conference on computer vision workshops (ICCV Workshops). IEEE, pp 2032–2039

    Google Scholar 

  14. Kwon YH, da Vitoria Lobo N (1999) Age classification from facial images. Comput Vis Image Underst 74(1):1–21

    Article  Google Scholar 

  15. Lanitis A, Taylor CJ, Cootes TF (2002) Toward automatic simulation of aging effects on face images. IEEE Trans Pattern Anal Mach Intell 24(4):442–455

    Article  Google Scholar 

  16. Geng X, Zhou Z-H, Zhang Y, Li G, Dai H (2006) Learning from facial aging patterns for automatic age estimation. In: Proceedings of the 14th ACM international conference on multimedia. ACM, pp 307–316

    Google Scholar 

  17. Ramanathan N, Chellappa R (2006) Face verification across age progression. IEEE Trans Image Process 15(11):3349–3361

    Article  Google Scholar 

  18. Zhang N, Paluri M, Ranzato M, Darrell T, Bourdev L (2014) Panda: pose aligned networks for deep attribute modeling. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1637–1644

    Google Scholar 

  19. Farhadi A, Endres I, Hoiem D, Forsyth D (2009) Describing objects by their attributes. In: 2009 IEEE conference on computer vision and pattern recognition CVPR 2009. IEEE, pp 1778–1785

    Google Scholar 

  20. Berg T, Berg A, Shih J (2010) Automatic attribute discovery and characterization from noisy web data. Eur Conf Comput Vis 6311:663–676

    Google Scholar 

  21. Liu J, Kuipers B, Savarese S (2011) Recognizing human actions by attributes. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 3337–3344

    Google Scholar 

  22. Scheirer WJ, Kumar N, Belhumeur PN, Boult TE (2012) Multi-attribute spaces: calibration for attribute fusion and similarity search. In: 2012 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 2933–2940

    Google Scholar 

  23. Newton C (2016) Facebook begins using artificial intelligence to describe photos to blind users. [Online; posted 5-April-2016].

    Google Scholar 

  24. Samangouei P, Patel VM, Chellappa R (2015) Attribute-based continuous user authentication on mobile devices. In: 2015 IEEE 7th international conference on biometrics theory, applications and systems (BTAS). IEEE, pp 1–8

    Google Scholar 

  25. Samangouei P, Chellappa R (2016) Convolutional neural networks for attribute-based active authentication on mobile devices. In: 2016 IEEE 8th international conference on biometrics theory, applications and systems (BTAS). IEEE, pp 1–8

    Google Scholar 

  26. Samangouei P, Hand E, Patel VM, Chellappa R (2017) Active authentication using facial attributes. Mobile Biometrics 3:131

    Google Scholar 

  27. Russakovsky O, Fei-Fei L (2012) Attribute learning in large-scale datasets. In: Kutulakos KN (ed) Trends and topics in computer vision. Springer, Berlin/Heidelberg, pp 1–14

    Google Scholar 

  28. Jojic N, Caspi Y (2004) Capturing image structure with probabilistic index maps. In: Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition, CVPR 2004, vol 1, pp I–I

    Google Scholar 

  29. Lazebnik S, Schmid C, Ponce J (2005) A sparse texture representation using local affine regions. IEEE Trans Pattern Anal Mach Intell 27:1265–1278

    Article  Google Scholar 

  30. Liu Y, Tsin Y, Lin W-C (2005) The promise and perils of near-regular texture. Int J Comput Vis 62:145–159

    Article  Google Scholar 

  31. Ferrari V, Zisserman A (2008) Learning visual attributes. In: Platt JC, Koller D, Singer Y, Roweis ST (eds) Advances in neural information processing systems 20. Curran Associates, Inc., pp 433–440

    Google Scholar 

  32. Parikh D, Grauman K (2011) Interactively building a discriminative vocabulary of nameable attributes. In: CVPR 2011, pp 1681–1688

    Google Scholar 

  33. Wang X, Ji Q (2013) A unified probabilistic approach modeling relationships between attributes and objects. In: 2013 IEEE international conference on computer vision, pp 2120–2127

    Google Scholar 

  34. Sun Y, Bo L, Fox D (2013) Attribute based object identification. In: 2013 IEEE international conference on robotics and automation, pp 2096–2103

    Google Scholar 

  35. Vedaldi A, Mahendran S, Tsogkas S, Maji S, Girshick R, Kannala J, Rahtu E, Kokkinos I, Blaschko MB, Weiss D, Taskar B, Simonyan K, Saphra N, Mohamed S (2014) Understanding objects in detail with fine-grained attributes. In: 2014 IEEE conference on computer vision and pattern recognition, pp 3622–3629

    Google Scholar 

  36. Wang X, Ji Q (2016) Object recognition with hidden attributes. In: Proceedings of the twenty-fifth international joint conference on artificial intelligence, IJCAI’16. AAAI Press, pp 3498–3504

    Google Scholar 

  37. Jhuang H, Serre T, Wolf L, Poggio T (2007) A biologically inspired system for action recognition. In: 2007 IEEE 11th international conference on computer vision, pp 1–8

    Google Scholar 

  38. Yao B, Jiang X, Khosla A, Lin AL, Guibas L, Fei-Fei L (2011) Human action recognition by learning bases of action attributes and parts. In: 2011 International conference on computer vision, pp 1331–1338

    Google Scholar 

  39. Sharma G, Jurie F, Schmid C (2013) Expanded parts model for human attribute and action recognition in still images. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition (CVPR)

    Google Scholar 

  40. Zhang Z, Wang C, Xiao B, Zhou W, Liu S (2013) Attribute regularization based human action recognition. IEEE Trans Inf Forensics Secur 8(10):1600–1609

    Article  Google Scholar 

  41. Tahmoush D (2015) Applying action attribute class validation to improve human activity recognition

    Google Scholar 

  42. Cai X, Zhou W, Li H (2015) Attribute mining for scalable 3D human action recognition. In: Proceedings of the 23rd ACM international conference on multimedia

    Google Scholar 

  43. Chen K, Ding G, Han J (2017) Attribute-based supervised deep learning model for action recognition. Front Comput Sci. abs/1707.09468, pp 219–229

    Google Scholar 

  44. Khan FS, van de Weijer J, Anwer RM, Bagdanov AD, Felsberg M, Laaksonen J (2018) Scale coding bag of deep features for human attribute and action recognition. Mach Vis Appl 29:55–71

    Article  Google Scholar 

  45. Zaki MH, Sayed T (2014) Using automated walking gait analysis for the identification of pedestrian attributes. Transp Res Part C: Emerg Technol 48: 16–36

    Article  Google Scholar 

  46. Deng Y, Luo P, Loy CC, Tang X (2014) Pedestrian attribute recognition at far distance. In: Proceedings of the 22nd ACM international conference on multimedia, MM ’14, pp 789–792

    Google Scholar 

  47. Deng Y, Luo P, Loy CC, Tang X (2015) Learning to recognize pedestrian attribute. CoRR abs/1501.00901

    Google Scholar 

  48. Zhu J, Liao S, Yi D, Lei Z, Li SZ (2015) Multi-label CNN based pedestrian attribute learning for soft biometrics

    Google Scholar 

  49. Yu K, Leng B, Zhang Z, Li D, Huang K (2016) Weakly-supervised learning of mid-level features for pedestrian attribute recognition and localization. CoRR, vol abs/1611.05603, pp 224–229

    Google Scholar 

  50. Lu Y, Kumar A, Zhai S, Cheng Y, Javidi T, Feris RS (2016) Fully-adaptive feature sharing in multi-task networks with applications in person attribute classification. CoRR, vol abs/1611.05377

    Google Scholar 

  51. Zhao X, Sang L, Ding G, Guo Y, Jin X (2018) Grouping attribute recognition for pedestrian with joint recurrent learning. In: Proceedings of the twenty-seventh international joint conference on artificial intelligence, IJCAI-18, International joint conferences on artificial intelligence organization, 7, pp 3177–3183

    Google Scholar 

  52. Chen Z, Li A, Wang Y (2019) Video-based pedestrian attribute recognition. CoRR, vol abs/1901.05742

    Google Scholar 

  53. Vaquero DA, Feris RS, Tran D, Brown L, Hampapur A, Turk M (2009) Attribute-based people search in surveillance environments. In: 2009 workshop on applications of computer vision (WACV). IEEE, pp 1–8

    Google Scholar 

  54. Huang GB, Mattar M, Berg T, E. Learned-Miller (2008) Labeled faces in the wild: a database forstudying face recognition in unconstrained environments. In: Workshop on faces in ‘Real-Life ’Images: detection, alignment, and recognition

    Google Scholar 

  55. Parikh D, Grauman K (2011) Relative attributes. In: 2011 IEEE international conference on computer vision (ICCV). IEEE, pp 503–510

    Google Scholar 

  56. Berg T, Belhumeur PN (2012) Tom-vs-pete classifiers and identity-preserving alignment for face verification. In: BMVC, Citeseer, vol 2, p 7

    Google Scholar 

  57. Liu Z, Luo P, Wang X, Tang X (2015) Deep learning face attributes in the wild. In: Proceedings of the ieee international conference on computer vision, pp 3730–3738

    Google Scholar 

  58. Hand EM, Chellappa R (2017) Attributes for improved attributes: a multi-task network utilizing implicit and explicit relationships for facial attribute classification. In: AAAI, pp 4068–4074

    Google Scholar 

  59. Wang J, Cheng Y, R. Schmidt Feris (2016) Walk and learn: facial attribute representation learning from egocentric video and contextual data. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2295–2304

    Google Scholar 

  60. Rudd EM, Günther M, Boult TE (2016) Moon: a mixed objective optimization network for the recognition of facial attributes. In: European conference on computer vision. Springer, pp 19–35

    Google Scholar 

  61. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

    Google Scholar 

  62. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 248–255

    Google Scholar 

  63. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

    Google Scholar 

  64. Ehrlich M, Shields TJ, Almaev T, Amer MR (2016) Facial attributes classification using multi-task representation learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 47–55

    Google Scholar 

  65. Rozsa A, Günther M, Rudd EM, Boult TE (2016) Are facial attributes adversarially robust? In: 2016 23rd International conference on pattern recognition (ICPR). IEEE, pp 3121–3127

    Google Scholar 

  66. Günther M, Rozsa A, Boult TE (2017) Affact: alignment-free facial attribute classification technique. In: 2017 IEEE international joint conference on biometrics (IJCB). IEEE, pp 90–99

    Google Scholar 

  67. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

    Google Scholar 

  68. Ding H, Zhou H, Zhou SK, Chellappa R (2018) A deep cascade network for unaligned face attribute classification. In: Thirty-second AAAI conference on artificial intelligence

    Google Scholar 

  69. Kalayeh MM, Gong B, Shah M (2017) Improving facial attribute prediction using semantic segmentation. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 4227–4235

    Google Scholar 

  70. Le V, Brandt J, Lin Z, Bourdev L, Huang TS (2012) Interactive facial feature localization. In: European conference on computer vision. Springer, pp 679–692

    Google Scholar 

  71. He K, Fu Y, Zhang W, Wang C, Jiang Y-G, Huang F, Xue X (2018) Harnessing synthesized abstraction images to improve facial attribute recognition. In: IJCAI, pp 733–740

    Google Scholar 

  72. Hand EM, Castillo C, Chellappa R (2018) Doing the best we can with what we have: multi-label balancing with selective learning for attribute prediction. AAAI, New Orleans

    Google Scholar 

  73. Hand EM, Castillo CD, Chellappa R (2018) Predicting facial attributes in video using temporal coherence and motion-attention. In: 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 84–92

    Google Scholar 

  74. Wolf L, Hassner T, Maoz I (2011) Face recognition in unconstrained videos with matched background similarity. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 529–534

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Nevada, Reno, NV, USA

    Nathan Thom & Emily M. Hand

Authors
  1. Nathan Thom
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Emily M. Hand
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Emily M. Hand .

Section Editor information

  1. University of Maryland, College Park, USA

    Rama Chellappa

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Thom, N., Hand, E.M. (2020). Facial Attribute Recognition: A Survey. In: Computer Vision. Springer, Cham. https://doi.org/10.1007/978-3-030-03243-2_815-1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-03243-2_815-1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-03243-2

  • Online ISBN: 978-3-030-03243-2

  • eBook Packages: Springer Reference Computer SciencesReference Module Computer Science and Engineering

Publish with us

Policies and ethics

Search

Navigation