Advertisement

Recognition of Facial Attributes Using Adaptive Sparse Representations of Random Patches

  • Domingo MeryEmail author
  • Kevin Bowyer
Conference paper
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8926)

Abstract

It is well known that some facial attributes –like soft biometric traits– can increase the performance of traditional biometric systems and help recognition based on human descriptions. In addition, other facial attributes –like facial expressions– can be used in human–computer interfaces, image retrieval, talking heads and human emotion analysis. This paper addresses the problem of automated recognition of facial attributes by proposing a new general approach called Adaptive Sparse Representation of Random Patches (ASR+). In the learning stage, random patches are extracted from representative face images of each class (e.g., in gender recognition –a two-class problem–, images of females/males) in order to construct representative dictionaries. In the testing stage, random test patches of the query image are extracted, and for each test patch a dictionary is built concatenating the ‘best’ representative dictionary of each class. Using this adapted dictionary, each test patch is classified following the Sparse Representation Classification (SRC) methodology. Finally, the query image is classified by patch voting. Thus, our approach is able to learn a model for each recognition task dealing with a larger degree of variability in ambient lighting, pose, expression, occlusion, face size and distance from the camera. Experiments were carried out on seven face databases in order to recognize facial expression, gender, race and disguise. Results show that ASR+ deals well with unconstrained conditions, outperforming various representative methods in the literature in many complex scenarios.

Keywords

Sparse representation Soft biometrics Gender recognition Race recognition Facial expression recognition 
Download to read the full conference paper text

References

  1. 1.
    Ahonen, T., Hadid, A., Pietikainen, M.: Face description with local binary patterns Application to face recognition. IEEE Trans. on Pattern Analysis and Machine Intelligence 28(12), 2037–2041 (2006)Google Scholar
  2. 2.
    Alexandre, L.A.: Gender recognition: A multiscale decision fusion approach. Pattern Recognition Letters 31(11), 1422–1427 (2010)CrossRefGoogle Scholar
  3. 3.
    Baluja, S., Rowley, H.A.: Boosting sex identification performance. International Journal of Computer Vision 71(1), 111–119 (2007)CrossRefGoogle Scholar
  4. 4.
    Bartlett, M.S., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., Movellan, J.: Recognizing facial expression: machine learning and application to spontaneous behavior. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2005) (2005)Google Scholar
  5. 5.
    Bekios-Calfa, J., Buenaposada, J.M., Baumela, L.: Robust gender recognition by exploiting facial attributes dependencies. Pattern Recognition Letters 36, 228–234 (2014)CrossRefGoogle Scholar
  6. 6.
    Boiman, O., Shechtman, E., Irani, M.: In defense of nearest-neighbor based image classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2008) (2008)Google Scholar
  7. 7.
    Chen, Y., Do, T.T., Tran, T.D.: Robust face recognition using locally adaptive sparse representation. In: IEEE International Conference on Image Processing (ICIP 2010), pp. 1657–1660 (2010)Google Scholar
  8. 8.
    Fu, S., He, H., Hou, Z.: Learning race from face: A survey. IEEE Trans. on Pattern Analysis and Machine Intelligence 36(12), 2483–2509 (2014)CrossRefGoogle Scholar
  9. 9.
    Fu, Y., Guo, G., Huang, T.S.: Age synthesis and estimation via faces: A survey. IEEE Trans. on Pattern Analysis and Machine Intelligence 32(11), 1955–1976 (2010)Google Scholar
  10. 10.
    Gallagher, A.C., Chen, T.: Understanding images of groups of people. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2009), pp. 256–263 (2009)Google Scholar
  11. 11.
    Guo, G., Mu, G.: A study of large-scale ethnicity estimation with gender and age variations. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2010), pp. 79–86 (2010)Google Scholar
  12. 12.
    Guo, G., Mu, G.: Joint estimation of age, gender and ethnicity: CCA vs. PLS. In: IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG 2013), pp. 1–6. IEEE (2013)Google Scholar
  13. 13.
    Kyperountas, M., Tefas, A., Pitas, I.: Salient feature and reliable classifier selection for facial expression classification. Pattern Recognition 43(3), 972–986 (2010)CrossRefzbMATHGoogle Scholar
  14. 14.
    Liang, D., Yang, J., Zheng, Z., Chang, Y.: A facial expression recognition system based on supervised locally linear embedding. Pattern Recognition Letters 26(15), 2374–2389 (2005)CrossRefGoogle Scholar
  15. 15.
    Liu, P., Han, S., Men, Z., Tong, Y.: Facial expression recognition via a boosted deep belief network. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2014) (2014)Google Scholar
  16. 16.
    Lu, L., Shi, P.: Fusion of multiple facial regions for expression-invariant gender classification. IEICE Electronics Express 6(10), 587–593 (2009)CrossRefGoogle Scholar
  17. 17.
    Lu, X., Jain, A.K.: Ethnicity identification from face images. In: Proceedings of SPIE Defense and Security Symposium, pp. 114–123 (2004)Google Scholar
  18. 18.
    Lucey, P., Cohn, J., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended cohn-kanade dataset (CK+): A complete dataset for action unit and emotion-specified expression. In: IEEE Workshop on CVPR for Human Communicative Behavior Analysis (2010)Google Scholar
  19. 19.
    Lyons, M.J., Budynek, J., Akamatsu, S.: Automatic classification of single facial images. IEEE Trans. on Pattern Analysis and Machine Intelligence 21(12), 1357–1362 (1999)Google Scholar
  20. 20.
    Martinez, A., Benavente, R.: The AR face database (June 1998). cVC Tech. Rep., No. 24Google Scholar
  21. 21.
    Moghaddam, B., Yang, M.H.: Learning gender with support faces. IEEE Trans. on Pattern Analysis and Machine Intelligence 24(5), 707–711 (2002)Google Scholar
  22. 22.
    Moon, H., Sharma, R., Jung, N.: Method and system for robust human ethnicity recognition using image feature-based probabilistic graphical models (2013). US Patent 8,379,937Google Scholar
  23. 23.
    Ng, C.B., Tay, Y.H., Goi, B.-M.: Recognizing human gender in computer vision: A survey. In: Anthony, P., Ishizuka, M., Lukose, D. (eds.) PRICAI 2012. LNCS, vol. 7458, pp. 335–346. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  24. 24.
    Phillips, P.J., Moon, H., Rizvi, S.A., Rauss, P.J.: The feret evaluation methodology for face-recognition algorithms. IEEE Trans. on Pattern Analysis and Machine Intelligence 22(10), 1090–1104 (2000)Google Scholar
  25. 25.
    Reid, D.A., Samangooei, S., Chen, C., Nixon, M.S., Ross, A.: Soft biometrics for surveillance: An overview. In: Handbook of Statistics, vol. 31, pp. 1–27. Elsevier (2013)Google Scholar
  26. 26.
    Salah, S.H., Du, H., Al-Jawad, N.: Fusing local binary patterns with wavelet features for ethnicity identification. In: Proceedings of IEEE International Conference on Signal and Image Processing (ICSIP 2013), pp. 330–336 (2013)Google Scholar
  27. 27.
    Samangooei, S., Guo, B., Nixon, M.S.: The use of semantic human description as a soft biometric. In: IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS 2008), pp. 1–7 (2008)Google Scholar
  28. 28.
    Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on local binary patterns: A comprehensive study. Image and Vision Computing 27(6), 803–816 (2009)CrossRefGoogle Scholar
  29. 29.
    Sivic, J., Zisserman, A.: Video Google: A text retrieval approach to object matching in videos. In: International Conference on Computer Vision (ICCV 2003), pp. 1470–1477 (2003)Google Scholar
  30. 30.
    Tosic, I., Frossard, P.: Dictionary learning. IEEE Signal Processing Magazine 28(2), 27–38 (2011)CrossRefGoogle Scholar
  31. 31.
    Vedaldi, A., Fulkerson, B.: VLfeat: An open and portable library of computer vision algorithms. In: MM 2010: Proceedings of the international conference on Multimedia, pp. 1469–1472, New York (October 2010)Google Scholar
  32. 32.
    Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. IEEE Trans. on Pattern Analysis and Machine Intelligence 31(2), 210–227 (2009)Google Scholar
  33. 33.
    Xu, Y., Zhang, D., Yang, J., Yang, J.Y.: A Two-Phase Test Sample Sparse Representation Method for Use With Face Recognition. IEEE Trans. on Circuits and Systems for Video Technology 21(9), 1255–1262 (2011)Google Scholar
  34. 34.
    Yang, Z., Li, M., Ai, H.: An experimental study on automatic face gender classification. In: 18th International Conference on Pattern Recognition (ICPR 2006), vol. 3, pp. 1099–1102 (2006)Google Scholar
  35. 35.
    Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Trans. on Pattern Analysis and Machine Intelligence 31(1), 39–58 (2009)Google Scholar
  36. 36.
    Zhong, L., Liu, Q., Yang, P., Liu, B., Huang, J., Metaxas, D.N.: Learning active facial patches for expression analysis. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2012) (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Department of Computer SciencePontificia Universidad Católica de ChileSantiagoChile
  2. 2.Department of Computer Science and EngineeringUniversity of Notre DameSouth BendUSA

Personalised recommendations

Cite paper