Semi-supervised Learning of Facial Attributes in Video

  • Neva Cherniavsky
  • Ivan Laptev
  • Josef Sivic
  • Andrew Zisserman
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6553)


In this work we investigate a weakly-supervised approach to learning facial attributes of humans in video. Given a small set of images labeled with attributes and a much larger unlabeled set of video tracks, we train a classifier to recognize these attributes in video data. We make two contributions. First, we show that training on video data improves classification performance over training on images alone. Second, and more significantly, we show that tracks in video provide a natural mechanism for generalizing training data – in this case to new poses, lighting conditions and expressions. The advantage of our method is demonstrated on the classification of gender and age attributes in the movie “Love, Actually”. We show that the semi-supervised approach adds a significant performance boost, for example for gender increasing average precision from 0.75 on static images alone to 0.85.


Facial Feature Video Data Average Precision Point Track Video Track 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Kumar, N., Belhumeur, P.N., Nayar, S.K.: FaceTracer: A Search Engine for Large Collections of Images with Faces. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 340–353. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  2. 2.
    Kumar, N., Berg, A.C., Belhumeur, P.N., Nayar, S.K.: Attribute and Simile Classifiers for Face Verification. In: Proc. International Conference on Computer Vision (2009)Google Scholar
  3. 3.
    Kenneth, A., Coltrane, S.: Gender displaying television commercials: A comparative study of television commercials in the 1950s to 1980s. Sex Roles 35 (1996)Google Scholar
  4. 4.
    Cour, T., Sapp, B., Jordan, C., Taskar, B.: Learning from ambiguously labeled images. In: Proc. Computer Vision and Pattern Recognition (2009)Google Scholar
  5. 5.
    Chapelle, O., Schölkopf, B., Zien, A. (eds.): Semi-Supervised Learning. MIT Press, Cambridge (2006)Google Scholar
  6. 6.
    Zhu, X.: Semi-supervised learning literature survey. Technical Report 1530, Computer Sciences, University of Wisconsin-Madison (2005)Google Scholar
  7. 7.
    Yan, R., Zhang, J., Yang, J., Hauptmann, A.G.: A discriminative learning framework with pairwise constraints for video object classification. PAMI 28 (2006)Google Scholar
  8. 8.
    Everingham, M., Sivic, J., Zisserman, A.: “Hello! My name is... Buffy” - automatic naming of characters in tv video. In: Proc. British Machine Vision Conference (2006)Google Scholar
  9. 9.
    Ramanan, D., Baker, S., Kakade, S.: Leveraging archival video for building face datasets. In: Proc. International Conference on Computer Vision (2007)Google Scholar
  10. 10.
    Sivic, J., Schaffalitzky, F., Zisserman, A.: Object level grouping for video shots. International Journal of Computer Vision 67, 189–210 (2006)CrossRefGoogle Scholar
  11. 11.
    Sivic, J., Everingham, M., Zisserman, A.: “Who are you?” - learning person specific classifiers from video. In: Proc. Computer Vision and Pattern Recognition (2009)Google Scholar
  12. 12.
    Bradski, G., Kaehler, A.: Learning OpenCV: Computer Vision with the OpenCV Library. O’Reilly, Cambridge (2008)Google Scholar
  13. 13.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proc. Computer Vision and Pattern Recognition (2001)Google Scholar
  14. 14.
    Shi, J., Tomasi, C.: Good features to track. In: Proceedings of the 1994 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 1994, pp. 593–600 (1994) Google Scholar
  15. 15.
    Felzenszwalb, P., Huttenlocher, D.: Pictorial structures for object recognition. International Journal of Computer Vision 61 (2005)Google Scholar
  16. 16.
    Swain, M.J., Ballard, D.H.: Color indexing. International Journal of Computer Vision 7, 11–32 (1991)CrossRefGoogle Scholar
  17. 17.
    Maji, S., Berg, A.: Max-margin additive models for detection. In: Proc. International Conference on Computer Vision (2009)Google Scholar
  18. 18.
    Fan, R., Chang, K., Hsieh, C., Wang, R., Lin, C.: LIBLINEAR: A library for large linear classification. Journal of Machine Learning Research 9, 1871–1874 (2008)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Neva Cherniavsky
    • 1
  • Ivan Laptev
    • 1
  • Josef Sivic
    • 1
  • Andrew Zisserman
    • 1
    • 2
  1. 1.Laboratoire d’Informatique de l’Ecole Normale Supérieure, ENS/INRIA/CNRS UMR 8548INRIA, WILLOWFrance
  2. 2.Dept. of Engineering ScienceUniversity of OxfordUK

Personalised recommendations