Context-Aided Human Recognition – Clustering

  • Yang Song
  • Thomas Leung
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3953)


Context information other than faces, such as clothes, picture-taken-time and some logical constraints, can provide rich cues for recognizing people. This aim of this work is to automatically cluster pictures according to person’s identity by exploiting as much context information as possible in addition to faces. Toward that end, a clothes recognition algorithm is first developed, which is effective for different types of clothes (smooth or highly textured). Clothes recognition results are integrated with face recognition to provide similarity measurements for clustering. Picture-taken-time is used when combining faces and clothes, and the cases of faces or clothes missing are handled in a principle way. A spectral clustering algorithm which can enforce hard constraints (positive and negative) is presented to incorporate logic-based cues (e.g. two persons in one picture must be different individuals) and user feedback. Experiments on real consumer photos show the effectiveness of the algorithm.


Face Recognition Context Information Face Detection Spectral Cluster Color Histogram 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Fergus, R., Perona, P., Zisserman, A.: Object class recognition by unsupervised scale-invariant learning. In: CVPR (2003)Google Scholar
  2. 2.
    Ioffe, S.: Red eye detection with machine learning. In: Proc. ICIP (2003)Google Scholar
  3. 3.
    Ioffe, S.: Probabilistic linear discriminant analysis. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 531–542. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  4. 4.
    Jordan, M.I., Jacobs, R.A.: Hierarchical mixtures of experts and the em algorithm. Neural Computation 6, 181–214 (1994)CrossRefGoogle Scholar
  5. 5.
    Leung, T.: Texton correlation for recognition. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 203–214. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  6. 6.
    Lowe, D.: Object recognition from local scale-invariant features. In: ICCV (1999)Google Scholar
  7. 7.
    Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. In: CVPR (2003)Google Scholar
  8. 8.
    Ng, A.Y., Jordan, M.I., Weiss, Y.: On spectral clustering: Analysis and an algorithm. In: NIPS 14 (2002)Google Scholar
  9. 9.
    Schneiderman, H., Kanade, T.: A statistical method for 3d object detection applied to faces and cars. In: Proc. CVPR (2000)Google Scholar
  10. 10.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. In: Proc. CVPR, June 1997, pp. 731–737 (1997)Google Scholar
  11. 11.
    Sivic, J., Zisserman, A.: Video google: A text retrieval approach to object matching in videos. In: Proc. ICCV (2003)Google Scholar
  12. 12.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proc. CVPR (2001)Google Scholar
  13. 13.
    Wagstaff, K., Cardie, C., Rogers, S., Schroedl, S.: Contrained k-means clustering with background knowledge. In: Proc. ICML (2001)Google Scholar
  14. 14.
    Weiss, Y.: Segmentation using eigenvectors. In: Proc. ICCV (1999)Google Scholar
  15. 15.
    Yu, S.X.: Computational Models of Perceptual Organization, Ph.d. thesis, Carnegie Mellon University (2003)Google Scholar
  16. 16.
    Yu, S.X., Shi, J.: Grouping with bias. In: NIPS (2001)Google Scholar
  17. 17.
    Yu, S.X., Shi, J.: Multiclass spectral clustering. In: Proc. ICCV (2003)Google Scholar
  18. 18.
    Zhang, L., Chen, L., Li, M., Zhang, H.: Automated annotation of human faces in family albums. In: MM 2003 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Yang Song
    • 1
  • Thomas Leung
    • 1
  1. 1.Fujifilm Software (California), Inc.San JoseUSA

Personalised recommendations