Multimedia Tools and Applications

, Volume 73, Issue 3, pp 1643–1661 | Cite as

Cross-modality based celebrity face naming for news image collections

  • Xueping SuEmail author
  • Jinye Peng
  • Xiaoyi Feng
  • Jun Wu
  • Jianping Fan
  • Li Cui


For automatically mining the underlying relationships between different famous persons in daily news, for example, building a news person based network with the faces as icons to facilitate face-based person finding, we need a tool to automatically label faces in new images with their real names. This paper studies the problem of linking names with faces from large-scale news images with captions. In our previous work, we proposed a method called Person-based Subset Clustering which is mainly based on face clustering for all face images derived from the same name. The location where a name appears in a caption, as well as the visual structural information within a news image provided informative cues such as who are really in the associated image. By combining the domain knowledge from the captions and the corresponding image we propose a novel cross-modality approach to further improve the performance of linking names with faces. The experiments are performed on the data sets including approximately half a million news images from Yahoo! news, and the results show that the proposed method achieves significant improvement over the clustering-only methods.


Affinity propagation cluster Cross-modality Face classification Rank aggregation 


  1. 1.
    Berg T, Berg A, Edwards J, Maire M, White R (2004) Names and faces in the news. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), page 848–854Google Scholar
  2. 2.
    Berg A, Berg A, Edwards J, Forsyth D (2005) Who’s in the picture. Adv Neural Inf Process Syst 17:137–144Google Scholar
  3. 3.
    Cunningham H, Maynard D, Bontcheva K, Tablan V (2002) Gate: a framework and graphical development environment for robust nlp tools and applications. In: 40th Anniversary Meeting of the Association for Computational Linguistics ( ACL), page 168–175Google Scholar
  4. 4.
    Frey BJ, Dueck D (2007) Clustering by passing messages between data points. Science 315:972–976CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Guillaumin M, Mensink T, Verbeek JJ, Schmid C (2008) Automatic face naming with caption-based supervision. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), page 1–8Google Scholar
  6. 6.
    Guillaumin M, Verbeek J, Schmid C (2010) Multiple instance metric learning from automatically labeled bags of faces. In: European Conference on Computer Vision (ECCV), page 634–647Google Scholar
  7. 7.
    Le D, Satoh S (2008) Unsupervised face annotation by mining the web. In: International Conference on Data Mining (ICDM), page 383–392Google Scholar
  8. 8.
    Mensink T, Verbeek J (2008) Improving people search using query expansions: how friends help to find people. In: European Conference on Computer Vision (ECCV), page 86–99Google Scholar
  9. 9.
    Mensink T, Verbeek J (2008) Improving people search using query expansions. In: European Conference on Computer Vision (ECCV), page 86–99Google Scholar
  10. 10.
    Ozkan D, Duygulu P (2010) Interesting faces: a graph-based approach for finding people in news. Pattern Recogn 43(5):1717–1735CrossRefGoogle Scholar
  11. 11.
    Peng Y, Ganesh A (2010) RASL: robust batch alignment of images by sparse and low-rank decomposition. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), page 763–770Google Scholar
  12. 12.
    Pham PT, Moens MF, Tuytelaars T (2010) Cross-media alignment of names and faces. IEEE Trans Multimed 12:13–27CrossRefGoogle Scholar
  13. 13.
    Pham PT, Tuytelaars T, Moens M-F (2010) Naming persons in news video with label propagation. IEEE Multimedia 18:44–55CrossRefGoogle Scholar
  14. 14.
    Poppe R (2012) Facing scalability: naming faces in an online social network. Pattern Recogn 45:2335–2347CrossRefGoogle Scholar
  15. 15.
    Su X-P, Peng J-Y, Feng X-Y, Wu J, Fan J-P (2011) Linking names and faces by person-based subset clustering. In: Proceedings of the Third International Conference on Internet Multimedia Computing and Service (ACM ICIMCS), page 120–123Google Scholar
  16. 16.
    Viola P, Jones M (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154CrossRefGoogle Scholar
  17. 17.
    Yang J, Chen M-Y, Hauptmann A (2004) Finding person x: correlating names with visual appearances. In: International Conference on Image and Video Retrieval, page 270–278Google Scholar
  18. 18.
    Zhang B, Shan S, Gao W, Chen X (2005) Local Gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition. In: International Conference on Computer Vision (ICCV), page 786–791Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Xueping Su
    • 1
    Email author
  • Jinye Peng
    • 1
  • Xiaoyi Feng
    • 1
  • Jun Wu
    • 1
  • Jianping Fan
    • 1
  • Li Cui
    • 1
  1. 1.School of Electronics and InformationNorthwestern Polytechnical UniversityXi’anChina

Personalised recommendations