Combining Language Sources and Robust Semantic Relatedness for Attribute-Based Knowledge Transfer

  • Marcus Rohrbach
  • Michael Stark
  • György Szarvas
  • Bernt Schiele
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6553)


Knowledge transfer between object classes has been identified as an important tool for scalable recognition. However, determining which knowledge to transfer where remains a key challenge. While most approaches employ varying levels of human supervision, we follow the idea of mining linguistic knowledge bases to automatically infer transferable knowledge. In contrast to previous work, we explicitly aim to design robust semantic relatedness measures and to combine different language sources for attribute-based knowledge transfer. On the challenging Animals with Attributes (AwA) data set, we report largely improved attribute-based zero-shot object class recognition performance that matches the performance of human supervision.


Knowledge Transfer Semantic Relatedness Object Class Language Source Solid Blue Curve 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ferrari, V., Zisserman, A.: Learning visual attributes. In: NIPS (2007)Google Scholar
  2. 2.
    Lampert, C.H., Nickisch, H., Harmeling, S.: Learning to detect unseen object classes by between-class attribute transfer. In: CVPR (2009)Google Scholar
  3. 3.
    Wang, G., Forsyth, D.: Joint learning of visual attributes, object classes and visual saliency. In: ICCV (2009)Google Scholar
  4. 4.
    Rohrbach, M., Stark, M., Szarvas, G., Gurevych, I., Schiele, B.: What helps where – and why? semantic relatedness for knowledge transfer. In: CVPR (2010)Google Scholar
  5. 5.
    Fink, M.: Object classification from a single example utilizing class relevance pseudo-metrics. In: NIPS (2004)Google Scholar
  6. 6.
    Marszalek, M., Schmid, C.: Semantic hierarchies for visual object recognition. In: CVPR (2007)Google Scholar
  7. 7.
    Zweig, A., Weinshall, D.: Exploiting object hierarchy: Combining models from different category levels. In: ICCV (2007)Google Scholar
  8. 8.
    Bart, E., Ullman, S.: Cross-generalization: Learning novel classes from a single example by feature replacement. In: CVPR (2005)Google Scholar
  9. 9.
    Thrun, S.: Is learning the n-th thing any easier than learning the first. In: NIPS 1996 (1996)Google Scholar
  10. 10.
    Fei-Fei, L., Fergus, R., Perona, P.: One-shot learning of object categories. PAMI 28 (2006)Google Scholar
  11. 11.
    Stark, M., Goesele, M., Schiele, B.: A shape-based object class model for knowledge transfer. In: ICCV (2009)Google Scholar
  12. 12.
    Kumar, N., Berg, A.C., Belhumeur, P.N., Nayar, S.K.: Attribute and simile classifiers for face verification. In: ICCV (2009)Google Scholar
  13. 13.
    Farhadi, A., Endres, I., Hoiem, D.: Attribute-centeric recognition for cross-category generalization. In: CVPR (2010)Google Scholar
  14. 14.
    Palatucci, M., Pomerleau, D., Hinton, G., Mitchell, T.: Zero-shot learning with semantic output codes. In: NIPS (2009)Google Scholar
  15. 15.
    Wang, J., Markert, K., Everingham, M.: Learning models for object recognition from natural language descriptions (2009)Google Scholar
  16. 16.
    Bart, E., Ullman, S.: Single-example learning of novel classes using representation by similarity. In: BMVC (2005)Google Scholar
  17. 17.
    Chen, H.H., Lin, M.S., Wei, Y.C.: Novel association measures using web search with double checking. In: ACL-44 (2006)Google Scholar
  18. 18.
    Delezoide, B., Pitel, G., Borgne, H.L., Greffenstette, G., Moëllic, P.A., Millet, C.: Object/background scene classification in photographs using linguistic statistics from the web. In: OntoImage (2008)Google Scholar
  19. 19.
    Kemp, C., Tenenbaum, J.B., Griffiths, T.L., Yamada, T., Ueda, N.: Learning systems of concepts with an infinite relational model. In: AAAI (2006)Google Scholar
  20. 20.
    Osherson, D.N., Stern, J., Wilkie, O., Stob, M., Smith, E.E.: Default probability. Cognitive Science 15, 251–269 (1991)CrossRefGoogle Scholar
  21. 21.
    Wu, T.F., Lin, C.J., Weng, R.C.: Probability estimates for multi-class classification by pairwise coupling. JMLR 2004 (2004)Google Scholar
  22. 22.
    Fellbaum, C.: WordNet: An Electronical Lexical Database. The MIT Press (1998)Google Scholar
  23. 23.
    Lin, D.: An information-theoretic definition of similarity. In: ICML (1998)Google Scholar
  24. 24.
    Zesch, T., Gurevych, I.: Wisdom of crowds versus wisdom of linguists - measuring the semantic relatedness of words. JNLE 16 (2010)Google Scholar
  25. 25.
    Berland, M., Charniak, E.: Finding parts in very large corpora. In: ACL (1999)Google Scholar
  26. 26.
    Pirrò, G., Seco, N.: Design, implementation and evaluation of a new semantic similarity metric combining features and intrinsic information content. In: ODBASE (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Marcus Rohrbach
    • 1
    • 2
  • Michael Stark
    • 1
    • 2
  • György Szarvas
    • 1
  • Bernt Schiele
    • 1
    • 2
  1. 1.Department of Computer ScienceTU DarmstadtGermany
  2. 2.Max Planck Institute for InformaticsSaarbrückenGermany

Personalised recommendations