Extracting Structures in Image Collections for Object Recognition

  • Sandra Ebert
  • Diane Larlus
  • Bernt Schiele
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6311)


Many computer vision methods rely on annotated image sets without taking advantage of the increasing number of unlabeled images available. This paper explores an alternative approach involving unsupervised structure discovery and semi-supervised learning (SSL) in image collections. Focusing on object classes, the first part of the paper contributes with an extensive evaluation of state-of-the-art image representations. Thus, it underlines the decisive influence of the local neighborhood structure and its direct consequences on SSL results and the importance of developing powerful object representations. In a second part, we propose and explore promising directions to improve results by looking at the local topology between images and feature combination strategies.


Local Structure Object Class Object Representation Image Representation Unlabeled Data 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Chum, O., Perdoch, M., Matas, J.: Geometric min-hashing: Finding a (thick) needle in a haystack. In: CVPR (2009)Google Scholar
  2. 2.
    Sivic, J., Russell, B., Zisserman, A., Freeman, W., Efros, A.: Unsupervised discovery of visual object class hierarchies. In: CVPR (2008)Google Scholar
  3. 3.
    Vijayanarasimhan, S., Grauman, K.: Keywords to visual categories: Multiple-instance learning for weakly supervised object categorization. In: CVPR (2008)Google Scholar
  4. 4.
    Li, L.-J., Fei-Fei, L.: Optimol: Automatic online picture collection via incremental model learning. IJCV (2009)Google Scholar
  5. 5.
    Vijayanarasimhan, S., Grauman, K.: Multi-level active prediction of useful image annotations for recognition. In: NIPS (2008)Google Scholar
  6. 6.
    Russell, B., Torralba, A., Murphy, K., Freeman, W.: LabelMe: A database and web-based tool for image annotation. IJCV (2008)Google Scholar
  7. 7.
    Verbeek, J., Triggs, B.: Scene segmentation with CRFs learned from partially labeled images. In: NIPS, vol. 20 (2008)Google Scholar
  8. 8.
    Fergus, R., Perona, P., Zisserman, A.: Weakly supervised scale-invariant learning of models for visual recognition. IJCV (2007)Google Scholar
  9. 9.
    Leistner, C., Saffari, A., Santner, J., Bischof, H.: Semi-supervised random forests. In: ICCV (2009)Google Scholar
  10. 10.
    Saffari, A., Leistner, C., Bischof, H.: Regularized multi-class semi-supervised boosting. In: CVPR (2009)Google Scholar
  11. 11.
    Fergus, R., Weiss, Y., Torralba, A.: Semi-supervised learning in gigantic image collections. In: NIPS (2009)Google Scholar
  12. 12.
    Chapelle, O., Schölkopf, B., Zien, A. (eds.): Semi-Supervised Learning. MIT Press, Cambridge (2006)Google Scholar
  13. 13.
    Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schölkopf, B.: Learning with local and global consistency. In: NIPS (2004)Google Scholar
  14. 14.
    Zhu, X., Ghahramani, Z., Lafferty, J.: Semi-supervised learning using gaussian fields and harmonic functions. In: ICML (2003)Google Scholar
  15. 15.
    Liu, W., Chang, S.: Robust multi-class transductive learning with graphs. In: CVPR (2009)Google Scholar
  16. 16.
    Leibe, B., Schiele, B.: Analyzing appearance and contour based methods for object categorization. In: CVPR (2003)Google Scholar
  17. 17.
    Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL VOC Challenge 2008 Results (2008)Google Scholar
  18. 18.
    Lampert, C., Nickisch, H., Harmeling, S.: Learning to detect unseen object classes by between-class attribute transfer. In: CVPR (2009)Google Scholar
  19. 19.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR (2005)Google Scholar
  20. 20.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: A holistic representation of the spatial envelope. IJCV (2001)Google Scholar
  21. 21.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: CVPR (2006)Google Scholar
  22. 22.
    Wolf, L., Hassner, T., Taigman, Y.: Descriptor based methods in the wild. In: ECCV (2008)Google Scholar
  23. 23.
    Mikolajczyk, K., Schmid, C.: Scale & affine invariant interest point detectors. IJCV (2004)Google Scholar
  24. 24.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. IJCV (2004)Google Scholar
  25. 25.
    van de Sande, K., Gevers, T., Snoek, C.: Evaluating color descriptors for object and scene recognition. PAMI (2010)Google Scholar
  26. 26.
    Hein, M., Maier, M.: Manifold denoising. In: NIPS (2006)Google Scholar
  27. 27.
    Bengio, Y., Delalleau, O., Le Roux, N.: Label propagation and quadratic criterion. In: [12]Google Scholar
  28. 28.
    Zhou, D., Huang, J., Schöhkopf, B.: Learning from labeled and unlabeled data on a directed graph. In: ICML (2005)Google Scholar
  29. 29.
    Perronnin, F., Liu, Y., Renders, J.: A family of contextual measures of similarity between distributions with application to image retrieval. In: CVPR (2009)Google Scholar
  30. 30.
    Gehler, P., Nowozin, S.: On feature combination for multiclass object classification. In: CVPR (2009)Google Scholar
  31. 31.
    Zhu, X.: Semi-supervised learning literature survey. Technical report, UW (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Sandra Ebert
    • 1
    • 2
  • Diane Larlus
    • 1
  • Bernt Schiele
    • 1
    • 2
  1. 1.Department of Computer ScienceTU DarmstadtGermany
  2. 2.MPI InformaticsSaarbruckenGermany

Personalised recommendations