Learning What and How of Contextual Models for Scene Labeling

  • Arpit Jain
  • Abhinav Gupta
  • Larry S. Davis
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6314)


We present a data-driven approach to predict the importance of edges and construct a Markov network for image analysis based on statistical models of global and local image features. We also address the coupled problem of predicting the feature weights associated with each edge of a Markov network for evaluation of context. Experimental results indicate that this scene dependent structure construction model eliminates spurious edges and improves performance over fully-connected and neighborhood connected Markov network.


Training Dataset Training Image Contextual Model Feature Weight Contextual Relationship 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Gupta, A., Davis, L.: Beyond Nouns: Exploiting Prepositions and Comparative Adjectives for Learning Visual Classifiers. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 16–29. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  2. 2.
    Carbonetto, P., Freitas, N., Barnard, K.: A statistical model for general contextual object recognition. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 350–362. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  3. 3.
    Divvala, S., Hoiem, D., Hays, J., Efros, A.A., Hebert, M.: An Empirical Study of Context in Object Detection. In: CVPR 2009 (2009)Google Scholar
  4. 4.
    Galleguillos, C., Rabinovich, A., Belongie, S.: Object Categorization using Co-Occurrence, Location and Appearance. In: CVPR 2008 (2008)Google Scholar
  5. 5.
    Li, J., Fei-Fei, L.: What, where and who? Classifying event by scene and object recognition. In: ICCV 2007 (2007)Google Scholar
  6. 6.
    He, X., Zemel, R.: Latent topic random fields: Learning using a taxonomy of labels. In: CVPR 2008 (2008)Google Scholar
  7. 7.
    Murphy, K., Torralba, A., Freeman, W.: Using the Forest to See the Trees:A Graphical Model Relating Features, Objects and Scenes. In: NIPS 2003 (2003)Google Scholar
  8. 8.
    Rabinovich, A., Vedaldi, A., Galleguillos, C., Wiewiora, E., Belongie, S.: Objects in Context. In: ICCV 2007 (2007)Google Scholar
  9. 9.
    Torralba, A., Murphy, K.P., Freeman, W.T.: Contextual Models for Object Detection using Boosted Random Fields. In: Adv. in Neural Information Processing Systems (NIPS), pp. 1401–1408 (2005)Google Scholar
  10. 10.
    Friedman, N.: The Bayesian structural EM algorithm. In: UAI 1998 (1998)Google Scholar
  11. 11.
    McDowell, L.K., Gupta, K., Aha, D.: Cautious Inference in Collective Classification. In: AAAI 2007 (2007)Google Scholar
  12. 12.
    Neville, J., Jensen, D.: Iterative Classification in Relational Data. In: AAAI 2000 Workshop on Learning Statistical Models from Relational Data (2000)Google Scholar
  13. 13.
    Rakotomamonjy, A., Bach, F., Canu, S., Grandvalet, Y.: More Efficiency in Multiple Kernel Learning. In: ICML 2007 (2007)Google Scholar
  14. 14.
    Galun, M., Sharon, E., Basri, R., Brandt, A.: Texture segmentation by multiscale aggregation of filter responses and shape elements. In: ICCV (2003)Google Scholar
  15. 15.
    Rabinovich, A., Lange, T., Buhmann, J., Belongie, S.: Model Order Selection and Cue Combination for Image Segmentation. In: CVPR 2006 (2006)Google Scholar
  16. 16.
    Shotton, J., Johnson, M., Cipolla, R.: Semantic Texton Forests for Image Categorization and Segmentation. In: CVPR 2008 (2008)Google Scholar
  17. 17.
    Hoiem, D., Efros, A.A., Hebert, M.: Geometric Context from a Single Image. In: ICCV 2005 (2005)Google Scholar
  18. 18.
    Oliva, A., Torralba, A.: Building the Gist of a Scene: The Role of Global Image Features in Recognition. In: Visual Perception 2006 (2006)Google Scholar
  19. 19.
    Russell, B.C., Torralba, A., Murphy, K.P., Freeman, W.T.: LabelMe: a database and web-based tool for image annotation. In: IJCV 2008 (2008)Google Scholar
  20. 20.
    Jain, P., Kapoor, A.: Probabilistic Nearest Neighbor Classifier with Active Learning,
  21. 21.
    Malisiewicz, T., Efros, A.: Beyond Categories: The Visual Memex Model for Reasoning About Object Relationships. In: NIPS (2009)Google Scholar
  22. 22.
    Gupta, A., Davis, L.S.: Objects in Action:An Approach for Combining Action Understanding and Object Perception. In: CVPR 2007 (2007)Google Scholar
  23. 23.
    Szummer, M., Kohli, P., Hoiem, D.: Learning CRFs using Graph Cuts. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 582–595. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  24. 24.
    Tsochantaridis, I., Joachims, T., Hofmann, T., Altun, Y., Singer, Y.: Large margin methods for structured and interdependent output variables. JMLR 6, 1453–1484 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Arpit Jain
    • 1
  • Abhinav Gupta
    • 2
  • Larry S. Davis
    • 1
  1. 1.University of Maryland College Park
  2. 2.Carnegie Mellon UniversityPittsburgh

Personalised recommendations