Segmentation and Classification of Objects with Implicit Scene Context

  • Jan D. Wegner
  • Bodo Rosenhahn
  • Uwe Sörgel
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7474)


We present a novel approach to segment and classify objects in images into two classes. A binary conditional random field (CRF) framework is augmented with an unsupervised clustering step learning contextual relations of objects, the so-called implicit scene context (ISC). Several experiments with simulated data, images from benchmark data sets, and aerial images of an urban area show improved results compared to a standard CRF.


segmentation classification conditional random field context clustering 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Liu, D.C., Nocedal, J.: On the Limited Memory BFGS method for large scale optimization. Mathematical Programming 45, 503–528 (1989)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Nocedal, J.: Updating Quasi-Newton Matrices with Limited Storage. Mathematics of Computation 35, 773–782 (1980)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Frey, B.J., MacKay, D.J.C.: A Revolution: Belief Trees: Belief Propagation in Graphs With Cycles. In: Jordan, M.I., Kearns, M.J., Solla, S.A. (eds.) Advances in Neural Information Processing Systems, pp. 479–485. MIT Press, Cambridge (1998)Google Scholar
  4. 4.
    Lafferty, J., McCallum, A., Pereira, F.: Conditional Random Fields: Probabilistic Models for segmenting and labeling sequence data. In: ICML, p. 8 (2001)Google Scholar
  5. 5.
    Belongie, S., Malik, J., Puzicha, J.: Shape Matching and Object Recognition Using Shape Contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(24), 509–522 (2002)CrossRefGoogle Scholar
  6. 6.
    Kumar, S., Hebert, M.: Discriminative random fields: A discriminative framework for contextual interaction in classification. In: ICCV, vol. 2, pp. 1150–1157 (2003)Google Scholar
  7. 7.
    Kumar, S., Hebert, M.: A Hierarchical Field Framework for Unified Context-Based Classification. In: ICCV, vol. 2, pp. 1284–1291 (2005)Google Scholar
  8. 8.
    Lim, J.J., Arbelaez, P., Gu, C., Malik, J.: Context by Region Ancestry. In: ICCV, pp. 1978–1985 (2009)Google Scholar
  9. 9.
    He, X., Zemel, R.S., Carreira-Perpiñán, M.: Multiscale Conditional Random Fields for Image Labeling. In: CVPR, p. 8 (2004)Google Scholar
  10. 10.
    Crowther, P.S., Cox, R.J.: A Method for Optimal Division of Data Sets for Use in Neural Networks. In: Khosla, R., Howlett, R.J., Jain, L.C. (eds.) KES 2005. LNCS (LNAI), vol. 3684, pp. 1–7. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  11. 11.
    Dalal, N., Triggs, B.: Histograms of Oriented Gradients for Human Detection. In: CVPR, p. 8 (2005)Google Scholar
  12. 12.
    Torralba, A., Murphy, K.P., Freeman, W.T.: Contextual Models for Object Detection Using BoostedGoogle Scholar
  13. 13.
    Savarese, S., Winn, J., Criminisi, A.: Discriminative Object Class Models of Appearance and Shape by Correlatons. In: CVPR, p. 8 (2006)Google Scholar
  14. 14.
    Shotton, J., Winn, J.M., Rother, C., Criminisi, A.: TextonBoost: Joint Appearance, Shape and Context Modeling for Multi-class Object Recognition and Segmentation. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 1–15. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  15. 15.
    Kumar, S., Hebert, M.: Discriminative Random Fields. International Journal of Computer Vision 68(2), 179–201 (2006)CrossRefGoogle Scholar
  16. 16.
    Sha, F., Pereira, F.: Shallow parsing with conditional random fields. In: HLT-NAACL, pp. 213–220 (2003)Google Scholar
  17. 17.
    Rabinovich, A., Vedaldi, A., Galleguillos, C., Wiewiora, E., Belongie, S.: Objects in Context. In: ICCV, p. 8 (2007)Google Scholar
  18. 18.
    Oliva, A., Torralba, A.: The role of context in object recognition. Trends in Cognitive Sciences 11(12), 520–527 (2007)CrossRefGoogle Scholar
  19. 19.
    Ball, G.H., Hall, D.J.: A clustering technique for summarizing multivariate data. Systems Research and Behavioral Science 12(2), 153–155 (1967)CrossRefGoogle Scholar
  20. 20.
    Vedaldi, A., Fulkerson, B.: VLFeat: An Open and Portable Library of Computer Vision Algorithms (2008),
  21. 21.
    Vedaldi, A., Soatto, S.: Quick Shift and Kernel Methods for Mode Seeking. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 705–718. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  22. 22.
    Kohli, P., Ladicky, L., Torr, P.H.S.: Robust Higher Order Potentials for Enforcing Label Consistency. In: CVPR, p. 8 (2008)Google Scholar
  23. 23.
    Galleguillos, C., McFee, B., Belongie, S., Lanckriet, G.: Multi-Class Object Localization by Combining Local Contextual Interactions. In: CVPR, p. 8 (2010)Google Scholar
  24. 24.
    Gould, S., Rodgers, J., Cohen, D., Elidan, G., Koller, D.: Multi-Class Segmentation with Relative Location Prior. International Journal of Computer Vision 80(3), 300–316 (2008)CrossRefGoogle Scholar
  25. 25.
    Carbonetto, P., de Freitas, N., Barnard, K.: A Statistical Model for General Contextual Object Recognition. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 350–362. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  26. 26.
    Ladicky, L., Russell, C., Kohli, P., Torr, P.H.S.: Associative Hierarchical CRFs for Object Class Image Segmentation. In: ICCV, p. 8 (2009)Google Scholar
  27. 27.
    Fulkerson, B., Vedaldi, A., Soatto, S.: Class Segmentation and Object Localization with Superpixel Neighborhoods. In: ICCV, p. 8 (2009)Google Scholar
  28. 28.
    Wegner, J.D., Rosenhahn, B., Soergel, U.: Implicit Scene Context for Object Segmentation and Classification. In: Mester, R., Felsberg, M. (eds.) DAGM 2011. LNCS, vol. 6835, pp. 31–40. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  29. 29.
    Korč, F., Förstner, W.: Interpreting terrestrial images of urban scenes using discriminative random fields. In: ISPRS Symposium Beijing, vol. 37(B3a), pp. 291–296 (2008)Google Scholar
  30. 30.
    Wegner, J.D., Rosenhahn, B., Soergel, U.: Segment-based building detection with Conditional Random Fields. In: Stilla, U., Juergens, C., Maktav, D. (eds.) 6th IEEE/GRSS/ISPRS Joint Urban Remote Sensing Event, pp. 205–208 (2011)Google Scholar
  31. 31.
    Munoz, D., Bagnell, J.A., Hebert, M.: Stacked Hierarchical Labeling. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6316, pp. 57–70. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  32. 32.
    Lempitsky, V., Vedaldi, A., Zisserman, A.: A Pylon Model for Semantic Segmentation. In: Shawe-Taylor, J., Zemel, R.S., Bartlett, P., Pereira, F.C.N., Weinberger, K.Q. (eds.) NIPS 2011, vol. 24, pp. 1485–1493 (2011)Google Scholar
  33. 33.
    Korč, F., Förstner, W.: eTRIMS Image Database for Interpreting Images of Man-Made Scenes (2009),
  34. 34.
    Sutton, C., McCallum, A.: An Introduction to Conditional Random Fields for Relational Learning. In: Getoor, L., Taskar, B. (eds.) Introduction to Statistical Relational Learning. MIT Press, Cambridge (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Jan D. Wegner
    • 1
  • Bodo Rosenhahn
    • 2
  • Uwe Sörgel
    • 1
  1. 1.Institute of Photogrammetry and GeoInformationGermany
  2. 2.Institut für InformationsverarbeitungLeibniz Universität HannoverGermany

Personalised recommendations