Multiple Region Categorization for Scenery Images

  • Tamar Avraham
  • Ilya Gurvich
  • Michael Lindenbaum
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6978)


We present two novel contributions to the problem of region classification in scenery/landscape images. The first is a model that incorporates local cues with global layout cues, following the statistical characteristics recently suggested in [1]. The observation that background regions in scenery images tend to horizontally span the image allows us to represent the contextual dependencies between background region labels with a simple graphical model, on which exact inference is possible. While background is traditionally classified using only local color and textural features, we show that using new layout cues significantly improves background region classification. Our second contribution addresses the problem of correct results being considered as errors in cases where the ground truth provides the structural class of a land region (e.g., mountain), while the classifier provides its coverage class (e.g., grass), or vice versa. We suggest an alternative labeling method that, while trained using ground truth that describes each region with one label, assigns both a structural and a coverage label for each land region in the validation set. By suggesting multiple labels, each describing a different aspect of the region, the method provides more information than that available in the ground truth.


region annotation multiple categorization exact inference scenery/landcape boundary shape contextual scene understanding 


  1. 1.
    Avraham, T., Lindenbaum, M.: Non-local characterization of scenery images: Statistics, 3D reasoning, and a generative model. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6315, pp. 99–112. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  2. 2.
    Torralba, A.B.: Contextual priming for object detection. IJCV 53(2), 169–191 (2003)CrossRefGoogle Scholar
  3. 3.
    Kumar, S., Hebert, M.: A hierarchical field framework for unified context-based classification. In: ICCV (2005)Google Scholar
  4. 4.
    He, X., Zemel, R.S., Ray, D.: Learning and incorporating top-down cues in image segmentation. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 338–351. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  5. 5.
    Rabinovich, A., Vedaldi, A., Galleguillos, C., Wiewiora, E., Belongie, S.: Objects in context. In: ICCV (2007)Google Scholar
  6. 6.
    Desai, C., Ramanan, D., Fowlkes, C.: Discriminative models for multi-class object layout. In: ICCV (2009)Google Scholar
  7. 7.
    Galleguillos, C., Belongie, S.: Context based object categorization: A critical survey. Comput. Vis. Image Understand (2010)Google Scholar
  8. 8.
    Deng, J., Berg, A.C., Li, K., Fei-Fei, L.: What does classifying more than 10,000 image categories tell us? In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6315, pp. 71–84. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  9. 9.
    Fergus, R., Bernal, H., Weiss, Y., Torralba, A.: Semantic label sharing for learning with many categories. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6311, pp. 762–775. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  10. 10.
    Feebaum, C.: Wordnet: An Electronic Lexical Database. Bradford Books (1998)Google Scholar
  11. 11.
    Niu, D., Dy, J.G., Jordan, M.I.: Multiple non-redundant spectral clustering views. In: ICML (2010)Google Scholar
  12. 12.
    Vogel, J., Schiele, B.: Semantic modeling of natural scenes for content-based image retrieval. IJCV 72(2), 133–157 (2007)CrossRefGoogle Scholar
  13. 13.
    Haralick, R.M., Shanmugam, K., Dinstein, I.: Textural features for image classification. IEEE Transactions on Systems, Man, and Cybernetics, 610–621 (1973)Google Scholar
  14. 14.
    Wu, T.F., Lin, C.J., Weng, R.C.: Probability estimates for multi-class classification by pairwise coupling. Journal of Machine Learning Research 5, 975–1005 (2004)MathSciNetzbMATHGoogle Scholar
  15. 15.
    Kschischang, F.R., Frey, B.J., Loeliger, H.A.: Factor graphs and the sum-product algorithm. IEEE Tran. on Information Theory 47(2), 498–519 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Russell, B.C., Torralba, A.: Labelme: a database and web-based tool for image annotation. IJCV 77, 157–173 (2008)CrossRefGoogle Scholar
  17. 17.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. IJCV 42(3), 145–175 (2001)CrossRefzbMATHGoogle Scholar
  18. 18.
    Chang, C., Lin, C.: LIBSVM: a library for support vector machines. (2001) Software available at

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Tamar Avraham
    • 1
  • Ilya Gurvich
    • 1
  • Michael Lindenbaum
    • 1
  1. 1.Computer Science DepartmentTechnion - I.I.T.HaifaIsrael

Personalised recommendations