Learning-Based Object Segmentation Using Regional Spatial Templates and Visual Features

  • Iker Gondra
  • Fahim Irfan Alam
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7594)


Semantically accurate segmentation of an object of interest (OOI) is a critical step in computer vision tasks. In order to bridge the gap between low-level visual features and high-level semantics, a more complete model of the OOI is needed. To this end, we revise the concept of directional spatial templates and introduce regional directional spatial templates as a means of including spatial relationships among OOI regions into the model. We present an object segmentation algorithm that learns a model which includes both visual and spatial information. Given a training set of images containing the OOI, each image is oversegmented into visually homogeneous regions. Next, Multiple Instance Learning identifies regions that are likely to be part of the OOI. For each pair of such regions and for each relationship, a regional template is formed. The computational cost of template generation is reduced by sampling the reference region with a pixel set that is descriptive of its shape. Experiments indicate that regional templates are an effective way of including spatial information into the model which in turn results in a very significant improvement in segmentation performance.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Carson, C., Belongie, S., Greenspan, H., Malik, J.: Blobworld: color and texture-based image segmentation using EM and its applications to image querying and classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(8), 1026–1038 (2002)CrossRefGoogle Scholar
  2. 2.
    Vicente, S., Kolmogorov, V., Rother, C.: Graph cut based image segmentation with connectivity priors. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)Google Scholar
  3. 3.
    Peng, J., Bhanu, B.: Closed-loop object recognition using reinforcement learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(2), 139–154 (1998)CrossRefGoogle Scholar
  4. 4.
    Crandall, D.J., Huttenlocher, D.P.: Weakly Supervised Learning of Part-Based Spatial Models for Visual Object Recognition. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 16–29. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  5. 5.
    Viola, P., Platt, J.C., Zhang, C.: Multiple instance boosting for object detection. Advances in Neural Information Processing Systems 18, 1417–1424 (2006)Google Scholar
  6. 6.
    Gondra, I., Xu, T.: A multiple instance learning based framework for semantic image segmentation. Multimedia Tools and Applications 48(2), 339–365 (2010)CrossRefGoogle Scholar
  7. 7.
    Dietterich, T., Lathrop, R., Lozano-Perez, T.: Solving the multiple-instance problem with axis-parallel rectangles. Artificial Intelligence 89(1-2), 31–71 (1997)CrossRefzbMATHGoogle Scholar
  8. 8.
    Todorovic, S., Ahuja, N.: Extracting subimages of an unknown category from a set of images. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 927–934 (2006)Google Scholar
  9. 9.
    Alam, F.I., Gondra, I.: A bayesian network-based tunable image segmentation algorithm for object recognition. In: Proceedings of IEEE International Symposium on Signal Processing and Information Technology, pp. 11–16 (2011)Google Scholar
  10. 10.
    Matsakis, P., Ni, J., Wang, X.: Object localization based on directional information: Case of 2d raster data. In: Proceedings of IAPR International Conference on Pattern Recognition, pp. 142–146 (2006)Google Scholar
  11. 11.
    Maron, O., Lozano-Perez, T.: A framework for multiple instance learning. In: Proceedings of the Conference on Advances in Neural Information Processing Systems, vol. 10, pp. 570–576 (1998)Google Scholar
  12. 12.
    Xu, T., Gondra, I., Chiu, D.: Adaptive kernel diverse density estimate for multiple instance learning. In: Proceedings of 2011 International Conference on Machine Learning and Data Mining, pp. 185–198 (2011)Google Scholar
  13. 13.
    Bloch, I.: Fuzzy relative position between objects in image processing: A morphological approach. IEEE Transactions on Pattern Analysis and Machine Intelligence 106, 657–664 (1999)CrossRefGoogle Scholar
  14. 14.
    Logan, G.D., Sadler, D.D.: A computational analysis of the apprehension of spatial relations. In: Language and Space. MIT Press, Cambridge (1996)Google Scholar
  15. 15.
    Gapp, K.P.: Basic meaning of spatial relations: Computation and evaluation in 3d space. In: Proceedings of National Conference on Artificial Intelligence, pp. 1393–1398 (1994)Google Scholar
  16. 16.
    Olivier, P., Tsuji, J.I.: Quantitative perceptual representation of prepositional semantics. Artificial Intelligence Review 8, 147–158 (1994)CrossRefGoogle Scholar
  17. 17.
    Goshtasby, A.: Description and discrimination of planar shapes using shape matrices. IEEE Transactions on Pattern Analysis and Machine Intelligence 7(6), 738–743 (1985)CrossRefGoogle Scholar
  18. 18.
    Arthur, D., Vassilvitskii, S.: K-means++: The advantages of careful seeding. Technical Report 2006-13, Stanford InfoLab (June 2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Iker Gondra
    • 1
  • Fahim Irfan Alam
    • 1
  1. 1.Department of Mathematics, Statistics and Computer ScienceSt. Francis Xavier UniversityNova ScotiaCanada

Personalised recommendations