Saliency Based Opportunistic Search for Object Part Extraction and Labeling

  • Yang Wu
  • Qihui Zhu
  • Jianbo Shi
  • Nanning Zheng
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5305)


We study the task of object part extraction and labeling, which seeks to understand objects beyond simply identifiying their bounding boxes. We start from bottom-up segmentation of images and search for correspondences between object parts in a few shape models and segments in images. Segments comprising different object parts in the image are usually not equally salient due to uneven contrast, illumination conditions, clutter, occlusion and pose changes. Moreover, object parts may have different scales and some parts are only distinctive and recognizable in a large scale. Therefore, we utilize a multi-scale shape representation of objects and their parts, figural contextual information of the whole object and semantic contextual information for parts. Instead of searching over a large segmentation space, we present a saliency based opportunistic search framework to explore bottom-up segmentation by gradually expanding and bounding the search domain. We tested our approach on a challenging statue face dataset and 3 human face datasets. Results show that our approach significantly outperforms Active Shape Models using far fewer exemplars. Our framework can be applied to other object categories.


Object Detection Shape Model Semantic Context Shape Match Object Part 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Felzenszwalb, P., McAllester, D., Ramanan, D.: A discriminatively trained, multiscale, deformable part model. In: CVPR (2008)Google Scholar
  2. 2.
    Ferrari, V., Jurie, F., Schmid, C.: Accurate object detection with deformable shape models learnt from images. In: CVPR, pp. 1–8 (2007)Google Scholar
  3. 3.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Pictorial structures for object recognition. IJCV 61(1), 55–79 (2005)CrossRefGoogle Scholar
  4. 4.
    Zhu, Q., Wang, L., Wu, Y., Shi, J.: Contour context selection for object detection: A single exemplar suffices. In: ECCV (2008)Google Scholar
  5. 5.
    Zhu, Q., Shi, J.: Untangling cycles for contour grouping. In: ICCV (2007)Google Scholar
  6. 6.
    Biederman, I.: Recognition by components: A theory of human image understanding. PsychR 94(2), 115–147 (1987)Google Scholar
  7. 7.
    Pentland, A.: Recognition by parts. In: ICCV, pp. 612–620 (1987)Google Scholar
  8. 8.
    Amit, Y., Trouve, A.: Pop: Patchwork of parts models for object recognition. IJCV 75(2), 267–282 (2007)CrossRefGoogle Scholar
  9. 9.
    Sudderth, E., Torralba, A., Freeman, W., Willsky, A.: Learning hierarchical models of scenes, objects, and parts. In: ICCV, 1331–1338 (2005)Google Scholar
  10. 10.
    Cootes, T., Taylor, C., Cooper, D., Graham, J.: Active shape models: Their training and application. CVIU 61(1), 38–59 (1995)Google Scholar
  11. 11.
    Cootes, T., Edwards, G., Taylor, C.: Active appearance models. PAMI 23(6), 681–685 (2001)Google Scholar
  12. 12.
    Belongie, S., Malik, J., Puzicha, J.: Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. (2002)Google Scholar
  13. 13.
    Wang, L., Shi, J., Song, G., fan Shen, I.: Object detection combining recognition and segmentation. In: ACCV (1), pp. 189–199 (2007)Google Scholar
  14. 14.
    Chen, C.: The First Emperor of China. Voyager Company (1994)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Yang Wu
    • 1
    • 2
  • Qihui Zhu
    • 2
  • Jianbo Shi
    • 2
  • Nanning Zheng
    • 1
  1. 1.Institute of Artificial Intelligence and RoboticsXi’an Jiaotong University 
  2. 2.Department of Computer and Information ScienceUniversity of Pennsylvania 

Personalised recommendations