Shape Matching and Recognition – Using Generative Models and Informative Features

  • Zhuowen Tu
  • Alan L. Yuille
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3023)


We present an algorithm for shape matching and recognition based on a generative model for how one shape can be generated by the other. This generative model allows for a class of transformations, such as affine and non-rigid transformations, and induces a similarity measure between shapes. The matching process is formulated in the EM algorithm. To have a fast algorithm and avoid local minima, we show how the EM algorithm can be approximated by using informative features, which have two key properties–invariant and representative. They are also similar to the proposal probabilities used in DDMCMC [13]. The formulation allows us to know when and why approximations can be made and justifies the use of bottom-up features, which are used in a wide range of vision problems. This integrates generative models and feature-based approaches within the EM framework and helps clarifying the relationships between different algorithms for this problem such as shape contexts [3] and softassign [5]. We test the algorithm on a variety of data sets including MPEG7 CE-Shape-1, Kimia silhouettes, and real images of street scenes. We demonstrate very effective performance and compare our results with existing algorithms. Finally, we briefly illustrate how our approach can be generalized to a wider range of problems including object detection.


Image Retrieval Real Image Target Shape Shape Match Informative Feature 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Abbasi, S., Mokh, F.: Robustness of Shape Similarity Retrieval under Affine. In: Proc. of Challenge of Image Retrieval (1999)Google Scholar
  2. 2.
    Ballard, D.H.: Generalizing the Hough Transform to Detect Arbitrary Shapes. Pattern Recognition 13(2) (1981)Google Scholar
  3. 3.
    Belongie, S., Malik, J., Puzicha, J.: Shape Matching and Object Recognition Using Shape Contexts. IEEE Trans. on PAMI 24(24) (2002)Google Scholar
  4. 4.
    Bookstein, F.L.: Principal Warps: Thin-Plate Splines and the Decomposition of Deformations. IEEE Trans. on PAMI 11(6) (1989)Google Scholar
  5. 5.
    Chui, H., Rangarajan, A.: A New Point Matching Algorithm for Non-rigid Registration. Computer Vision and Image Understanding (March 2003)Google Scholar
  6. 6.
    Grenander, U.: General Pattern Theory: A Mathematical Study of Regular Structures, Oxford (1994)Google Scholar
  7. 7.
    Latechi, L.J., Lakamper, R., Eckhardt, U.: Shape Descriptors for Non-rigid Shapes with a Single Closed Contour. In: Proc. of CVPR (2000)Google Scholar
  8. 8.
    Neal, R., Hinton, G.E.: A View Of The Em Algorithm That Justifies Incremental, Sparse, And Other Variants. Learning in Graphical Models (1998)Google Scholar
  9. 9.
    Rangarajan, A., Coughlan, J.M., Yuille, A.L.: A Bayesian Network for Relational Shape Matching. In: Proc. of ICCV, Nice, France (2003)Google Scholar
  10. 10.
    Sebastian, T.B., Klein, P.N., Kimia, B.B.: On Aligning Curves. IEEE Trans. on PAMI 25(1) (2003)Google Scholar
  11. 11.
    Sebastian, T.B., Klein, P.N., Kimia, B.B.: Recognition of Shapes by Editing their Shock Graphs, accepted by IEEE Trans. on PAMI (2003)Google Scholar
  12. 12.
    Thayananthan, A., Stenger, B., Torr, P.H.S., Cipolla, R.: Shape Context and Chamfer Matching in Cluttered Scenes. In: CVPR (2003)Google Scholar
  13. 13.
    Tu, Z., Chen, X., Yuille, A., Zhu, S.C.: Image Parsing: Unifying Segmentation, Detection and Recognition. In: Proc. of ICCV, Nice, France (2003)Google Scholar
  14. 14.
    Veltkamp, R.C., Hagedoorn, M.: State of the Art in Shape Matching, Technical Report UU-CS-1999-27, Utrecht (1999)Google Scholar
  15. 15.
    Yuille, A.L., Grzywacz, N.M.: A Computational Theory for the Perception of Coherent Visual Motion. Nature 333(6168) (1988)Google Scholar
  16. 16.
    Zhu, S.C., Yuille, A.L.: FORMS: A Flexible Object Recognition and Modeling System. IJCV 20(3) (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Zhuowen Tu
    • 1
  • Alan L. Yuille
    • 1
  1. 1.Departments of StatisticsUniversity of CaliforniaLos AngelesUSA

Personalised recommendations