Finding Deformable Shapes Using Loopy Belief Propagation

  • James M. Coughlan
  • Sabino J. Ferreira
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2352)


A novel deformable template is presented which detects and localizes shapes in grayscale images. The template is formulated as a Bayesian graphical model of a two-dimensional shape contour, and it is matched to the image using a variant of the belief propagation (BP) algorithm used for inference on graphical models. The algorithm can localize a target shape contour in a cluttered image and can accommodate arbitrary global translation and rotation of the target as well as significant shape deformations, without requiring the template to be initialized in any special way (e.g. near the target).

The use of BP removes a serious restriction imposed in related earlier work, in which the matching is performed by dynamic programming and thus requires the graphical model to be tree-shaped (i.e. without loops). Although BP is not guaranteed to converge when applied to inference on non-tree-shaped graphs, we find empirically that it does converge even for deformable template models with one or more loops. To speed up the BP algorithm, we augment it by a pruning procedure and a novel technique, inspired by the 20 Questions (divide-and-conquer) search strategy, called ”focused message updating.” These modifications boost the speed of convergence by over an order of magnitude, resulting in an algorithm that detects and localizes shapes in grayscale images in as little as several seconds on an 850 MHz AMD processor.


Grayscale Image Background Clutter Hand Shape Reference Shape Edge Strength 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    S. Belongie, J. Malik and J. Puzicha, “Shape Matching and Object Recognition Using Shape Contexts,” accepted for publication in PAMI.Google Scholar
  2. 2.
    H. Chui and A. Rangarajan, “A new algorithm for non-rigid point matching,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2000.Google Scholar
  3. 3.
    J. Coughlan, A.L. Yuille, C. English, D. Snow. ”Efficient Deformable Template Detection and Localization without User Initialization.” Computer Vision and Image Understanding, Vol. 78, No. 3, pp. 303–319. June 2000.CrossRefGoogle Scholar
  4. 4.
    J. Coughlan and S. Ferreira. “Finding Deformable Shapes using Loopy Belief Propagation.” Technical report in preparation.Google Scholar
  5. 5.
    F. Fleuret and D. Geman, “Coarse-to-fine face detection,” Inter. Journal of Computer Vision, 41, 85–107, 2001.zbMATHCrossRefGoogle Scholar
  6. 6.
    D. Geman. and B. Jedynak. “An active testing model for tracking roads in satellite images”. IEEE Trans. Patt. Anal. and Machine Intel. Vol. 18. No. 1, pp 1–14. January. 1996.CrossRefGoogle Scholar
  7. 7.
    U. Grenander, Y. Chow and D. M. Keenan, Hands: a Pattern Theoretic Study of Biological Shapes, Springer-Verlag, 1991.Google Scholar
  8. 8.
    S. Konishi, A. L. Yuille, J. M. Coughlan, and S. C. Zhu. “Fundamental Bounds on Edge Detection: An Information Theoretic Evaluation of Different Edge Cues.” Proc. Int’l conf. on Computer Vision and Pattern Recognition, 1999.Google Scholar
  9. 9.
    B. Lowerre and R. Reddy, “The Harpy Speech Understanding System.” In Trends in Speech Recognition. Ed. W. Lea. Prentice Hall. 1980.Google Scholar
  10. 10.
    K.P. Murphy, Y. Weiss and M.I. Jordan. “Loopy belief propagation for approximate inference: an empirical study”. In Proceedings of Uncertainty in AI. 1999.Google Scholar
  11. 11.
    J. Pearl. Probabilistic Reasoning in Intelligent Systems. Morgan Kaufman. 1988.Google Scholar
  12. 12.
    H. Schneiderman and T. Kanade. ”A Statistical Method for 3D Object Detection Applied to Faces and Cars”. IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2000).Google Scholar
  13. 13.
    P. Viola and M. Jones. “Rapid Object Detection using a Boosted Cascade of Simple Features,” CVPR 2001.Google Scholar
  14. 14.
    M. Wainwright, T. Jaakkola and A. Willsky. “Tree-based reparameterization for approximate estimation on loopy graphs.” To appear in NIPS 2001.Google Scholar
  15. 15.
    L. Wiscott and C. von der Marlsburg, “A Neural System for the Recognition of Partially Occluded Objects in Cluttered Scenes”. Neural Computation. 7(4):935–948. 1993.Google Scholar
  16. 16.
    J.S. Yedidia, W.T. Freeman, Y. Weiss. “Bethe Free Energies, Kikuchi Approximations, and Belief Propagation Algorithms”. 2001. MERL Cambridge Research Technical Report TR 2001-16.Google Scholar
  17. 17.
    A. L. Yuille, “Deformable Templates for Face Recognition”. Journal of Cognitive Neuroscience. Vol 3, Number 1. 1991.Google Scholar
  18. 18.
    A.L. Yuille and J. Coughlan. “Twenty Questions, Focus of Attention, and A*: A theoretical comparison of optimization strategies.” In Proceedings EMMCVPR’ 97. Venice. Italy. 1997.Google Scholar
  19. 19.
    A.L. Yuille and J. Coughlan. “Visual Search: Fundamental Bounds, Order Parameters, and Phase Transitions.” IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Vol. 22, No. 2, pp. 1–14. February 2000.CrossRefGoogle Scholar
  20. 20.
    A. L. Yuille, “CCCP Algorithms to Minimize the Bethe and Kikuchi Free Energies: Convergent Alternatives to Belief Propagation.” Neural Computation. In press.Google Scholar
  21. 21.
    S. C. Zhu, R. Zhang, and Z. W. Tu, ”Integrating Top-down/Bottom-up for Object Recognition by Data Driven Markov Chain Monte Carlo”, Proc. of Int’l Conf. on Computer Vision and Pattern Recognition, Hilton Head, SC, 2000.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • James M. Coughlan
    • 1
  • Sabino J. Ferreira
    • 2
  1. 1.Smith-Kettlewell InstituteSan FranciscoUSA
  2. 2.Department of StatisticsFederal University of Minas Gerais UFMGBelo Horizonte MGBrasil

Personalised recommendations