Finding Deformable Shapes Using Loopy Belief Propagation
A novel deformable template is presented which detects and localizes shapes in grayscale images. The template is formulated as a Bayesian graphical model of a two-dimensional shape contour, and it is matched to the image using a variant of the belief propagation (BP) algorithm used for inference on graphical models. The algorithm can localize a target shape contour in a cluttered image and can accommodate arbitrary global translation and rotation of the target as well as significant shape deformations, without requiring the template to be initialized in any special way (e.g. near the target).
The use of BP removes a serious restriction imposed in related earlier work, in which the matching is performed by dynamic programming and thus requires the graphical model to be tree-shaped (i.e. without loops). Although BP is not guaranteed to converge when applied to inference on non-tree-shaped graphs, we find empirically that it does converge even for deformable template models with one or more loops. To speed up the BP algorithm, we augment it by a pruning procedure and a novel technique, inspired by the 20 Questions (divide-and-conquer) search strategy, called ”focused message updating.” These modifications boost the speed of convergence by over an order of magnitude, resulting in an algorithm that detects and localizes shapes in grayscale images in as little as several seconds on an 850 MHz AMD processor.
KeywordsGrayscale Image Background Clutter Hand Shape Reference Shape Edge Strength
Unable to display preview. Download preview PDF.
- 1.S. Belongie, J. Malik and J. Puzicha, “Shape Matching and Object Recognition Using Shape Contexts,” accepted for publication in PAMI.Google Scholar
- 2.H. Chui and A. Rangarajan, “A new algorithm for non-rigid point matching,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2000.Google Scholar
- 4.J. Coughlan and S. Ferreira. “Finding Deformable Shapes using Loopy Belief Propagation.” Technical report in preparation.Google Scholar
- 7.U. Grenander, Y. Chow and D. M. Keenan, Hands: a Pattern Theoretic Study of Biological Shapes, Springer-Verlag, 1991.Google Scholar
- 8.S. Konishi, A. L. Yuille, J. M. Coughlan, and S. C. Zhu. “Fundamental Bounds on Edge Detection: An Information Theoretic Evaluation of Different Edge Cues.” Proc. Int’l conf. on Computer Vision and Pattern Recognition, 1999.Google Scholar
- 9.B. Lowerre and R. Reddy, “The Harpy Speech Understanding System.” In Trends in Speech Recognition. Ed. W. Lea. Prentice Hall. 1980.Google Scholar
- 10.K.P. Murphy, Y. Weiss and M.I. Jordan. “Loopy belief propagation for approximate inference: an empirical study”. In Proceedings of Uncertainty in AI. 1999.Google Scholar
- 11.J. Pearl. Probabilistic Reasoning in Intelligent Systems. Morgan Kaufman. 1988.Google Scholar
- 12.H. Schneiderman and T. Kanade. ”A Statistical Method for 3D Object Detection Applied to Faces and Cars”. IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2000).Google Scholar
- 13.P. Viola and M. Jones. “Rapid Object Detection using a Boosted Cascade of Simple Features,” CVPR 2001.Google Scholar
- 14.M. Wainwright, T. Jaakkola and A. Willsky. “Tree-based reparameterization for approximate estimation on loopy graphs.” To appear in NIPS 2001.Google Scholar
- 15.L. Wiscott and C. von der Marlsburg, “A Neural System for the Recognition of Partially Occluded Objects in Cluttered Scenes”. Neural Computation. 7(4):935–948. 1993.Google Scholar
- 16.J.S. Yedidia, W.T. Freeman, Y. Weiss. “Bethe Free Energies, Kikuchi Approximations, and Belief Propagation Algorithms”. 2001. MERL Cambridge Research Technical Report TR 2001-16.Google Scholar
- 17.A. L. Yuille, “Deformable Templates for Face Recognition”. Journal of Cognitive Neuroscience. Vol 3, Number 1. 1991.Google Scholar
- 18.A.L. Yuille and J. Coughlan. “Twenty Questions, Focus of Attention, and A*: A theoretical comparison of optimization strategies.” In Proceedings EMMCVPR’ 97. Venice. Italy. 1997.Google Scholar
- 20.A. L. Yuille, “CCCP Algorithms to Minimize the Bethe and Kikuchi Free Energies: Convergent Alternatives to Belief Propagation.” Neural Computation. In press.Google Scholar
- 21.S. C. Zhu, R. Zhang, and Z. W. Tu, ”Integrating Top-down/Bottom-up for Object Recognition by Data Driven Markov Chain Monte Carlo”, Proc. of Int’l Conf. on Computer Vision and Pattern Recognition, Hilton Head, SC, 2000.Google Scholar