Image and Pattern Clustering
- 2k Downloads
Clustering, or grouping samples which share similar features, is a recurrent problem in computer vision and pattern recognition. The core element of a clustering algorithm is the similarity measure. In this regard information theory offers a wide range of measures (not always metrics) which inspire clustering algorithms through their optimization. In addition, information theory also provides both theoretical frameworks and principles to formulate the clustering problem and provide effective algorithms. Clustering is closely related to the segmentation problem, already presented in Chapter 3. In both problems, finding the optimal number of clusters or regions is a challenging task. In the present chapter we cover this question in depth. To that end we explore several criteria for model order selection.
All the latter concepts are developed through the description and discussion of several information theoretic clustering algorithms: Gaussian mixtures, Information Bottleneck, Robust Information Clustering (RIC) and IT-based Mean Shift. At the end of the chapter we also discuss basic strategies to form clustering ensembles.
KeywordsMutual Information Gaussian Mixture Model Shannon Entropy Expectation Maximization Algorithm Pattern Cluster
Unable to display preview. Download preview PDF.
- A. Peñalver, F. Escolano, and J.M. Sáez. “EBEM: An Entropy-Based EM Algorithm for Gaussian Mixture Models”. International Conference on Pattern Recognition, Hong Kong (China) (2006)Google Scholar
- A. Peñalver, F. Escolano, and J.M. Sáez. “Two Entropy-Based Methods for Learning Unsupervised Gaussian Mixture Models”. SSPR/SPR — LNCS (2006)Google Scholar
- N. Srebro, G. Shakhnarovich, and S. Roweis. “An Investigation of Computational and Informational Limits in Gaussian Mixture Clustering”. International Conference on Machine Learning (2006)Google Scholar
- N.Z. Tishby, F. Pereira, and W. Bialek. “The Information Bottleneck method”. 37th Allerton Conference on Communication, Control and Computing (1999)Google Scholar
- N. Slonim and N. Tishby. “Agglomerative Information Bottleneck”. In Proceeding of Neural Information Processing Systems (1999)Google Scholar