Pairwise Clustering with Matrix Factorisation and the EM Algorithm
In this paper we provide a direct link between the EM algorithm and matrix factorisation methods for grouping via pairwise clustering. We commence by placing the pairwise clustering process in the setting of the EM algorithm. We represent the clustering process using two sets of variables which need to be estimated. The first of these are cluster-membership indicators. The second are revised link-weights between pairs of nodes. We work with a model of the grouping process in which both sets of variables are drawn from a Bernoulli distribution. The main contributioin in this paper is to show how the cluster-memberships may be estimated using the leading eigenvector of the revised link-weight matrices. We also establish convergence conditions for the resulting pair-wise clustering process. The method is demonstrated on the problem of multiple moving object segmentation.
KeywordsMotion Vector Matrix Factorisation Cluster Membership Link Weight Motion Segmentation
Unable to display preview. Download preview PDF.
- 2.Marina Meilă and Jianbo Shi. Learning segmentation by random walks. In Advances in Neural Information Processing Systems 13, pages 873–879. MIT Press, 2001.Google Scholar
- 4.J. S. Bridle. Training stochastic model recognition algorithms can lead to maximum mutual information estimation of parameters. In NIPS 2, pages 211–217, 1990.Google Scholar
- 5.Fan R. K. Chung. Spectral Graph Theory. American Mathematical Society, 1997.Google Scholar
- 8.M. Doob D. Cvetković and H. Sachs. Spectra of Graphs:Theory and Application. Academic Press, 1980.Google Scholar
- 10.J. M. Leitao M. Figueiredo and A. K. Jain. On fitting mixture models. In Proceedings of the Second Int. Workshop on Energy Minimization Methods in Comp. Vis. and Pattern Recognition, number 1654 in Lecture Notes in Comp. Science, pages 54–67, 1999.Google Scholar
- 12.J. Risanen. Stochastic Complexity in Statistical Enquiry. World Scientific, 1989.Google Scholar
- 13.A. Robles-Kelly and E. R. Hancock. A maximum likelihood framework for iterative eigendecomposition. In Proc. of the IEEE International Conference on Conputer Vision, 2001.Google Scholar
- 15.J. Shi and J. Malik. Normalized cuts and image segmentations. In Proc. IEEE CVPR, pages 731–737, 1997.Google Scholar
- 16.Naftali Tishby and Noam Slonim. Data clustering by markovian relaxation and the information bottleneck method. In Todd K. Leen, Thomas G. Dietterich, and Volker Tresp, editors, Advances in Neural Information Processing Systems 13, pages 640–646. MIT Press, 2001.Google Scholar
- 17.R. S. Varga. Matrix Iterative Analysis. Springer, second edition, 2000.Google Scholar
- 19.D. Weinshall Y. Gdalyahu and M. Werman. A randomized algorithm for pairwise clustering. In Advances in Neural Information Processing Systems 11, pages 424–430. MIT Press, 1999.Google Scholar