Pairwise Clustering with Matrix Factorisation and the EM Algorithm

  • Antonio Robles-Kelly
  • Edwin R. Hancock
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2351)


In this paper we provide a direct link between the EM algorithm and matrix factorisation methods for grouping via pairwise clustering. We commence by placing the pairwise clustering process in the setting of the EM algorithm. We represent the clustering process using two sets of variables which need to be estimated. The first of these are cluster-membership indicators. The second are revised link-weights between pairs of nodes. We work with a model of the grouping process in which both sets of variables are drawn from a Bernoulli distribution. The main contributioin in this paper is to show how the cluster-memberships may be estimated using the leading eigenvector of the revised link-weight matrices. We also establish convergence conditions for the resulting pair-wise clustering process. The method is demonstrated on the problem of multiple moving object segmentation.


Motion Vector Matrix Factorisation Cluster Membership Link Weight Motion Segmentation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    A. Dempster, N. Laird and D. Rubin. Maximum-likehood from incomplete data via the EM algorithm. J. Royal Statistical Soc. Ser. B (methodological), 39:1–38, 1977.zbMATHMathSciNetGoogle Scholar
  2. 2.
    Marina Meilă and Jianbo Shi. Learning segmentation by random walks. In Advances in Neural Information Processing Systems 13, pages 873–879. MIT Press, 2001.Google Scholar
  3. 3.
    A. G. Bors and I. Pitas. Optical flow estimation and moving object segmentation based on median radial basis function network. IEEE Trans. on Image Processing, 7(5):693–702, 1998.CrossRefGoogle Scholar
  4. 4.
    J. S. Bridle. Training stochastic model recognition algorithms can lead to maximum mutual information estimation of parameters. In NIPS 2, pages 211–217, 1990.Google Scholar
  5. 5.
    Fan R. K. Chung. Spectral Graph Theory. American Mathematical Society, 1997.Google Scholar
  6. 6.
    J. S. Shyn C. H. Hsieh, P. C. Lu and E. H. Lu. Motion estimation algorithm using inter-block correlation. IEE Electron. Lett., 26(5):276–277, 1990.CrossRefGoogle Scholar
  7. 7.
    L. Dieci. Considerations on computing real logarithms of matrices,hamiltonian logarithms and skew-symmetric logarithms. Linear Algebra and its Applications, 244:35–54, 1996.zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    M. Doob D. Cvetković and H. Sachs. Spectra of Graphs:Theory and Application. Academic Press, 1980.Google Scholar
  9. 9.
    T. Hofmann and M. Buhmann. Pairwise data clustering by deterministic annealing. IEEE Tansactions on Pattern Analysis and Machine Intelligence, 19(1):1–14, 1997.CrossRefGoogle Scholar
  10. 10.
    J. M. Leitao M. Figueiredo and A. K. Jain. On fitting mixture models. In Proceedings of the Second Int. Workshop on Energy Minimization Methods in Comp. Vis. and Pattern Recognition, number 1654 in Lecture Notes in Comp. Science, pages 54–67, 1999.Google Scholar
  11. 11.
    Z. Ghahramani N. Ueda, R. Nakano and G. E. Hinton. Smem algorithm for mixture models. Neural Computation, 12(9):2109–2128, 2000.CrossRefGoogle Scholar
  12. 12.
    J. Risanen. Stochastic Complexity in Statistical Enquiry. World Scientific, 1989.Google Scholar
  13. 13.
    A. Robles-Kelly and E. R. Hancock. A maximum likelihood framework for iterative eigendecomposition. In Proc. of the IEEE International Conference on Conputer Vision, 2001.Google Scholar
  14. 14.
    S. Sarkar and K. L. Boyer. Quantitative measures of change based on feature organization: Eigenvalues and eigenvectors. Computer Vision and Image Understanding, 71(1):110–136, 1998.CrossRefGoogle Scholar
  15. 15.
    J. Shi and J. Malik. Normalized cuts and image segmentations. In Proc. IEEE CVPR, pages 731–737, 1997.Google Scholar
  16. 16.
    Naftali Tishby and Noam Slonim. Data clustering by markovian relaxation and the information bottleneck method. In Todd K. Leen, Thomas G. Dietterich, and Volker Tresp, editors, Advances in Neural Information Processing Systems 13, pages 640–646. MIT Press, 2001.Google Scholar
  17. 17.
    R. S. Varga. Matrix Iterative Analysis. Springer, second edition, 2000.Google Scholar
  18. 18.
    N. Vlassis and A. Likas. A kurtosis-based dynamic approach to gaussian mixture modeling. IEEE Trans. in Systems, Man and Cybernetics, 29(4):393–399, 1999.CrossRefGoogle Scholar
  19. 19.
    D. Weinshall Y. Gdalyahu and M. Werman. A randomized algorithm for pairwise clustering. In Advances in Neural Information Processing Systems 11, pages 424–430. MIT Press, 1999.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Antonio Robles-Kelly
    • 1
  • Edwin R. Hancock
    • 1
  1. 1.Department of Computer ScienceUniversity of YorkYorkUK

Personalised recommendations