Abstract
It has been recently demonstrated that the classical EM algorithm for learning Gaussian mixture models can be successfully implemented in a decentralized manner by resorting to gossip-based randomized distributed protocols. In this paper we describe a gossip-based implementation of an alternative algorithm for learning Gaussian mixtures in which components are added to the mixture one after another. Our new Greedy Gossip-based Gaussian mixture learning algorithm uses gossip-based parallel search, starting from multiple initial guesses, for finding good components to add to the mixture in each component allocation step. It can be executed on massive networks of small computing devices, converging to a solution exponentially faster than its centralized version, while reaching the same quality of generated models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
McLachlan, G.J., Peel, D.: Finite Mixture Models. John Wiley & Sons, Chichester (2000)
Moore, A.W.: Very fast EM-based mixture model clustering using multiresolution kd-trees. In: Advances in Neural Information Processing Systems, vol. 11. MIT Press, Cambridge (1999)
Tipping, M.E., Bishop, C.M.: Mixtures of probabilistic principal component analysers. Neural Computation 11(2), 443–482 (1999)
Titsias, M., Likas, A.: Shared kernel models for class conditional density estimation. IEEE Trans. Neural Networks 12(5), 987–997 (2001)
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Statist. Soc. B 39, 1–38 (1977)
Meila, M., Heckerman, D.: An experimental comparison of several clustering and initialization methods. In: Proc. 14th Ann. Conf. on Uncertainty in Artificial Intelligence, San Francisco, CA, pp. 386–395 (1998)
Li, J.Q., Barron, A.R.: Mixture density estimation. In: Advances in Neural Information Processing Systems, vol. 12. The MIT Press, Cambridge (2000)
Vlassis, N., Likas, A.: A greedy EM algorithm for Gaussian mixture learning. Neural Processing Letters 15(1), 77–87 (2002)
Verbeek, J.J., Vlassis, N., Kröse, B.: Efficient greedy learning of Gaussian mixture models. Neural Computation 15(2), 469–485 (2003)
Ueda, N., Nakano, R., Ghahramani, Z., Hinton, G.E.: SMEM algorithm for mixture models. Neural Computation 12, 2109–2128 (2000)
Kowalczyk, W., Vlassis, N.: Newscast EM. In: Saul, L.K., Weiss, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems, vol. 17. MIT Press, Cambridge (2005)
Neal, R.M., Hinton, G.E.: A view of the EM algorithm that justifies incremental, sparse, and other variants. In: Jordan, M.I. (ed.) Learning in graphical models, pp. 355–368. Kluwer Academic Publishers, Dordrecht (1998)
Karp, R., Schindelhauer, C., Shenker, S., Vöcking, B.: Randomized rumour spreading. In: Proc. 41th IEEE Symp. on Foundations of Computer Science, Redondo Beach, CA (2000)
Kempe, D., Dobra, A., Gehrke, J.: Gossip-based computation of aggregate information. In: Proc. 44th IEEE Symp. on Foundations of Computer Science, Cambridge, MA (2003)
Boyd, S., Ghosh, A., Prabhakar, B., Shah, D.: Gossip algorithms: Design, analysis and applications. In: Proc. IEEE Infocom, Miami, FL (2005)
Forman, G., Zhang, B.: Distributed data clustering can be efficient and exact. ACM SIGKDD Explorations 2(2), 34–38 (2000)
Nowak, R.D.: Distributed EM algorithms for density estimation and clustering in sensor networks. IEEE Trans. on Signal Processing 51(8), 2245–2253 (2003)
Dasgupta, S.: Learning mixtures of Gaussians. In: Proc. IEEE Symp. on Foundations of Computer Science, New York (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Vlassis, N., Sfakianakis, Y., Kowalczyk, W. (2005). Gossip-Based Greedy Gaussian Mixture Learning. In: Bozanis, P., Houstis, E.N. (eds) Advances in Informatics. PCI 2005. Lecture Notes in Computer Science, vol 3746. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11573036_33
Download citation
DOI: https://doi.org/10.1007/11573036_33
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29673-7
Online ISBN: 978-3-540-32091-3
eBook Packages: Computer ScienceComputer Science (R0)