Abstract
The mixture modeling framework is widely used in many applications. In this paper, we propose a component reduction technique, that collapses a mixture model into a mixture with fewer components. For fitting a mixture model to data, the EM (Expectation-Maximization) algorithm is usually used. Our algorithm is derived by extending mixture model learning using the EM-algorithm.
In this extension, a difficulty arises from the fact that some crucial quantities cannot be evaluated analytically. We overcome this difficulty by introducing an effective approximation. The effectiveness of our algorithm is demonstrated by applying it to a simple synthetic component reduction task and a phoneme clustering problem.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Vasconcelos, N., Lippman, A.: Learning mixture hierarchies. In: Kearns, M.J., Solla, S.A., Cohn, D. (eds.) Advances in Neural Information Processing Systems, vol. 11, pp. 606–612 (1999)
Dempster, A., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society B 39, 1–38 (1977)
McLachlan, G.J., Krishnan, T.: The EM Algorithm and Extensions. John Wiley and Sons Inc., Chichester (1997)
McLachlan, G., Peel, D.: Finite Mixture Models. John Wiley and Sons Inc., Chichester (2000)
Goldberger, J., Roweis, S.: Hierarchical clustering of a mixture model. In: Saul, L.K., Weiss, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems 17, pp. 505–512. MIT Press, Cambridge (2005)
Hastie, T., Tibshirani, R., Friedman, J.H.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, Heidelberg (2001)
Strehl, A., Ghosh, J.: Cluster ensembles - a knowledge reuse framework for combining multiple partitions. Machine Learning Research 3, 583–617 (2002)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Maebashi, K., Suematsu, N., Hayashi, A. (2008). Component Reduction for Hierarchical Mixture Model Construction. In: Ishikawa, M., Doya, K., Miyamoto, H., Yamakawa, T. (eds) Neural Information Processing. ICONIP 2007. Lecture Notes in Computer Science, vol 4985. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69162-4_34
Download citation
DOI: https://doi.org/10.1007/978-3-540-69162-4_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-69159-4
Online ISBN: 978-3-540-69162-4
eBook Packages: Computer ScienceComputer Science (R0)