Abstract
Gaussian mixture model (GMM) is one of the important models to approximate probability distributions. There are various methods for Gaussian mixture estimation such as the EM algorithm, sampling method, and the Bayes method. In this paper, we are concerned with the Gaussian mixture estimation problem using the variational Bayes (VB), which is an approximation of the Bayes method. In the VB, it is important to choose its initial values carefully since the objective function of the problem is multimodal. In this paper, we propose a method which employs primitive initial point (PIP) as an initial value of the VB and performs multi-directional search from the PIP. We present the motivation and rationale of our method and demonstrate its effectiveness through numerical experiments using real data sets.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
McLachlan, G.J., Peel, D.: Finite mixture models. John Wiley & Sons, Chichester (2000)
Attias, H.: Inferring parameters and structure of latent variable models by variational bayes. In: Proc. of the 15th Conf. on Uncertainty in Artificial Intelligence, pp. 21–30 (1999)
Ueda, N., Ghahramani, Z.: Bayesian model search for mixture models based on optimizing variational bounds. Neural Networks 15(10), 1223–1241 (2002)
Saul, L.K., Jaakkola, T., Jordan, M.I.: Mean field theory for sigmoid belief networks. Journal of Artificial Intelligence Research 4, 61–76 (1996)
Opper, M., Saad, D. (eds.): Advanced mean field method: theory and practice. MIT Press, Cambridge (2001)
Corduneanu, A., Bishop, C.M.: Variational Bayesian model Selection for mixture distributions. In: Proc. of 8th Int. Conf. on Artificial Intelligence and Statistics, pp. 27–34 (2001)
Ueda, N., Nakano, R.: Deterministic annealing EM algorithm. Neural Networks 11(2), 271–282 (1998)
Katahira, K., Watanabe, K., Okada, M.: Deterministic annealing variant of variational Bayes method. In: Proc. of Int. Workshop on Statistical-Mechanical Informatics, pp. 65–73 (2007)
Ishikawa, Y., Nakano, R.: Landscape of a likelihood surface for a Gaussian mixture and its use for the EM algorithm. In: Proc. of the Int. Joint Conf. on Neural Networks, pp. 1434–1440 (2006)
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Heidelberg (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ishikawa, Y., Takeuchi, I., Nakano, R. (2009). Variational Bayes from the Primitive Initial Point for Gaussian Mixture Estimation. In: Leung, C.S., Lee, M., Chan, J.H. (eds) Neural Information Processing. ICONIP 2009. Lecture Notes in Computer Science, vol 5863. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-10677-4_18
Download citation
DOI: https://doi.org/10.1007/978-3-642-10677-4_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-10676-7
Online ISBN: 978-3-642-10677-4
eBook Packages: Computer ScienceComputer Science (R0)