Abstract
A normal mixture model, which belongs to singular learning machines, is widely used in statistical pattern recognition. In singular learning machines, the Bayesian learning provides the better generalization performance than the maximum likelihood estimation. However, it needs huge computational cost to realize the Bayesian posterior distribution by the conventional Monte Carlo method. In this paper, we propose that the exchange Monte Carlo method is appropriate for the Bayesian learning in singular learning machines, and experimentally show that it provides better generalization performance in the Bayesian learning of a normal mixture model than the conventional Monte Carlo method.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Watanabe, S.: Algebraic analysis for nonidentifiable learning machines. Neural Computation 13(4), 899–933 (2001)
Yamazaki, K., Watanabe, S.: Singularities in mixture models and upper bounds of stochastic complexity. Neural Networks 16(7), 1029–1038 (2003)
Nakano, N., Takahashi, K., Watanabe, S.: On the Evaluation Criterion of the MCMC Method in Singular Learning Machines. Trans. of IEICE. J88-D-2(10), 2011–2020 (2005)
Iba, Y.: Extended Ensemble Monte Carlo. International Journal of Modern Physics C12, 623–656 (2001)
Hukushima, K., Nemoto, K.: Exchange Monte Carlo Method and Application to Spin Glass Simulation. Journal of the Physical Society of Japan 65(6), 1604–1608 (1996)
Sengupta, P., Sandvik, A.W., Campbell, D.K.: Bond-order-wave phase and quantum phase transitions in the one dimensional extended Hubbard model. Physical Review B 65, 155113 (2002)
Pinn, K., Wieczerkowski, C.: Number of magic squares from parallel tempering Monte Carlo. Int. J. Mod. Phys. C9, 541 (1998)
Hukushima, K.: Extended ensemble Monte Carlo approach to hardly relaxing problems. Computer Physics Communications 147, 77–82 (2002)
Nagata, K., Watanabe, S.: Exchange Monte Carlo Method for Bayesian Learning in Singular Learning Machines. In: Proc of International Joint Conference on Neural Networks (IJCNN2006) (to appear)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nagata, K., Watanabe, S. (2006). Generalization Performance of Exchange Monte Carlo Method for Normal Mixture Models. In: Corchado, E., Yin, H., Botti, V., Fyfe, C. (eds) Intelligent Data Engineering and Automated Learning – IDEAL 2006. IDEAL 2006. Lecture Notes in Computer Science, vol 4224. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11875581_15
Download citation
DOI: https://doi.org/10.1007/11875581_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-45485-4
Online ISBN: 978-3-540-45487-8
eBook Packages: Computer ScienceComputer Science (R0)