Abstract
We propose a Bayesian method for learning Bayesian network models using Markov chain Monte Carlo (MCMC). In contrast to most existing MCMC approaches that define components in term of single edges, our approach is to decompose a Bayesian network model in larger dependence components defined by Markov blankets. The idea is based on the fact that MCMC performs significantly better when choosing the right decomposition, and that edges in the Markov blanket of the vertices form a natural dependence relationship. Using the ALARM and Insurance networks, we show that this decomposition allows MCMC to mix more rapidly, and is less prone to getting stuck in local maxima compared to the single edge approach.
Chapter PDF
Similar content being viewed by others
References
Beinlich, I.A., Suermondt, H.J., Chavez, R.M., Cooper, G.F.: The ALARM monitoring system: A case study with two probabilistic inference techniques for belief networks. In: Proc. of the European Conf. on AI in Medicine (1989)
Binder, J., Koller, D., Russell, S.J., Kanazawa, K.: Adaptive probabilistic networks with hidden variables. Machine Learning 29, 213–244 (1997)
Castelo, R., Kocka, T.: On inclusion-driven learning of Bayesian networks. J. of Machine Learning Research 4, 527–574 (2003)
Friedman, N., Koller, D.: Being Bayesian about network structure. A Bayesian approach to structure discovery in Bayesian networks. Machine Learning 50(1–2), 95–125 (2003)
Giudici, P., Castelo, R.: Improving Markov chain Monte Carlo model search for data mining. Machine Learning 50(1), 127–158 (2003)
Giudici, P., Green, P.: Decomposable graphical gaussian model determination. Biometrika 86(4), 785–801 (1999)
Green, P.: Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82, 711–732 (1998)
Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning 20, 197–243 (1995)
Kocka, T., Castelo, R.: Improved learning of Bayesian networks. In: Koller, D., Breese, J. (eds.) Proc. of the Conf. on Uncertainty in AI, pp. 269–276 (2001)
Madigan, D., Raftery, A.: Model selection and accounting for model uncertainty in graphical models using Occam’s window. J. of the Am. Stat. Assoc. 89, 1535–1546 (1994)
Madigan, D., York, J.: Bayesian graphical models for discrete data. Intl. Statistical Review 63, 215–232 (1995)
Robert, C.P., Casella, G.: Monte Carlo statistical methods, 3rd edn. Springer, Heidelberg (2002)
Spiegelhalter, D.J., Lauritzen, S.L.: Sequential updating of conditional probabilities on directed graphical structures. Networks 20, 579–605 (1990)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Riggelsen, C. (2005). MCMC Learning of Bayesian Network Models by Markov Blanket Decomposition. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds) Machine Learning: ECML 2005. ECML 2005. Lecture Notes in Computer Science(), vol 3720. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11564096_33
Download citation
DOI: https://doi.org/10.1007/11564096_33
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29243-2
Online ISBN: 978-3-540-31692-3
eBook Packages: Computer ScienceComputer Science (R0)