Bayesian and information-theoretic priors for Bayesian network parameters

  • Petri Kontkanen
  • Petri Myllymäki
  • Tomi Silander
  • Henry Tirri
  • Peter Grünwalde
Bayesian Networks
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1398)


We consider Bayesian and information-theoretic approaches for determining non-informative prior distributions in a parametric model family. The information-theoretic approaches are based on the recently modified definition of stochastic complexity by Rissanen, and on the Minimum Message Length (MML) approach by Wallace. The Bayesian alternatives include the uniform prior, and the equivalent sample size priors. In order to be able to empirically compare the different approaches in practice, the methods are instantiated for a model family of practical importance, the family of Bayesian networks.


Bayesian Network Prior Distribution Dirichlet Form Predictive Distribution Fisher Information Matrix 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    R.A. Baxter and J.O. Oliver. MDL and MML: Similarities and differences. Technical Report 207, Department of Computer Science, Monash University, 1994.Google Scholar
  2. 2.
    J.O. Berger. Statistical Decision Theory and Bayesian Analysis. Springer-Verlag, New York, 1985.Google Scholar
  3. 3.
    B.S. Claxke and A.R. Barron. Jeffrey's prior is asymptotically least favorable under entropy risk. Journal of Statistical Planning and Inference, 41:37–60, 1994.CrossRefGoogle Scholar
  4. 4.
    G. Cooper and E. Herskovits. A Bayesian method for the induction of probabilistic networks from data. Machine Learning, 9:309–347, 1992.Google Scholar
  5. 5.
    D. Heckerman. A tutorial on learning with Bayesian networks. Technical Report MSR-TR-95-06, Microsoft Research, Advanced Technology Division, One Microsoft Way, Redmond, WA 98052, 1996.Google Scholar
  6. 6.
    D. Heckerman, D. Geiger, and D.M. Chickering. Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning, 20(3):197–243, September 1995.Google Scholar
  7. 7.
    P. Kontkanen, P. Myllymäki, T. Silander, H. Tirri, and P. Grünwald. On predictive distributions and Bayesian networks. Technical Report NC-TR-97-032, ESPRIT Working Group on Neural and Computational Learning (NeuroCOLT), 1997.Google Scholar
  8. 8.
    P. Kontkanen, P. Myllymäki, T. Silander, H. Tirri, and P. Grünwald. A comparison of non-informative priors for Bayesian networks. Technical Report NC-TR-98-002, ESPRIT Working Group on Neural and Computational Learning (NeuroCOLT), 1998.Google Scholar
  9. 9.
    J. Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers,San Mateo, CA, 1988.Google Scholar
  10. 10.
    J. Rissanen. Stochastic complexity. Journal of the Royal Statistical Society, 49(3):223–239 and 252–265, 1987.Google Scholar
  11. 11.
    J. Rissanen. Stochastic Complexity in Statistical Inquiry. World Scientific Publishing Company, New Jersey, 1989.Google Scholar
  12. 12.
    J. Rissanen. Fisher information and stochastic complexity. IEEE Transactions on Information Theory, 42(1):40–47, January 1996.CrossRefGoogle Scholar
  13. 13.
    R.D. Shachter. Probabilistic inference and influence diagrams. Operations Research, 36(4):589–604, July-August 1988.Google Scholar
  14. 14.
    C.S. Wallace and D.M Boulton. An information measure for classifiation. Computer Journal, 11:185–194, 1968.Google Scholar
  15. 15.
    C.S. Wallace and P.R. Freeman. Estimation and inference by compact coding. Journal of the Royal Statistical Society, 49(3):240–265, 1987.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1998

Authors and Affiliations

  • Petri Kontkanen
    • 1
  • Petri Myllymäki
    • 1
  • Tomi Silander
    • 1
  • Henry Tirri
    • 1
  • Peter Grünwalde
    • 2
  1. 1.Complex Systems Computation Group (CoSCo) P.O.Box 26, Department of Computer ScienceFIN-00014 University of HelsinkiFinland
  2. 2.Department of Algorithms and ArchitecturesCWINetherlands

Personalised recommendations