Probabilistic network construction using the minimum description length principle

  • Remco R. Bouckaert
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 747)


This paper presents a procedure for the construction of probabilistic networks from a database of observations based on the minimum description length principle. On top of the advantages of the Bayesian approach the minimum description length principle offers the advantage that every probabilistic network structure that represents the same set of independencies gets assigned the same quality. This makes it is very suitable for the order optimization procedure as described in [4]. Preliminary test results show that the algorithm performs comparable to the algorithm based on the Bayesian approach [6].


Network Structure Bayesian Approach Directed Acyclic Graph Minimum Description Length Conditional Probability Table 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    S. Andreassen, M. Wolbye, B. Falck, and S.K. Andersen. Munim — a causal probabilistic network for interpretation of electromyographic findings. In Proceedings of the IJCAI, pages 366–372, Milan, Italy, 1987.Google Scholar
  2. [2]
    I. Beinlich, H. Seurmondt, R. Chavez, and G. Cooper. The alarm monitoring system: a case study with two probabilistic inference techniques for belief networks. In Proceedings Artificial Intelligence in Medical Care, pages 247–256, London, 1989.Google Scholar
  3. [3]
    R.R. Bouckaert. Belief network construction using the minimum description length principle. Technical Report RUU-CS-93-??, To appear, Utrecht University, The Netherlands, 1992.Google Scholar
  4. [4]
    R.R. Bouckaert. Optimizing causal orderings for generating dags from data. In Proceedings Uncertainty in Artificial Intelligence, pages 9–16, 1992.Google Scholar
  5. [5]
    C.K. Chow and C.N. Liu. Approximating discrete probability distributions with dependency trees. IEEE Trans. on Information Theory, IT-14, pages 462–467, 1986.Google Scholar
  6. [6]
    G.F. Cooper and E. Herskovits. A bayesian method for the induction of probabilistic networks from data. Machine Learning, pages 309–347, 1992.Google Scholar
  7. [7]
    D. Heckerman, E. Horvitz, and B. Nathwani. Towards normative expert systems: Part I, the pathfinder project. Methods of Information in Medicine, 31:90–105, 1992.Google Scholar
  8. [8]
    M. Henrion. An introduction to algorithms for inference in belief nets. In Proceedings Uncertainty in Artificial Intelligence 5, pages 129–138, 1990.Google Scholar
  9. [9]
    E. Herskovits. Computer-based probabilistic-network construction. PhD thesis, Section of Medical Informatics, University of Pittsburgh, 1991.Google Scholar
  10. [10]
    S. Højsgaard and B. Thiesson. Bifrost-block recursive models induced from relevant knowledge, observationsm and statistical techniques. Technical Report R 92-2010, Institute for Electronic Systems, University of Aalborg, Denmark, June 1992.Google Scholar
  11. [11]
    S.L. Lauritzen and D.J. Spiegelhalter. Local computations with probabilities on graphical structures and their applications to expert systems (with discussion). J.R. Stat. Soc. (Series B),Vol. 50, pages 157–224, 1988.Google Scholar
  12. [12]
    S.L. Lauritzen, B. Thiesson, and D.J. Spiegelhalter. Diagnostic systems created by model selection methods — a case study. In Proceedings 4th International Workshop on AI and Statistics, 1993.Google Scholar
  13. [13]
    J. Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufman, inc., San Mateo, CA, 1988.Google Scholar
  14. [14]
    G. Rebane and J. Pearl. The recovery of causal polytrees from statistical data. In Proceedings Uncertainty in Artificial Intelligence, pages 222–228, 1987.Google Scholar
  15. [15]
    J. Rissanen. Stochastic complexity and modeling. Annals of Statistics, 14(3):1080–1100, 1986.Google Scholar
  16. [16]
    J. Rissanen. Stochastic complexity. Journal of the Royal Statistical Society B, 49(3):223–239, 1987.Google Scholar
  17. [17]
    R.D. Robinson. Counting unlabeled acyclic digraphs. In Proceedings of the fifth Australian Conference on Combinatorial Mathematics, pages 28–43, Melbourn, Australia, 1976.Google Scholar
  18. [18]
    M.A. Shwe, B. Middleton, D.E. Heckerman, M. Henrion, E. Horvitz, H. Lehmann, and G. Cooper. Probabilistic diagnosis using a reformulation of the internist-1/qmr knowledge base: I the probabilistic modal and inference algorithms. Methods of Information in Medicine, 30:241–255, 1991.Google Scholar
  19. [19]
    P. Spirtes, C. Glymour, and R. Scheines. Causation, Prediction, and Search. To appear, 1993.Google Scholar
  20. [20]
    T. Verma and J. Pearl. Causal networks: Semantics and expressiveness. In Proceedings Uncertainty in Artificial Intelligence, pages 352–359, 1988.Google Scholar
  21. [21]
    N. Wermuth and S.L. Lauritzen. Graphical and recursive models for contingency tables. Biometrika, 72:537–552, 1983.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1993

Authors and Affiliations

  • Remco R. Bouckaert
    • 1
  1. 1.Department of Computer ScienceUtrecht UniversityTB UtrechtThe Netherlands

Personalised recommendations