Some experimental results on learning probabilistic and possibilistic networks with different evaluation measures

  • Christian Borgelt
  • Rudolf Kruse
Invited Papers
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1244)


A large part of recent research on probabilistic and possibilistic inference networks has been devoted to learning them from data. In this paper we discuss two search methods and several evaluation measures usable for this task. We consider a scheme for evaluating induced networks and present experimental results obtained from an application of INES (Induction of NEtwork Structures), a prototype implementation of the described methods and measures.


Mutual Information Evaluation Measure Marginal Distribution Domain Expert Information Gain 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    S.K. Andersen, K.G. Olesen, F.V. Jensen, and F. Jensen. HUGIN — A shell for building Bayesian belief universes for expert systems. Proc. 11th Int. J. Conf. on Artificial Intelligence, 1080–1085, 1989Google Scholar
  2. 2.
    L. Breiman, J.H. Friedman, R.A. Olshen, and C.J. Stone. Classification and Regression Trees, Wadsworth International Group, Belmont, CA, 1984Google Scholar
  3. 3.
    C.K. Chow and C.N. Liu. Approximating Discrete Probability Distributions with Dependence Trees. IEEE Trans. on Information Theory 14(3):462–467, IEEE 1968Google Scholar
  4. 4.
    G.F. Cooper and E. Herskovits. A Bayesian Method for the Induction of Probabilistic Networks from Data. Machine Learning 9:309–347, Kluwer 1992Google Scholar
  5. 5.
    Z. Daróczy. Generalized Information Functions. Information and Control 16:36–51, 1970Google Scholar
  6. 6.
    J. Gebhardt and R. Kruse. A Possibilistic Interpretation of Fuzzy Sets in the Context Model. Proc. IEEE Int. Conf. on Fuzzy Systems, 1089–1096, San Diego 1992.Google Scholar
  7. 7.
    J. Gebhardt and R. Kruse. POSSINFER — A Software Tool for Possibilistic Inference. In: D. Dubois, H. Prade, and R. Yager, eds. Fuzzy Set Methods in Information Engineering: A Guided Tour of Applications, Wiley 1995Google Scholar
  8. 8.
    J. Gebhardt and R. Kruse. Learning Possibilistic Networks from Data. Proc. 5th Int. Workshop on AI and Statistics, 233–244, Fort Lauderdale, 1995Google Scholar
  9. 9.
    J. Gebhardt and R. Kruse. Tightest Hypertree Decompositions of Multivariate Possibility Distributions. Proc. Int. Conf. on Information Processing and Management of Uncertainty in Knowledge-based Systems, 1996Google Scholar
  10. 10.
    R.V.L. Hartley. Transmission of Information. The Bell Systems Technical Journal 7:535–563, 1928Google Scholar
  11. 11.
    D. Heckerman. Probabilistic Similarity Networks. MIT Press 1991Google Scholar
  12. 12.
    D. Heckerman, D. Geiger, and D.M. Chickering. Learning Bayesian Networks: The Combination of Knowledge and Statistical Data. Machine Learning 20:197–243, Kluwer 1995Google Scholar
  13. 13.
    M. Higashi and G.J. Klir. Measures of Uncertainty and Information based on Possibility Distributions. Int. Journal of General Systems 9:43–58, 1982Google Scholar
  14. 14.
    G.J. Klir and M. Mariano. On the Uniqueness of a Possibility Measure of Uncertainty and Information. Fuzzy Sets and Systems 24:141–160, 1987Google Scholar
  15. 15.
    I. Kononenko. On Biases in Estimating Multi-Valued Attributes. Proc. 1st Int. Conf. on Knowledge Discovery and Data Mining, 1034–1040, Montreal, 1995Google Scholar
  16. 16.
    R. Kruse, E. Schwecke, and J. Heinsohn. Uncertainty and Vagueness in Knowledge-based Systems: Numerical Methods. Springer, Berlin1991Google Scholar
  17. 17.
    R. Kruse, J. Gebhardt, and F. Klawonn. Foundations of Fuzzy Systems, John Wiley & Sons, Chichester, England 1994Google Scholar
  18. 18.
    S. Kullback and R.A. Leibler. On Information and Sufficiency. Ann. Math. Statistics 22:79–86, 1951Google Scholar
  19. 19.
    S.L. Lauritzen and D.J. Spiegelhalter. Local Computations with Probabilities on Graphical Structures and Their Application to Expert Systems. Journal of the Royal Statistical Society, Series B, 2(50):157–224, 1988Google Scholar
  20. 20.
    R.L. de Mantaras. A Distance-based Attribute Selection Measure for Decision Tree Induction. Machine Learning 6:81–92, Kluwer 1991Google Scholar
  21. 21.
    H.T. Nguyen. Using Random Sets. Information Science 34:265–274, 1984Google Scholar
  22. 22.
    J. Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference (2nd edition). Morgan Kaufman, New York 1992Google Scholar
  23. 23.
    J.R. Quinlan. Induction of Decision Trees. Machine Learning 1:81–106, 1986Google Scholar
  24. 24.
    J.R. Quinlan. C4-5: Programs for Machine Learning, Morgan Kaufman, 1993Google Scholar
  25. 25.
    L.K. Rasmussen. Blood Group Determination of Danish Jersey Cattle in the F-blood Group System. Dina Research Report no. 8, 1992Google Scholar
  26. 26.
    J. Rissanen. A Universal Prior for Integers and Estimation by Minimum Description Length. Annals of Statistics 11:416–431, 1983Google Scholar
  27. 27.
    A. Saffiotti and E. Umkehrer. PULCINELLA: A General Tool for Propagating Uncertainty in Valuation Networks. Proc. 7th Conf. on Uncertainty in AI, 323–331, San Mateo 1991Google Scholar
  28. 28.
    G. Shafer and P.P. Shenoy. Local Computations in Hypertrees. Working Paper 201, School of Business, University of Kansas, Lawrence 1988Google Scholar
  29. 29.
    C.E. Shannon. The Mathematical Theory of Communication. The Bell Systems Technical Journal 27:379–423, 1948Google Scholar
  30. 30.
    M. Singh and M. Valtorta. An Algorithm for the Construction of Bayesian Network Structures from Data. Proc. 9th Conf. on Uncertainty in AI, 259–265, Morgan Kaufman, 1993Google Scholar
  31. 31.
    L. Wehenkel. On Uncertainty Measures Used for Decision Tree Induction. Proc. IPMU, 1996Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1997

Authors and Affiliations

  • Christian Borgelt
    • 1
  • Rudolf Kruse
    • 1
  1. 1.Department of Information and Communication SystemsOtto-von-Guericke-University of MagdeburgMagdeburgGermany

Personalised recommendations