Abstract
A topic in probabilistic network learning is to exploit local network structure, i.e., to capture regularities in the conditional probability distributions, and to learn networks with such local structure from data. In this paper we present a modification of the learning algorithm for Bayesian networks with a local decision graph representation suggested in Chickering et al. (1997), which is often more efficient. It rests on the idea to exploit the decision graph structure not only to capture a larger set of regularities than decision trees can, but also to improve the learning process. In addition, we study the influence of the properties of the evaluation measure used on the learning time and identify three classes of evaluation measures.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
Bibliography
S.K. Andersen, K.G. Olesen, F.V. Jensen, and F. Jensen. HUGIN — A shell for building Bayesian belief universes for expert systems. Proc. 11th Int. J. Conf. on Artificial Intelligence, 1080–1085, 1989
P.W. Bairn. A Method for Attribute Selection in Inductive Learning Systems. IEEE Trans. on Pattern Analysis and Machine Intelligence, 10: 888–896, 1988
C. Borgelt and R. Kruse. Evaluation Measures for Learning Probabilistic and Possibilistic Networks. Proc. 6th IEEE Int. Conf. on Fuzzy Systems (FUZZ-IEEE’97), Vol. 2: pp. 1034–1038, Barcelona, Spain, 1997
C. Borgelt and R. Kruse. Some Experimental Results on Learning Probabilistic and Possibilistic Networks with Different Evaluation Measures. Proc. 1st Int. Joint Conference on Qualitative and Quantitative Practical Reasoning (ECSQARU/FAPR’97), pp. 71–85, Springer, Berlin, Germany, 1997 )
C. Borgelt and R. Kruse. Graphical Models — Methods for Data Analysis and Mining. J. Wiley & Sons, Chichester, United Kingdom 2002
C. Boutilier, N. Friedman, M. Goldszmidt, and D. Koller. Context Specific Independence in Bayesian Networks. Proc. 12th Conf. on Uncertainty in Artificial Intelligence (UAI’96), Portland, OR, 1996
L. Breiman, J.H. Friedman, R.A. Olshen, and C.J. Stone. Classification and Regression Trees, Wadsworth International Group, Belmont, CA, 1984
W. Buntine. Theory Refinement on Bayesian Networks. Proc. 7th Conf. on Uncertainty in Artificial Intelligence, pp. 52–60, Morgan Kaufman, Los Angeles, CA, 1991
D.M. Chickering, D. Heckerman, and C. Meek. A Bayesian Approach to Learning Bayesian Networks with Local Structure. Proc. 13th Conf. on Uncertainty in Artificial Intelligence (UAI’97), pp. 80–89, Morgan Kaufman, San Franscisco, CA, 1997
C.K. Chow and C.N. Liu. Approximating Discrete Probability Distributions with Dependence Trees. IEEE Trans. on Information Theory 14 (3): 462–467, 1968
G.F. Cooper and E. Herskovits. A Bayesian Method for the Induction of Probabilistic Networks from Data. Machine Learning 9: 309–347, 1992
J. Gebhardt and R. Kruse. The context model — an integrating view of vagueness and uncertainty Int. Journal of Approximate Reasoning 9: 283–314, 1993
J. Gebhardt and R. Kruse. POSSINFER A Software Tool for Possibilistic Inference. In: D. Dubois, H. Prade, and R. Yager, eds. Fuzzy Set Methods in Information Engineering: A Guided Tour of Applications, Wiley 1995
J. Gebhardt and R. Kruse. Learning Possibilistic Networks from Data. Proc. 5th Int. Workshop on Artificial Intelligence and Statistics, 233–244, Fort Lauderdale, 1995
J. Gebhardt and R. Kruse. Tightest Hypertree Decompositions of Multivariate Possibility Distributions. Proc. Int. Conf. on Information Processing and Management of Uncertainty in Knowledge-based Systems, 1996
J. Gebhardt. Learning from Data: Possibilistic Graphical Models. Habil. thesis, University of Braunschweig, Germany 1997
D. Geiger and D. Heckerman. Advances in Probabilistic Reasoning. Proc. 7th Conf. on Uncertainty in Artificial Intelligence (UAI’91), pp. 118–126, Morgan Kaufman, San Franscisco, CA, 1997
D. Heckerman. Probabilistic Similarity Networks. MIT Press 1991
D. Heckerman, D. Geiger, and D.M. Chickering. Learning Bayesian Networks: The Combination of Knowledge and Statistical Data. Machine Learning 20: 197–243, 1995
M. Higashi and G.J. Klir. Measures of Uncertainty and Information based on Possibility Distributions. Int. Journal of General Systems 9: 43–58, 1982
K. Kira and L. Rendell. A Practical Approach to Feature Selection. Proc. 9th Int. Conf. on Machine Learning (ICML’92), pp. 250–256, Morgan Kaufman, San Franscisco, CA, 1992
G.J. Klir and M. Mariano. On the Uniqueness of a Possibility Measure of Uncertainty and Information. Fuzzy Sets and Systems 24: 141–160, 1987
I. Kononenko. Estimating Attributes: Analysis and Extensions of RELIEF. Proc. 7th Europ. Conf. on Machine Learning (ECML’94), Springer, New York, NY, 1994
I. Kononenko. On Biases in Estimating Multi-Valued Attributes. Proc. 1st Int. Conf. on Knowledge Discovery and Data Mining, 1034–1040, Montreal, 1995
R.E. Krichevsky and V.K. Trofimov. The Performance of Universal Coding. IEEE Trans. on Information Theory, 27 (2): 199–207, 1983
R. Kruse, E. Schwecke, and J. Heinsohn. Uncertainty and Vagueness in Knowledge-based Systems: Numerical Methods. Series: Artificial Intelligence, Springer, Berlin 1991
R. Kruse, J. Gebhardt, and F. Klawonn. Foundations of Fuzzy Systems, John Wiley & Sons, Chichester, England 1994
S. Kullback and R.A. Leibler. On Information and Sufficiency. Ann. Math. Statistics 22: 79–86, 1951
S.L. Lauritzen and D.J. Spiegelhalter. Local Computations with Probabilities on Graphical Structures and Their Application to Expert Systems. Journal of the Royal Statistical Society, Series B, 2 (50): 157–224, 1988
R. Lopez de Mantaras. A Distance-based Attribute Selection Measure for Decision Tree Induction. Machine Learning 6: 81–92, 1991
H.T. Nguyen. Using Random Sets. Information Science 34: 265–274, 1984
J. Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference (2nd edition). Morgan Kaufman, New York 1992
D. Poole. Probabilistic Horn Abduction and Bayesian Networks. Artificial Intelligence, 64 (1): 81–129, 1993
J.R. Quinlan. C.4.5: Programs for Machine Learning, Morgan Kaufman, 1993
L.K. Rasmussen. Blood Group Determination of Danish Jersey Cattle in the F-blood Group System. Dina Research Report no. 8, 1992
J. Rissanen. Stochastic Complexity. Journal of the Royal Statistical Society (Series B), 49: 223–239, 1987
A. Saffiotti and E. Umkehrer. PULCINELLA: A General Tool for Propagating Uncertainty in Valuation Networks. Proc. 7th Conf. on Uncertainty in AI, 323–331, San Mateo 1991
G. Shafer and P.P. Shenoy. Local Computations in Hypertrees. Working Paper 201, School of Business, University of Kansas, Lawrence 1988
P.P. Shenoy. Valuation-based Systems: A Framework for Managing Uncertainty in Expert Systems. Working Paper 226, School of Business, University of Kansas, Lawrence, 1991
J.E. Smith, S. Holtzman, and J.E. Matheson. Structuring Conditional Relationships in Influence Diagrams Operations Research, 41 (2): 280–297, 1993
L. Wehenkel. On Uncertainty Measures Used for Decision Tree Induction. Proc. IPMU, 1996
X. Zhou and T.S. Dillon. A statistical-heuristic Feature Selection Criterion for Decision Tree Induction. IEEE Trans. on Pattern Analysis and Machine Intelligence, 13: 834–841, 1991
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Wien
About this chapter
Cite this chapter
Borgelt, C., Kruse, R. (2003). Local Structure Learning in Graphical Models. In: Della Riccia, G., Dubois, D., Kruse, R., Lenz, HJ. (eds) Planning Based on Decision Theory. International Centre for Mechanical Sciences, vol 472. Springer, Vienna. https://doi.org/10.1007/978-3-7091-2530-4_7
Download citation
DOI: https://doi.org/10.1007/978-3-7091-2530-4_7
Publisher Name: Springer, Vienna
Print ISBN: 978-3-211-40756-1
Online ISBN: 978-3-7091-2530-4
eBook Packages: Springer Book Archive