Skip to main content
SpringerLink
Log in

An Introduction to Probabilistic Graphical Models

  • Chapter
Estimation of Distribution Algorithms

Part of the book series: Genetic Algorithms and Evolutionary Computation ((GENA,volume 2))

Abstract

In this chapter we will introduce two probabilistic graphical models -Bayesian networks and Gaussian networks-that will be used to carry out factorization of the probability distribution of the selected individuals in the Estimation of Distribution Algorithms based approaches. For both paradigms we will present different algorithms to induce the underlying model from data, as well as some methods to simulate such models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Akaike, H. (1974). New look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6):716–723.

    Article  MathSciNet  MATH  Google Scholar 

  • Best, D. J. (1979). Some easily programmed pseudo-random normal generators. Aust. Comp. J., 11:60–62.

    MATH  Google Scholar 

  • Bouckaert, R. R. (1194). A stratified simulation scheme for inference in Bayesian belief networks, pages 110–117. Morgan Kaufmann Publishers, San Francisco.

    Google Scholar 

  • Bouckaert, R. R. (1995). Bayesian Belief Networks: From Construction to Inference. PhD Thesis, University of Utrecht.

    Google Scholar 

  • Bouckaert, R. R., Castillo, E., and Gutiérrez, J. M. (1996). A modified simulation scheme for inference in Bayesian networks. International Journal of Approximate Reasoning, 14:55–80.

    Article  MATH  Google Scholar 

  • Box, G. E. P. and Muller, M. E. (1958). A note on the generation of random normal deviates. Ann. Math. Statist., 29:610–611.

    Article  MATH  Google Scholar 

  • Brent, R. P. (1974). A Gaussian pseudo random generator. Comm. Assoc. Comput. Mach., 17:704–706.

    MATH  Google Scholar 

  • Buntine, W. (1991). Theory refinement in Bayesian networks. In Proceedings of the Seventh Conference on Uncertainty in Artificial Intelligence, pages 52–60.

    Google Scholar 

  • Buntine, W. (1996). A guide to the literature on learning probabilistic networks from data. IEEE Transactions on Knowledge and Data Engineering, 8(2):195–210.

    Article  Google Scholar 

  • Castillo, E., Gutiérrez, J. M., and Hadi, A. S. (1997). Expert Systems and Probabilistic Network Models. Springer-Verlag, New York.

    Book  Google Scholar 

  • Charniak, E. (1991). Bayesian networks without tears. AI Magazine, 12:50–63.

    Google Scholar 

  • Chavez, R. M. and Cooper, G. F. (1990). A randomized approximation al-gorithm for probabilistic inference on Bayesian belief networks. Networks,20(5):661–685.

    Article  MathSciNet  MATH  Google Scholar 

  • Chickering, D. M., Geiger, D., and Heckerman, D. (1994). Learning Bayesian networks is NP—hard. Technical report, Microsoft Research, Redmond, WA.

    Google Scholar 

  • Chickering, D. M., Geiger, D., and Heckerman, D. (1995). Learning Bayesian networks: Search methods and experimental results. In Preliminary Papers of the Fifth International Workshop on Artificial Intelligence and Statistics, pages 112–128.

    Google Scholar 

  • Chickering, M. (1996). Learning equivalence classes of Bayesian networks structures. In Proceedings of the Twelfth Conference on Uncertainty in Artificial Intelligence, pages 150–157, Portland, OR. Morgan Kaufmann.

    Google Scholar 

  • Chow, C. and Liu, C. (1968). Approximating discrete probability distributions with dependence trees. IEEE Transactions on Information Theory, 14:462467.

    Google Scholar 

  • Cooper, G. F. (1990). The computational complexity of probabilistic inference using belief networks. Artificial Intelligence, 42:393–405.

    Article  MathSciNet  MATH  Google Scholar 

  • Cooper, G. F. and Herskovits, E. A. (1992). A Bayesian method for the induc-tion of probabilistic networks from data. Machine Learning, 9:309–347.

    MATH  Google Scholar 

  • Cowell, R. G., Dawid, A. P., Lauritzen, S. L., and Spiegelhalter, D. J. (1999).Probabilistic Networks and Expert Systems. Springer-Verlag, New York. Dagum

    Google Scholar 

  • P. and Horvitz, E. (1993). A Bayesian analysis of simulation algorithmsfor inference in belief networks. Networks, 23(5):499–516.

    Article  MathSciNet  Google Scholar 

  • Dawid, A. P. (1979). Conditional independence in statistical theory. Journal of the Royal Statistics Society, Series B, 41:1–31.

    MathSciNet  MATH  Google Scholar 

  • de Campos, L. M. (1998). Automatic learning of graphical models. I: Basic methods. In Gamez, J. A. and Puerta, J. M., editors, Probabilistic Expert System, pages 113–140. Ediciones de la Universidad de Castilla—La Mancha. (In Spanish).

    Google Scholar 

  • DeGroot, M. (1970). Optimal Statistical Decisions. McGraw—Hill, New York.

    Google Scholar 

  • Dempster, A. P. (1972). Covariance selection. Biometrika, 32:95–108.

    Google Scholar 

  • Fung, R. M. and Chang, K. C. (1990). Weighting and integrating evidence for stochastic simulation in Bayesian networks. In Henrion, M., Shachter, R. D., Kanal, L. N., and Lemmer, J. F., editors, Uncertainty in Artificial Intelligence, volume 5, pages 209–220, Amsterdam. Elsevier.

    Google Scholar 

  • Fung, R. M. and Del Favero, B. (1994). Backward simulation in Bayesian net-works. In Proceedings of the Tenth Conference on Uncertainty in Artificial Intelligence, pages 227–234. Morgan Kaufmann Publishers, San Francisco.

    Google Scholar 

  • Geiger, D. and Heckerman, D. (1994). Learning Gaussian networks. Technical report, Microsoft Advanced Technology Division, Microsoft Corporation, Seattle, Washington.

    Google Scholar 

  • Heckerman, D. (1995). A tutorial on learning with Bayesian networks. Technical report, Microsoft Advanced Technology Division, Microsoft Corporation, Seattle, Washington.

    Google Scholar 

  • Heckerman, D., Geiger, D., and Chickering, D. M. (1995). Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning, 20:197–243.

    MATH  Google Scholar 

  • Heckerman, D. and Wellman, M. P. (1995). Bayesian networks. Communications of the ACM, 38:27–30.

    Article  Google Scholar 

  • Henrion, M. (1988). Propagating uncertainty in Bayesian networks by probabilistic logic sampling. In Lemmer, J. F. and Kanal, L. N., editors, Uncertainty in Artificial Intelligence, volume 2, pages 149–163. North-Holland, Amsterdam.

    Google Scholar 

  • Henrion, M., Breese, J. S., and Horvitz, E. J. (1991). Decision analysis and expert systems. AI Magazine, 12:64–91.

    Google Scholar 

  • Herskovits, E. and Cooper, G. (1990). Kutató: An entropy-driven system for construction of probabilistic expert systems from database. Technical report, Knowledge Systems Laboratory, Medical Computer Science, Stanford University, California.

    Google Scholar 

  • Howard, R. and Matheson, J. (1981). Influence diagrams. In Howard, R. and Matheson, J., editors, Readings on the Principles and Applications of Decision Analysis, volume 2, pages 721–764. Strategic Decision Group, Menlo Park, CA.

    Google Scholar 

  • Hryceij, T. (1990). Gibbs sampling in Bayesian networks. Artificial Intelligence, 46(3):351–363.

    Article  MathSciNet  Google Scholar 

  • Jensen, C. S., Kong, A., and Kjærulff, U. (1993). Blocking Gibbs samplingin very large probabilistic expert systems. Technical report, Department ofMathematics and Computer Science, University of Aalborg, Denmark.

    Google Scholar 

  • Jensen, F. V. (1996). An introduction to Bayesian networks. University Collegeof London.

    Google Scholar 

  • Jensen, F. V., Olesen, K. G., and Andersen, S. K. (1990). An algebra of Bayesianbelief universes for knowledge based systems. Networks, 28(5):637–659.

    Article  MathSciNet  Google Scholar 

  • Krause, P. J. (1998). Learning probabilistic networks. Technical report, PhilipsResearch Laboratories.

    Google Scholar 

  • Kreiner, S. (1989). On tests of conditional independence. Technical report, Statistical Research Unit, University of Copenhagen.

    Google Scholar 

  • Kullback, S. (1951). On information and sufficiency. Annals of Math. Stats., 22:79–86.

    Article  MathSciNet  MATH  Google Scholar 

  • Lam, W. and Bacchus, F. (1994). Learning Bayesian belief networks. An approach based on the MDL principle. Computational Intelligence, 10(4):269–293.

    Article  Google Scholar 

  • Larrañaga, P., Kuijpers, C. M. H., Murga, R. H., and Yurramendi, Y. (1996a). Searching for the best ordering in the structure learning of Bayesian net-works. IEEE Transactions on Systems, Man and Cybernetics, 41(4):487–493.

    Google Scholar 

  • Larrañaga, P., Lozano, J. A., and Bengoetxea, E. (2001). Estimation of Dis-tribution Algorithms based on multivariate normal and Gaussian networks. Technical Report KZZA-IK-1–01, epartment of Computer Science and Ar-tificial Intelligence, University of the Basque Country, Spain.

    Google Scholar 

  • Larrañaga, P., Poza, M., Yurramendi, Y., Murga, R. H., and Kuijpers, C. M. H. (1996b). Structure learning of Bayesian networks by genetic algorithms. A performance analysis of control parameters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(9):912–926.

    Article  Google Scholar 

  • Lauritzen, S. L. (1996). Graphical Models. Oxford University Press.

    Google Scholar 

  • Lauritzen, S. L. and Spiegelhalter, D. J. (1988). Local computations with probabilities on graphical structures and their application to expert systems. Journal of the Royal Statistical Society, Series B, 50(2):157–224.

    MathSciNet  MATH  Google Scholar 

  • Madigan, D. and Raftery, A. (1994). Model selection and accounting for model uncertainty in graphical models using Occams window. Journal of the American Statistical Association, 89:1535–1546.

    Article  MATH  Google Scholar 

  • Marsaglia, G., Ananthanarayanan, K., and Paul, N. J. (1976). Improvements on fast methods for generating normal random variables. Information Processing Letters, 5:27–30.

    Article  MathSciNet  MATH  Google Scholar 

  • McCarthy, J. (1956). Measures of the value of information. In Proceedings of the National Academy of Sciences, pages 645–655.

    Google Scholar 

  • Myers, J. W., Laskey, K. B., and Levitt, T. (1999). Learning Bayesian networks from incomplete data with stochastic search algorithms. In Proceedings ofthe Fifteenth Conference on Uncertainty in Artificial Intelligence, pages 476–485.

    Google Scholar 

  • Neapolitan, E. (1990). Probabilistic Reasoning in Expert Systems. John Wiley and Sons, New York.

    Google Scholar 

  • Pearl, J. (1987). Evidential reasoning using stochastic simulation of causal models. Artificial Intelligence, 32:245–257.

    Article  MathSciNet  MATH  Google Scholar 

  • Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, Palo Alto, CA.

    Google Scholar 

  • Pearl, J. (1993). Belief networks revisited. Artificial Intelligence, 59:49–56. Pearl, J. (1996). Fusion, propagation and structuring in belief networks. Artificial Intelligence, 29:241–288.

    MathSciNet  Google Scholar 

  • Ripley, B. D. (1983). Computer generation of random variables: a tutorial. Int. Statist. Rev., 51:301–319.

    Article  MathSciNet  MATH  Google Scholar 

  • Ripley, B. D. (1987). Stochastic Simulation. John Wiley and Sons.

    Google Scholar 

  • Robinson, R. W. (1977). Counting unlabelled acyclic digraphs. In Lecture Notesin Mathematics: Combinatorial Mathematics V, pages 28–43. Springer-Verlag.

    Google Scholar 

  • Salmerón, A., Cano, A., and Moral, S. (2000). Importance sampling in Bayesiannetworks using probability trees. Computational Statistics and Data Analy-sis, 34:387–413.

    Article  MATH  Google Scholar 

  • Sangüesa, R. and Cortés, U. (1998). Learning causal networks from data: a survey and a new algorithm for recovering possibilistic causal networks. AI Communications, 10:31–61.

    Google Scholar 

  • Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 7(2):461–464.

    Article  Google Scholar 

  • Shachter, R. and Kenley, C. (1989). Gaussian influence diagrams. Management Science, 35:527–550.

    Article  Google Scholar 

  • Shachter, R. D. and Peot, M. A. (1990). Simulation approaches to general probabilistic inference on belief networks. In Henrion, M., Shachter, R. D., Kanal, L. N., and Lemmer, J. F., editors, Uncertainty in Artificial Intelligence, volume 5, pages 221–234. Elsevier, Amsterdam.

    Google Scholar 

  • Shafer, G. R. (1996). Probabilistic Expert Systems. Society for Industrial and Applied Mathematics.

    Google Scholar 

  • Shwe, M. and Cooper, G. (1991). An empirical analysis of likelihood-weighting simulation on a large multiply connected medical belief network. Comput. and Biomed. Res., 24:453–475.

    Article  Google Scholar 

  • Smith, P. W. and Whittaker, J. (1998). Edge exclusion tests for graphical Gaussian models. In Learning in Graphical Models, pages 555–574. Kluwer Academic Publishers, Dordrecht, The Netherlands.

    Chapter  Google Scholar 

  • Speed, T. P. and Kiiveri, H. (1986). Gaussian Markov distributions over finite graphs. Annals of Statistics, 14:138–150.

    Article  MathSciNet  MATH  Google Scholar 

  • Spirtes, P., Glymour, C., and Scheines, R. (1991). An algorithm for fast recovery of sparse causal graphs. Social Science Computing Reviews, 9:62–72.

    Article  Google Scholar 

  • Spirtes, P., Glymour, C., and Scheines, R. (1993). Causation, Prediction, and Search. Lecture Notes in Statistics 81, Springer-Verlag.

    Google Scholar 

  • Wermuth, N. (1976). Model search among multiplicative models. Biometrics, 32:253–263.

    Article  MathSciNet  MATH  Google Scholar 

  • Whittaker, J. (1990). Graphical models in applied multivariate statistics. John Wiley and Sons.

    Google Scholar 

  • Wong, M. L., Lam, W., and Leung, K. S. (1999). Using evolutionary computation and minimum description length principle for data mining of probabilistic knowledge. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(2):174–178.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Artificial Intelligence, University of the Basque Country, Spain

    P. Larrañaga

Authors
  1. P. Larrañaga
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of the Basque Country, Spain

    Pedro Larrañaga  & Jose A. Lozano  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer Science+Business Media New York

About this chapter

Cite this chapter

Larrañaga, P. (2002). An Introduction to Probabilistic Graphical Models. In: Larrañaga, P., Lozano, J.A. (eds) Estimation of Distribution Algorithms. Genetic Algorithms and Evolutionary Computation, vol 2. Springer, Boston, MA. https://doi.org/10.1007/978-1-4615-1539-5_2

Download citation

  • DOI: https://doi.org/10.1007/978-1-4615-1539-5_2

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-1-4613-5604-2

  • Online ISBN: 978-1-4615-1539-5

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation