Skip to main content
SpringerLink
Log in

A Survey of First-Order Probabilistic Models

  • Chapter
Innovations in Bayesian Networks

Part of the book series: Studies in Computational Intelligence ((SCI,volume 156))

Abstract

There has been a long standing division in Artificial Intelligence between logical and probabilistic reasoning approaches. While probabilistic models can deal well with inherent uncertainty in many real-world domains, they operate on a mostly propositional level. Logic systems, on the other hand, can deal with much richer representations, especially first-order ones, but treat uncertainty only in limited ways. Therefore, an integration of these types of inference is highly desirable, and many approaches have been proposed, especially from the 1990s on. These solutions come from many different subfields and vary greatly in language, features and (when available at all) inference algorithms. Therefore their relation to each other is not always clear, as well as their semantics. In this survey, we present the main aspects of the solutions proposed and group them according to language, semantics and inference algorithm. In doing so, we draw relations between them and discuss particularly important choices and tradeoffs.

Currently at the Computer Science Division of University of California, Berkeley.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach, 2nd edn. Prentice-Hall, Englewood Cliffs (2003)

    Google Scholar 

  2. Buchanan, B., Shortliffe, E.: Rule-Based Expert Systems: The MYCIN experiments of the Stanford Heuristic Programming Project. Addison-Wesley, Reading (1984)

    Google Scholar 

  3. Pearl, J.: Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann, San Mateo (Calif.) (1988)

    Google Scholar 

  4. Nilsson, N.J.: Probabilistic logic. Artificial intelligence 28(1), 71–88 (1986)

    Article  MATH  MathSciNet  Google Scholar 

  5. Bacchus, F.: Representing and reasoning with probabilistic knowledge: a logical approach to probabilities. MIT Press, Cambridge (1990)

    Google Scholar 

  6. Halpern, J.Y.: An analysis of first-order logics of probability. In: Proceedings of IJCAI 1989, 11th International Joint Conference on Artificial Intelligence, Detroit, US, pp. 1375–1381 (1990)

    Google Scholar 

  7. Shortliffe, E.H.: Mycin: a rule-based computer program for advising physicians regarding antimicrobial therapy selection. PhD thesis, Stanford University (1975)

    Google Scholar 

  8. Clark, K.L., McCabe, F.G.: Prolog: A language for implementing expert systems. In: Hayes, J.E., Michie, D., Pao, Y.H. (eds.) Machine Intelligence, Ellis Horwood, Chichester, vol. 10, pp. 455–470 (1982)

    Google Scholar 

  9. Shapiro, E.: Logic programs with uncertainties: A tool for implementing expert systems. In: Proc. IJCAI 1983, pp. 529–532. William Kaufmann, San Francisco (1983)

    Google Scholar 

  10. Heckerman, D.: Probabilistic interpretation for MYCIN’s certainty factors. In: Kanal, L.N., Lemmer, J. (eds.) Uncertainty in Artificial Intelligence, pp. 167–196. Kluwer Science Publishers, Dordrecht (1986)

    Google Scholar 

  11. Lucas, P.: Certainty-factor-like structures in bayesian belief networks. Knowledge-Based Systems 14, 325–327 (2001)

    Article  Google Scholar 

  12. Carnap, R.: The Logical Foundations of Probability. University of Chicago Press, Chicago (1950)

    Google Scholar 

  13. Hailperin, T.: Probabilistic logic. Notre Dame Journal of Formal Logic 25(3), 198–212 (1984)

    Article  MATH  MathSciNet  Google Scholar 

  14. Fagin, R., Halpern, J.Y., Megiddo, N.: A logic for reasoning about probabilities. Information and Computation 87(1/2), 78–128 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  15. Fenstad, J.E.: The structure of probabilities defined on first-order languages. Studies in Inductive Logic and Probabilities, pp. 251–262. California Press (1980)

    Google Scholar 

  16. Gaifman, H.: Concerning measures in first-order calculi. Israel Journal of Mathematics 2, 1–18 (1964)

    Article  MATH  MathSciNet  Google Scholar 

  17. Gaifman, H., Snir, M.: Probabilities over rich languages, testing and randomness. Journal of Symbolic Logic 47(3), 495–548 (1982)

    Article  MATH  MathSciNet  Google Scholar 

  18. Kifer, M., Li, A.: On the semantics of rule-based expert systems with uncertainty. In: Gyssens, M., Van Gucht, D., Paredaens, J. (eds.) ICDT 1988. LNCS, vol. 326, pp. 102–117. Springer, Heidelberg (1988)

    Google Scholar 

  19. Wüthrich, B.: Probabilistic knowledge bases. IEEE Trans. Knowl. Data Eng. 7(5), 691–698 (1995)

    Article  Google Scholar 

  20. Ng, R.T., Subrahmanian, V.S.: Probabilistic logic programming. Information and Computation 101(2), 150–201 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  21. Lakshmanan, L.V.S., Sadri, F.: Probabilistic deductive databases. In: Symposium on Logic Programming, pp. 254–268 (1994)

    Google Scholar 

  22. Lakshmanan, L.V.S.: An epistemic foundation for logic programming with uncertainty. In: Foundations of Software Technology and Theoretical Computer Science, pp. 89–100 (1994)

    Google Scholar 

  23. Ngo, L., Haddawy, P.: Probabilistic logic programming and Bayesian networks. In: Asian Computing Science Conference, pp. 286–300 (1995)

    Google Scholar 

  24. Poole, D.: Probabilistic Horn abduction and Bayesian networks. Artificial Intelligence 64(1), 81–129 (1993)

    Article  MATH  Google Scholar 

  25. Zhang, N.L., Poole, D.: A simple approach to Bayesian network computations. In: Proceedings of the Tenth Biennial Canadian Artificial Intelligence Conference (1994)

    Google Scholar 

  26. Lukasiewicz, T.: Probabilistic deduction with conditional constraints over basic events. J. Artif. Intell. Res (JAIR) 10, 199–241 (1999)

    MATH  MathSciNet  Google Scholar 

  27. Frisch, A.M., Haddawy, P.: Anytime deduction for probabilistic logic. Artificial Intelligence 69(1–2), 93–122 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  28. Koller, D., Halpern, J.Y.: Irrelevance and conditioning in first-order probabilistic logic. In: Shrobe, H., Senator, T. (eds.) Proceedings of the Thirteenth National Conference on Artificial Intelligence and the Eighth Innovative Applications of Artificial Intelligence Conference, vol. 2, pp. 569–576. AAAI Press, Menlo Park (1996)

    Google Scholar 

  29. Riezler, S.: Probabilistic constraint logic programming (1997)

    Google Scholar 

  30. Jaffar, J., Lassez, J.L.: Constraint logic programming. In: POPL 1987: Proceedings of the 14th ACM SIGACT-SIGPLAN symposium on Principles of programming languages, pp. 111–119. ACM Press, New York (1987)

    Chapter  Google Scholar 

  31. Lari, K., Young, S.: The estimation of stochastic context-free grammars using the inside-outside algorithm. Computer Speech and Language 4, 35–56 (1990)

    Article  Google Scholar 

  32. Baker, J.: Trainable grammars for speech recognition. In: Speech communication papers presented at the 97th Meeting of the Acoustical Society, pp. 547–550 (1979)

    Google Scholar 

  33. Baum, L.E.: An inequality and associated maximization technique in statistical estimation for probabilistic functions of markov processes. Inequalities 3, 1–8 (1972)

    Google Scholar 

  34. Muggleton, S.: Stochastic logic programs. In: De Raedt, L. (ed.) Proceedings of the 5th International Workshop on Inductive Logic Programming, Department of Computer Science, Katholieke Universiteit Leuven, vol. 29 (1995)

    Google Scholar 

  35. Cussens, J.: Loglinear models for first-order probabilistic reasoning. In: Laskey, K.B., Prade, H. (eds.) Proceedings of the 15th Annual Conference on Uncertainty in AI (UAI 1999), pp. 126–133. Morgan Kaufmann, San Francisco (1999)

    Google Scholar 

  36. Muggleton, S.: Inverse entailment and Progol. New Generation Computing, Special issue on Inductive Logic Programming 13(3–4), 245–286 (1995)

    Google Scholar 

  37. Della Pietra, S., Della Pietra, V.J., Lafferty, J.D.: Inducing features of random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(4), 380–393 (1997)

    Article  Google Scholar 

  38. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society. Series B (Methodological) 39(1), 1–38 (1977)

    MATH  MathSciNet  Google Scholar 

  39. Lukasiewicz, T.: Probabilistic logic programming. In: European Conference on Artificial Intelligence, pp. 388–392 (1998)

    Google Scholar 

  40. Baral, C., Gelfond, M., Rushton, J.N.: Probabilistic reasoning with answer sets. In: LPNMR, pp. 21–33 (2004)

    Google Scholar 

  41. Pearl, J.: Causality: Models, Reasoning and Inference. Cambridge University Press, Cambridge (2000)

    MATH  Google Scholar 

  42. Sato, T., Kameya, Y.: Prism: A language for symbolic-statistical modeling. In: IJCAI, pp. 1330–1339 (1997)

    Google Scholar 

  43. Sato, T., Kameya, Y.: A viterbi-like algorithm and em learning for statistical abduction (2000)

    Google Scholar 

  44. Milch, B., Marthi, B., Russell, S., Sontag, D., Ong, D.L., Kolobov, A.: BLOG: Probabilistic models with unknown objects. In: Proc. IJCAI (2005)

    Google Scholar 

  45. Spiegelhalter, D., Thomas, A., Best, N., Gilks, W.: Bugs: Bayesian inference using gibbs sampling, version 0.30. Technical report, MRC Biostatistics Unit, University of Cambridge (1994)

    Google Scholar 

  46. Heckerman, D., Chickering, D.M., Meek, C., Rounthwaite, R., Kadie, C.M.: Dependency networks for inference, collaborative filtering, and data visualization. Journal of Machine Learning Research 1, 49–75 (2000)

    Article  Google Scholar 

  47. de Salvo Braz, R.: Lifted First-Order Probabilistic Inference. PhD thesis University of Illinois at Urbana-Champaign (2007)

    Google Scholar 

  48. de Salvo Braz, R., Amir, E., Roth, D.: Lifted first-order probabilistic inference. In: Getoor, L., Taskar, B. (eds.) An Introduction to Statistical Relational Learning, pp. 433–451. MIT Press, Cambridge (2007)

    Google Scholar 

  49. Breese, J.S.: Construction of belief and decision networks. Computational Intelligence 8, 624–647 (1991)

    Article  Google Scholar 

  50. Wellman, M.P., Breese, J.S., Goldman, R.P.: From knowledge bases to decision models. Knowledge Engineering Review 7, 35–53 (1992)

    Article  Google Scholar 

  51. Horsch, M., Poole, D.: A dynamic approach to probabilistic inference using bayesian networks. In: Proceedings of the 6th Conference of Uncertainty in Artificial Intelligence, pp. 155–161. Morgan Kaufmann, San Francisco (1990)

    Google Scholar 

  52. Goldman, R.P., Cherniak, E.: A language for construction of belief networks. IEEE Trans. Pattern Anal. Mach. Intell. 15(3), 196–208 (1993)

    Article  Google Scholar 

  53. Poole, D.: The independent choice logic for modelling multiple agents under uncertainty. Artificial Intelligence 94(1–2), 7–56 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  54. Glesner, S., Koller, D.: Constructing flexible dynamic belief networks from first-order probalistic knowledge bases. In: Symbolic and Quantitative Approaches to Reasoning and Uncertainty, pp. 217–226 (1995)

    Google Scholar 

  55. Haddawy, P.: Generating bayesian networks from probability logic knowledge bases. In: de Mantaras, R.L., Poole, D. (eds.) Uncertainty In Artificial Intelligence 10 (UAI 1994), pp. 262–269. Morgan Kaufmann, San Francisco (1994)

    Google Scholar 

  56. Koller, D., Pfeffer, A.: Learning probabilities for noisy first-order rules. In: IJCAI, pp. 1316–1323 (1997)

    Google Scholar 

  57. Jaeger, M.: Relational Bayesian networks. In: Kaufmann, M. (ed.) Proceedings of the 13th Conference on Uncertainty in Artificial Intelligence, pp. 266–273 (1997)

    Google Scholar 

  58. Getoor, L., Friedman, N., Koller, D., Pfeffer, A.: Learning probabilistic relational models. In: Džeroski, S., Lavrac, N. (eds.) Relational Data Mining, pp. 307–335. Springer, Heidelberg (2001)

    Google Scholar 

  59. Koller, D., Pfeffer, A.: Probabilistic frame-based systems. In: Proceedings of the 15th National Conference on Artificial Intelligence (AAAI), pp. 580–587 (1998)

    Google Scholar 

  60. Minsky, M.: A framework for representing knowledge. In: Computation & intelligence: collected readings, pp. 163–189. American Association for Artificial Intelligence, Menlo Park (1995)

    Google Scholar 

  61. Levesque, H.J., Brachman, R.J.: Expressiveness and tractability in knowledge representation and reasoning. Computational Intelligence 3, 78–93 (1987)

    Article  Google Scholar 

  62. Koller, D., Levy, A.Y., Pfeffer, A.: P-CLASSIC: A tractable probablistic description logic. In: AAAI/IAAI, pp. 390–397 (1997)

    Google Scholar 

  63. Getoor, L.: Learning probabilistic relational models with structural uncertainty. In: Choueiry, B.Y., Walsh, T. (eds.) SARA 2000. LNCS (LNAI), vol. 1864, pp. 322–329. Springer, Heidelberg (2000)

    Google Scholar 

  64. Taskar, B., Abbeel, P., Koller, D.: Discriminative probabilistic models for relational data. In: Proc. Eighteenth Conference on Uncertainty in Artificial Intelligence (UAI), Edmonton, Canada (2002)

    Google Scholar 

  65. Neville, J.: Statistical models and analysis techniques for learning in relational data. PhD thesis, University of Massachusetts Amherst (2006)

    Google Scholar 

  66. Kersting, K., De Raedt, L.: Bayesian logic programs. In: Cussens, J., Frisch, A. (eds.) Proceedings of the Work-in-Progress Track at the 10th International Conference on Inductive Logic Programming, pp. 138–155 (2000)

    Google Scholar 

  67. Fierens, D., Blockeel, H., Bruynooghe, M., Ramon, J.: Logical bayesian networks and their relation to other probabilistic logical models. In: ILP, pp. 121–135 (2005)

    Google Scholar 

  68. Santos Costa, V., Page, D., Qazi, M., Cussens, J.: Clp(bn): Constraint logic programming for probabilistic knowledge. In: Proceedings of the 19th Annual Conference on Uncertainty in Artificial Intelligence (UAI 2003), pp. 517–552. Morgan Kaufmann, San Francisco (2003)

    Google Scholar 

  69. Angelopoulos, N.: Probabilistic finite domains: A brief overview. In: Stuckey, P.J. (ed.) ICLP 2002. LNCS, vol. 2401, p. 475. Springer, Heidelberg (2002)

    Google Scholar 

  70. Richardson, M., Domingos, P.: Markov logic networks. Technical report, Department of Computer Science, University of Washington (2004)

    Google Scholar 

  71. Besag, J.: Statistical analysis of non-lattice data. The Statistician 24(3), 179–195 (1975)

    Article  MathSciNet  Google Scholar 

  72. Laskey, K.B.: First-order Bayesian logic. Technical report, George Mason University Department of Systems Engineering and Operations Research (2003)

    Google Scholar 

  73. Poole, D.: First-order probabilistic inference. In: Proceedings of the 18th International Joint Conference on Artificial Intelligence, pp. 985–991 (2003)

    Google Scholar 

  74. Mitchell, T.M.: Machine Learing. McGraw-Hill Higher Education, New York (1997)

    Google Scholar 

  75. Cumby, C., Roth, D.: Relational representations that facilitate learning. In: Cohn, A.G., Giunchiglia, F., Selman, B. (eds.) KR 2000: Principles of Knowledge Representation and Reasoning, pp. 425–434. Morgan Kaufmann, San Francisco (2000)

    Google Scholar 

  76. Lavrac, N., Dzeroski, S.: Inductive Logic Programming: Techniques and Applications. Routledge, New York, 10001 (1992)

    Google Scholar 

  77. Flach, P., Lachiche, N.: IBC: A first-order Bayesian classifier. In: Džeroski, S., Flach, P. (eds.) ILP 1999. LNCS (LNAI), vol. 1634, pp. 92–103. Springer, Heidelberg (1999)

    Google Scholar 

  78. Lachiche, N., Flach, P.A.: 1BC2: a true first-order Bayesian classifier. In: Matwin, S., Sammut, C. (eds.) ILP 2002. LNCS (LNAI), vol. 2583, pp. 133–148. Springer, Heidelberg (2003)

    Google Scholar 

  79. Neville, J., Jensen, D., Gallagher, B.: Simple estimators for relational bayesian classifiers. In: ICDM 2003: Proceedings of the Third IEEE International Conference on Data Mining. IEEE Computer Society Press, Washington (2003)

    Google Scholar 

  80. Quinlan, J.: Learning logical definitions from relations. Machine Learning 5, 239–266 (1990)

    Google Scholar 

  81. Raedt, L.D., Dehaspe, L.: Clausal discovery. Machine Learning 26(2–3), 99–146 (1997)

    Article  MATH  Google Scholar 

  82. Raedt, L.D., Kersting, K.: Probabilistic inductive logic programming. In: ALT, pp. 19–36 (2004)

    Google Scholar 

  83. Kersting, K., De Raedt, L.: Towards combining inductive logic programming with Bayesian networks. In: Rouveirol, C., Sebag, M. (eds.) ILP 2001. LNCS (LNAI), vol. 2157, pp. 104–117. Springer, Heidelberg (2001)

    Google Scholar 

  84. Rabiner, L.R.: A tutorial on hidden markov models and selected applications in speech recognition. In: Readings in speech recognition, pp. 267–296. Morgan Kaufmann Publishers Inc., San Francisco (1990)

    Google Scholar 

  85. Murphy, K.P.: Dynamic bayesian networks: representation, inference and learning. PhD thesis, University of California, Berkeley, Chair-Stuart Russell (2002)

    Google Scholar 

  86. Diligenti, M., Frasconi, P., Gori, M.: Hidden tree markov models for document image classification. IEEE Trans. Pattern Anal. Mach. Intell. 25(4), 519–523 (2003)

    Article  Google Scholar 

  87. Anderson, C.R., Domingos, P., Weld, D.S.: Relational Markov models and their application to adaptive web navigation. In: KDD 2002: Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 143–152. ACM Press, New York (2002)

    Chapter  Google Scholar 

  88. Buntine, W.L.: Operations for learning with graphical models. Journal of Artificial Intelligence Research 2, 159–225 (1994)

    Google Scholar 

  89. Miolsness, E.: Labeled graph notations for graphical models: Extended report. Technical report, University of California Irvine—Information and Computer Sciences (2004)

    Google Scholar 

  90. de Salvo Braz, R., Amir, E., Roth, D.: Lifted first-order probabilistic inference. In: Proceedings of IJCAI 2005, 19th International Joint Conference on Artificial Intelligence (2005)

    Google Scholar 

  91. Pfeffer, A., Koller, D.: Semantics and inference for recursive probability models. In: AAAI/IAAI, pp. 538–544 (2000)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana, IL, 61801

    Rodrigo de Salvo Braz, Eyal Amir & Dan Roth

Authors
  1. Rodrigo de Salvo Braz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eyal Amir
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dan Roth
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Statistics and Applied Probability, University of California, Santa Barbara, CA, 93106, USA

    Dawn E. Holmes

  2. University of South Australia Adelaide, Mawson Lakes, SA, 5095, Australia

    Lakhmi C. Jain (Professor of Knowledge-Based Engineering) (Professor of Knowledge-Based Engineering)

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Braz, R.d.S., Amir, E., Roth, D. (2008). A Survey of First-Order Probabilistic Models. In: Holmes, D.E., Jain, L.C. (eds) Innovations in Bayesian Networks. Studies in Computational Intelligence, vol 156. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-85066-3_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-85066-3_12

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-85065-6

  • Online ISBN: 978-3-540-85066-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation