Skip to main content
SpringerLink
Log in

New Advances in Logic-Based Probabilistic Modeling by PRISM

  • Chapter
Probabilistic Inductive Logic Programming

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 4911))

Abstract

We review a logic-based modeling language PRISM and report recent developments including belief propagation by the generalized inside-outside algorithm and generative modeling with constraints. The former implies PRISM subsumes belief propagation at the algorithmic level. We also compare the performance of PRISM with state-of-the-art systems in statistical natural language processing and probabilistic inference in Bayesian networks respectively, and show that PRISM is reasonably competitive.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Sato, T., Kameya, Y.: PRISM: A language for symbolic-statistical modeling. In: Proceedings of the 15th International Joint Conference on Artificial Intelligence (IJCAI 1997), pp. 1330–1335 (1997)

    Google Scholar 

  2. Sato, T.: A statistical learning method for logic programs with distribution semantics. In: Proceedings of the 12th International Conference on Logic Programming (ICLP 1995), pp. 715–729 (1995)

    Google Scholar 

  3. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. Royal Statistical Society B39(1), 1–38 (1977)

    MathSciNet  Google Scholar 

  4. Tamaki, H., Sato, T.: OLD resolution with tabulation. In: Shapiro, E. (ed.) ICLP 1986. LNCS, vol. 225, pp. 84–98. Springer, Heidelberg (1986)

    Google Scholar 

  5. Zhou, N.F., Sato, T.: Efficient fixpoint computation in linear tabling. In: Proceedings of the 5th ACM-SIGPLAN International Conference on Principles and Practice of Declarative Programming (PPDP 2003), pp. 275–283 (2003)

    Google Scholar 

  6. Sato, T., Kameya, Y., Abe, S., Shirai, K.: Fast EM learning of a family of PCFGs. Technical Report (Dept. of CS) TR01-0006, Tokyo Institute of Technology (2001)

    Google Scholar 

  7. Cussens, J.: Parameter estimation in stochastic logic programs. Machine Learning 44(3), 245–271 (2001)

    Article  MATH  Google Scholar 

  8. Sato, T., Kameya, Y., Zhou, N.F.: Generative modeling with failure in PRISM. In: Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI 2005), pp. 847–852 (2005)

    Google Scholar 

  9. Pearl, J.: Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Francisco (1988)

    Google Scholar 

  10. Sato, T.: Inside-Outside probability computation for belief propagation. In: Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI 2007), pp. 2605–2610 (2007)

    Google Scholar 

  11. Lauritzen, S., Spiegelhalter, D.: Local computations with probabilities on graphical structures and their applications to expert systems. Journal of the Royal Statistical Society, B 50, 157–224 (1988)

    MATH  MathSciNet  Google Scholar 

  12. Jensen, F.V.: An Introduction to Bayesian Networks. UCL Press (1996)

    Google Scholar 

  13. Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2), 257–286 (1989)

    Article  Google Scholar 

  14. Baker, J.K.: Trainable grammars for speech recognition. In: Proceedings of Spring Conference of the Acoustical Society of America, pp. 547–550 (1979)

    Google Scholar 

  15. Eisner, J., Goldlust, E., Smith, N.: Compiling Comp Ling: Weighted dynamic programming and the Dyna language. In: Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT-EMNLP)., pp. 281–290 (2005)

    Google Scholar 

  16. Darwiche, A.: A compiler for deterministic, decomposable negation normal form. In: Proceedings of the 18th national conference on Artificial intelligence (AAAI 2002), pp. 627–634 (2002)

    Google Scholar 

  17. Chavira, M., Darwiche, A.: Compiling Bayesian networks with local structure. In: Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI 2005), pp. 1306–1312 (2005)

    Google Scholar 

  18. Chavira, M., Darwiche, A., Jaeger, M.: Compiling relational bayesian networks for exact inference. International Journal of Approximate Reasoning 42, 4–20 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  19. Doets, K.: From Logic to Logic Programming. The MIT Press, Cambridge (1994)

    MATH  Google Scholar 

  20. Manning, C.D., Schütze, H.: Foundations of Statistical Natural Language Processing. The MIT Press, Cambridge (1999)

    MATH  Google Scholar 

  21. Castillo, E., Gutierrez, J.M., Hadi, A.S.: Expert Systems and Probabilistic Network Models. Springer, Heidelberg (1997)

    Google Scholar 

  22. Chow, Y., Teicher, H.: Probability Theory, 3rd edn. Springer, Heidelberg (1997)

    MATH  Google Scholar 

  23. Poole, D.: Probabilistic Horn abduction and Bayesian networks. Artificial Intelligence 64(1), 81–129 (1993)

    Article  MATH  Google Scholar 

  24. Poole, D.: The independent choice logic for modeling multiple agents under uncertainty. Artificial Intelligence 94(1-2), 7–56 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  25. Clark, K.: Negation as failure. In: Gallaire, H., Minker, J. (eds.) Logic and Databases, pp. 293–322. Plenum Press, New York (1978)

    Google Scholar 

  26. Mateescu, R., Dechter, R.: The relationship between AND/OR search spaces and variable elimination. In: Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence (UAI 2005), pp. 380–387 (2005)

    Google Scholar 

  27. Sato, T.: Modeling scientific theories as PRISM programs. In: Proceedings of ECAI 1998 Workshop on Machine Discovery, pp. 37–45 (1998)

    Google Scholar 

  28. Mitomi, H., Fujiwara, F., Yamamoto, M., Sato, T.: Bayesian classification of human custom based on stochastic context-free grammar (in Japanese). IEICE Transaction on Information and Systems J88-D-II(4), 716–726 (2005)

    Google Scholar 

  29. Wang, S., Wang, S., Greiner, R., Schuurmans, D., Cheng, L.: Exploiting syntactic, semantic and lexical regularities in language modeling via directed Markov random fields. In: Proceedings of the 22th International Conference on Machine Learning (ICML 2005), pp. 948–955 (2005)

    Google Scholar 

  30. Sato, T., Kameya, Y.: Parameter learning of logic programs for symbolic-statistical modeling. Journal of Artificial Intelligence Research 15, 391–454 (2001)

    MATH  MathSciNet  Google Scholar 

  31. Smyth, P., Heckerman, D., Jordan, M.: Probabilistic independence networks for hidden Markov probability models. Neural Computation 9(2), 227–269 (1997)

    Article  MATH  Google Scholar 

  32. Kask, K., Dechter, R., Larrosa, J., Cozman, F.: Bucket-tree elimination for automated reasoning. ICS Technical Report Technical Report No.R92, UC Irvine (2001)

    Google Scholar 

  33. Shafer, G., Shenoy, P.: Probability propagation. Annals of Mathematics and Artificial Intelligence 2, 327–352 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  34. McAllester, D., Collins, M., Pereira, F.: Case-factor diagrams for structured probabilistic modeling. In: Proceedings of the 20th Annual Conference on Uncertainty in Artificial Intelligence (UAI2004), Arlington, Virginia, pp. 382–391. AUAI Press (2004)

    Google Scholar 

  35. Minato, S., Satoh, K., Sato, T.: Compiling bayesian networks by symbolic probability calculation based on zero-suppressed bdds. In: Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI 2007), pp. 2550–2555 (2007)

    Google Scholar 

  36. Charniak, E.: Tree-bank grammars. In: Proceedings of the 13th National Conference on Artificial Intelligence(AAAI 1996), pp. 1031–1036 (1996)

    Google Scholar 

  37. Marcus, M., Santorini, B., Marcinkiewicz, M.: Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics 19, 313–330 (1993)

    Google Scholar 

  38. Boutilier, C., Friedman, N., Goldszmidt, M., Koller, D.: Context-specific independence in Bayesian networks. In: Procceding of the 12th Conference on Uncertainty in Artificial Intelligence (UAI 1996), pp. 115–123 (1996)

    Google Scholar 

  39. Chi, Z., Geman, S.: Estimation of probabilistic context-free grammars. Computational Linguistics 24(2), 299–305 (1998)

    MathSciNet  Google Scholar 

  40. Wetherell, C.S.: Probabilistic languages: A review and some open questions. Computing Surveys 12(4), 361–379 (1980)

    Article  MATH  MathSciNet  Google Scholar 

  41. Abney, S.: Stochastic attribute-value grammars. Computational Linguistics 23(4), 597–618 (1997)

    MathSciNet  Google Scholar 

  42. Schmid, H.: A generative probability model for unification-based grammars. In: Proceedings of the 21st International Conference on Computational Linguistics (COLING 2002, pp. 884–896 (2002)

    Google Scholar 

  43. Sag, I., Wasow, T.: Syntactic Theory: A Formal Introduction. CSLI Publications, Stanford (1999)

    MATH  Google Scholar 

  44. Sato, T.: First Order Compiler: A deterministic logic program synthesis algorithm. Journal of Symbolic Computation 8, 605–627 (1989)

    MATH  MathSciNet  Google Scholar 

  45. Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proceedings of the 18th International Conference on Machine Learning (ICML 2001, pp. 282–289 (2001)

    Google Scholar 

  46. Sato, T., Kameya, Y.: Negation elimination for finite PCFGs. In: Etalle, S. (ed.) LOPSTR 2004. LNCS, vol. 3573, pp. 119–134. Springer, Heidelberg (2005)

    Google Scholar 

  47. Kameya, Y., Sato, T.: Efficient EM learning for parameterized logic programs. In: Palamidessi, C., Moniz Pereira, L., Lloyd, J.W., Dahl, V., Furbach, U., Kerber, M., Lau, K.-K., Sagiv, Y., Stuckey, P.J. (eds.) CL 2000. LNCS (LNAI), vol. 1861, pp. 269–294. Springer, Heidelberg (2000)

    Chapter  Google Scholar 

  48. Nilsson, N.J.: Probabilistic logic. Artificial Intelligence 28, 71–87 (1986)

    Article  MATH  MathSciNet  Google Scholar 

  49. Frish, A., Haddawy, P.: Anytime deduction for probabilistic logic. Journal of Artificial Intelligence 69, 93–122 (1994)

    Article  Google Scholar 

  50. Lukasiewicz, T.: Probabilistic deduction with conditional constraints over basic events. Journal of Artificial Intelligence Research 10, 199–241 (1999)

    MATH  MathSciNet  Google Scholar 

  51. Ng, R., Subrahmanian, V.S.: Probabilistic logic programming. Information and Computation 101, 150–201 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  52. Lakshmanan, L.V.S., Sadri, F.: Probabilistic deductive databases. In: Proceedings of the 1994 International Symposium on Logic Programming (ILPS 1994), pp. 254–268 (1994)

    Google Scholar 

  53. Dekhtyar, A., Subrahmanian, V.S.: Hybrid probabilistic programs. In: Proceedings of the 14th International Conference on Logic Programming (ICLP 1997), pp. 391–405 (1997)

    Google Scholar 

  54. Saad, E., Pontelli, E.: Toward a more practical hybrid probabilistic logic programming framework. In: Hermenegildo, M.V., Cabeza, D. (eds.) PADL 2004. LNCS, vol. 3350, pp. 67–82. Springer, Heidelberg (2005)

    Google Scholar 

  55. Taskar, B., Abbeel, P., Koller, D.: Discriminative probabilistic models for relational data. In: Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence (UAI 2002), pp. 485–492 (2002)

    Google Scholar 

  56. Richardson, M., Domingos, P.: Markov logic networks. Machine Learning 62, 107–136 (2006)

    Article  Google Scholar 

  57. Breese, J.S.: Construction of belief and decision networks. Computational Intelligence 8(4), 624–647 (1992)

    Article  Google Scholar 

  58. Wellman, M., Breese, J., Goldman, R.: From knowledge bases to decision models. Knowledge Engineering Review 7(1), 35–53 (1992)

    Article  Google Scholar 

  59. Koller, D., Pfeffer, A.: Learning probabilities for noisy first-order rules. In: Proceedings of the 15th International Joint Conference on Artificial Intelligence (IJCAI 1997), pp. 1316–1321 (1997)

    Google Scholar 

  60. Ngo, L., Haddawy, P.: Answering queries from context-sensitive probabilistic knowledge bases. Theoretical Computer Science 171, 147–177 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  61. Friedman, N., Getoor, L., Koller, D., Pfeffer, A.: Learning probabilistic relational models. In: Proceedings of the 16th International Joint Conference on Artificial Intelligence (IJCAI 1999), pp. 1300–1309 (1999)

    Google Scholar 

  62. Kristian Kersting, K., De Raedt, L.: Bayesian logic programs. In: Cussens, J., Frisch, A.M. (eds.) ILP 2000. LNCS (LNAI), vol. 1866, pp. 138–155. Springer, Heidelberg (2000)

    Google Scholar 

  63. Jaeger, J.: Complex probabilistic modeling with recursive relational Bayesian networks. Annals of Mathematics and Artificial Intelligence 32(1-4), 179–220 (2001)

    Article  MathSciNet  Google Scholar 

  64. Getoor, L., Friedman, N., Koller, D.: Learning probabilistic models of relational structure. In: Proceedings of the 18th International Conference on Machine Learning (ICML 2001), pp. 170–177 (2001)

    Google Scholar 

  65. Kersting, K., De Raedt, L.: Basic principles of learning bayesian logic programs. Technical Report Technical Report No. 174, Institute for Computer Science, University of Freiburg (2002)

    Google Scholar 

  66. Chavira, M., Darwiche, A., Jaeger, M.: Compiling relational bayesian networks for exact inference. In: Proceedings of the Second European Workshop on Probabilistic Graphical Models (PGM 2004), pp. 49–56 (2004)

    Google Scholar 

  67. Fierens, D., Blockeel, H., Bruynooghe, M., Ramon, J.: Logical Bayesian networks and their relation to other probabilistic logical models. In: Kramer, S., Pfahringer, B. (eds.) ILP 2005. LNCS (LNAI), vol. 3625, pp. 121–135. Springer, Heidelberg (2005)

    Google Scholar 

  68. Muggleton, S.: Stochastic logic programs. In: de Raedt, L. (ed.) Advances in Inductive Logic Programming, pp. 254–264. IOS Press, Amsterdam (1996)

    Google Scholar 

  69. Vennekens, J., Verbaeten, S., Bruynooghe, M.: Logic programs with annotated disjunctions. In: Demoen, B., Lifschitz, V. (eds.) ICLP 2004. LNCS, vol. 3132, pp. 431–445. Springer, Heidelberg (2004)

    Google Scholar 

  70. Baral, C., Gelfond, M., Rushton, N.: Probabilistic reasoning with answer sets. In: Lifschitz, V., Niemelä, I. (eds.) LPNMR 2004. LNCS (LNAI), vol. 2923, pp. 21–33. Springer, Heidelberg (2003)

    Google Scholar 

  71. Kersting, K., De Raedt, L., Raiko, T.: Logical hidden Markov models. Journal of Artificial Intelligence Research 25, 425–456 (2006)

    MathSciNet  MATH  Google Scholar 

  72. De Raedt, L., Angelika, K., Toivonen, H.: ProbLog: A probabilistic Prolog and its application in link discoverry. In: Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI 2007) (2007)

    Google Scholar 

  73. Pfeffer, A.: IBAL: A probabilistic rational programming language. In: Proceedings of the 17th International Conference on Artificial Intelligence (IJCAI 2001), pp. 733–740 (2001)

    Google Scholar 

  74. Laskey, K.: MEBN: A logic for open-world probabilistic reasoning. C4I Center Technical Report C4I06-01, George Mason University Department of Systems Engineering and Operations Research (2006)

    Google Scholar 

  75. Milch, B., Marthi, B., Russell, S., Sontag, D., Ong, D., Kolobov, A.: BLOG: Probabilistic models with unknown objects. In: Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI 2005), pp. 1352–1359 (2005)

    Google Scholar 

  76. Milch, B., Marthi, B., Sontag, D., Russell, S., Ong, D., Kolobov, A.: Approximate Inference for Infinite Contingent Bayesian Networks. In: Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics (AISTATS 2005), pp. 1352–1359 (2005)

    Google Scholar 

  77. Pynadath, D.V., Wellman, M.P.: Generalized queries on probabilistic context-free grammars. IEEE Transaction on Pattern Analysis and Machine Intelligence 20(1), 65–77 (1998)

    Article  Google Scholar 

  78. Rauzy, A., Chatelet, E., Dutuit, Y., Berenguer, C.: A practical comparison of methods to assess sum-of-products. Reliability Engineering and System Safety 79, 33–42 (2003)

    Article  Google Scholar 

  79. Stolcke, A.: An efficient probabilistic context-free parsing algorithm that computes prefix probabilities. Computational Linguistics 21(2), 165–201 (1995)

    MathSciNet  Google Scholar 

  80. Schwarz, G.: Estimating the dimension of a model. Annals of Statistics 6(2), 461–464 (1978)

    Article  MATH  MathSciNet  Google Scholar 

  81. Cheeseman, P., Stutz, J.: Bayesian classification (AutoClass): Theory and results. In: Fayyad, U., Piatesky, G., Smyth, P., Uthurusamy, R. (eds.) Advances in Knowledge Discovery and Data Mining, pp. 153–180. The MIT Press, Cambridge (1995)

    Google Scholar 

  82. Izumi, Y., Kameya, Y., Sato, T.: Parallel EM learning for symbolic-statistical models. In: Proceedings of the International Workshop on Data-Mining and Statistical Science (DMSS 2006), pp. 133–140 (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tokyo Institute of Technology, Ookayama Meguro Tokyo, Japan

    Taisuke Sato & Yoshitaka Kameya

Authors
  1. Taisuke Sato
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yoshitaka Kameya
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Luc De Raedt Paolo Frasconi Kristian Kersting Stephen Muggleton

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Sato, T., Kameya, Y. (2008). New Advances in Logic-Based Probabilistic Modeling by PRISM. In: De Raedt, L., Frasconi, P., Kersting, K., Muggleton, S. (eds) Probabilistic Inductive Logic Programming. Lecture Notes in Computer Science(), vol 4911. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-78652-8_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-78652-8_5

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-78651-1

  • Online ISBN: 978-3-540-78652-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation