Machine Learning

, Volume 83, Issue 2, pp 241–264 | Cite as

Constraint-based probabilistic modeling for statistical abduction

  • Taisuke Sato
  • Masakazu Ishihata
  • Katsumi Inoue


We introduce a new framework for logic-based probabilistic modeling called constraint-based probabilistic modeling which defines CBPMs (constraint-based probabilistic models) , i.e. conditional joint distributions P(⋅∣KB) over independent propositional variables constrained by a knowledge base KB consisting of clauses. We first prove that generative models such as PCFGs and discriminative models such as CRFs have equivalent CBPMs as long as they are discrete. We then prove that CBPMs in infinite domains exist which give existentially closed logical consequences of KB probability one. Finally we derive an EM algorithm for the parameter learning of CBPMs and apply it to statistical abduction.


Probabilistic model Constraint Abduction 


  1. Akers, S. (1978). Binary decision diagrams. IEEE Transactions on Computers, 27(6), 509–516. CrossRefMATHGoogle Scholar
  2. Boutilier, C., Friedman, N., Goldszmidt, M., & Koller, D. (1996). Context-specific independence in Bayesian networks. In Proceedings of the 12th conference on uncertainty in artificial intelligence (UAI’96) (pp. 115–123). Google Scholar
  3. Bryant, R. (1986). Graph-based algorithms for boolean function manipulation. IEEE Transactions on Computers, 35(8), 677–691. CrossRefMATHGoogle Scholar
  4. Chavira, M., & Darwiche, A. (2005). Compiling Bayesian networks with local structure. In Proceedings of the 19th international joint conference on artificial intelligence (IJCAI’05) (pp. 1306–1312). Google Scholar
  5. Chen, J., Muggleton, S., & Santos, J. (2008). Learning probabilistic logic models from probabilistic examples. Machine Learning, 73, 55–85. CrossRefGoogle Scholar
  6. Chow, Y., & Teicher, H. (1997). Probability theory (3rd ed.). Berlin: Springer. MATHGoogle Scholar
  7. Cussens, J. (2001). Parameter estimation in stochastic logic programs. Machine Learning, 44(3), 245–271. CrossRefMATHGoogle Scholar
  8. De Raedt, L., & Kersting, K. (2008). Probabilistic inductive logic programming. In L. De Raedt, P. Frasconi, K. Kersting, & S. Muggleton (Eds.), Probabilistic inductive logic programming—theory and applications. Lecture notes in computer science (pp. 1–27). Berlin: Springer. CrossRefGoogle Scholar
  9. De Raedt, L., Kimmig, A., & Toivonen, H. (2007). ProbLog: a probabilistic Prolog and its application in link discovery. In Proceedings of the 20th international joint conference on artificial intelligence (IJCAI’07) (pp. 2468–2473). Google Scholar
  10. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 39(1), 1–38. MATHMathSciNetGoogle Scholar
  11. Feller, W. (1971). An introduction to probability theory and its applications. New York: Wiley. MATHGoogle Scholar
  12. Getoor, L., & Taskar, B. (Eds.) (2007). Introduction to statistical relational learning. Cambridge: MIT Press. MATHGoogle Scholar
  13. Inoue, K., Sato, T., Ishihata, M., Kameya, Y., & Nabeshima, H. (2009). Evaluating abductive hypotheses using an EM algorithm on BDDs. In Proceedings of the 21st international joint conference on artificial intelligence (IJCAI’09) (pp. 810–815). Google Scholar
  14. Ishihata, M., Kameya, Y., Sato, T., & Minato, S. (2008). Propositionalizing the EM algorithm by BDDs (Technical Report TR08-0004). Dept. of CS, Tokyo Institute of Technology. Google Scholar
  15. Kersting, K., & De Raedt, L. (2001). Adaptive Bayesian logic programs. In Proceedings of the 11th international conference on inductive logic programming (ILP’01) (pp. 104–117). Google Scholar
  16. Lafferty, J., McCallum, A., & Pereira, F. (2001). Conditional random fields: probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th international conference on machine learning (ICML’01) (pp. 282–289). Google Scholar
  17. Laskey, K. (2006). MEBN: a logic for open-world probabilistic reasoning (C4I Center Technical Report C4I06-01). George Mason University Department of Systems Engineering and Operations Research. Google Scholar
  18. Lloyd, J. W. (1984). Foundations of logic programming. Berlin: Springer. MATHGoogle Scholar
  19. McAllester, D., Collins, M., & Pereira, F. (2004). Case-factor diagrams for structured probabilistic modeling. In Proceedings of the 20th annual conference on uncertainty in artificial intelligence (UAI’04) (pp. 382–391). Arlington: AUAI. Google Scholar
  20. Milch, B., Marthi, B., Sontag, D., Russell, S., Ong, D., & Kolobov, A. (2005). Approximate inference for infinite contingent Bayesian networks. In Proceedings of the 10th international workshop on artificial intelligence and statistics (AISTATS’05) (pp. 1352–1359). Google Scholar
  21. Minato, S. (2001). Zero-suppressed BDDs and their applications. International Journal on Software Tools for Technology Transfer, 3(2), 156–170. MATHGoogle Scholar
  22. Muggleton, S. (1996). Stochastic logic programs. In L. De Raedt (Ed.), Advances in inductive logic programming (pp. 254–264). Amsterdam: IOS. Google Scholar
  23. Pfeffer, A., & Koller, D. (2000). Semantics and inference for recursive probability models. In Proceedings of the 7th national conference on artificial intelligence (AAAI’00) (pp. 538–544). Google Scholar
  24. Poole, D. (1993). Probabilistic Horn abduction and Bayesian networks. Artificial Intelligence, 64(1), 81–129. CrossRefGoogle Scholar
  25. Poole, D. (1997). The independent choice logic for modeling multiple agents under uncertainty. Artificial Intelligence, 94(1–2), 7–56. CrossRefMATHMathSciNetGoogle Scholar
  26. Poole, D. (2000). Abducing through negation as failure: stable models within independent choice logic. Journal of Logic Programming, 44, 5–35. CrossRefMATHMathSciNetGoogle Scholar
  27. Richardson, M., & Domingos, P. (2006). Markov logic networks. Machine Learning, 62, 107–136. CrossRefGoogle Scholar
  28. Riezler, S. (1998). Probabilistic constraint logic programming. PhD thesis, Universität Tübingen. Google Scholar
  29. Sato, T., & Kameya, Y. (2001). Parameter learning of logic programs for symbolic-statistical modeling. Journal of Artificial Intelligence Research, 15, 391–454. MATHMathSciNetGoogle Scholar
  30. Sato, T., & Kameya, Y. (2002). Statistical abduction with tabulation. In A. Kakas & F. Sadri (Eds.), LNAI : Vol. 2408. Computational logic: logic programming and beyond (pp. 567–587). Berlin: Springer. CrossRefGoogle Scholar
  31. Sato, T., & Kameya, Y. (2008). New advances in logic-based probabilistic modeling by PRISM. In L. De Raedt, P. Frasconi, K. Kersting, & S. Muggleton (Eds.), LNAI : Vol. 4911. Probabilistic inductive logic programming (pp. 118–155). Berlin: Springer. CrossRefGoogle Scholar
  32. Singla, P., & Domingos, P. (2007). Markov logic in infinite domains. In Proceedings of the twenty-third conference on uncertainty in artificial intelligence (UAI’07) (pp. 368–375). Google Scholar

Copyright information

© The Author(s) 2010

Authors and Affiliations

  • Taisuke Sato
    • 1
  • Masakazu Ishihata
    • 1
  • Katsumi Inoue
    • 2
  1. 1.Tokyo Institute of TechnologyMeguro-ku, TokyoJapan
  2. 2.National Institute of InformaticsChiyoda-ku, TokyoJapan

Personalised recommendations