Learning Compact Markov Logic Networks with Decision Trees

  • Hassan Khosravi
  • Oliver Schulte
  • Jianfeng Hu
  • Tianxiang Gao
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7207)


Markov Logic Networks (MLNs) are a prominent model class that generalizes both first-order logic and undirected graphical models (Markov networks). The qualitative component of an MLN is a set of clauses and the quantitative component is a set of clause weights. Generative MLNs model the joint distribution of relationships and attributes. A state-of-the-art structure learning method is the moralization approach: learn a 1st-order Bayes net, then convert it to conjunctive MLN clauses. The moralization approach takes advantage of the high-quality inference algorithms for MLNs and their ability to handle cyclic dependencies. A weakness of the moralization approach is that it leads to an unnecessarily large number of clauses. In this paper we show that using decision trees to represent conditional probabilities in the Bayes net is an effective remedy that leads to much more compact MLN structures. The accuracy of predictions is competitive with the unpruned model and in many cases superior.


Decision Tree Inductive Logic Programming Markov Network Conditional Probability Table Markov Logic Network 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Domingos, P., Richardson, M.: Markov logic: A unifying framework for statistical relational learning. In: [16]Google Scholar
  2. 2.
    Kok, S., Summer, M., Richardson, M., Singla, P., Poon, H., Lowd, D., Wang, J., Domingos, P.: The Alchemy system for statistical relational AI. Technical report, University of Washington (2009)Google Scholar
  3. 3.
    Khosravi, H., Schulte, O., Man, T., Xu, X., Bina, B.: Structure learning for Markov logic networks with many descriptive attributes. In: Proceedings of the Twenty-Fourth Conference on Artificial Intelligence (AAAI), pp. 487–493 (2010)Google Scholar
  4. 4.
    Kersting, K., de Raedt, L.: Bayesian logic programming: Theory and tool. In: [16], ch. 10, pp. 291–318Google Scholar
  5. 5.
    Boutilier, C., Friedman, N., Goldszmidt, M., Koller, D.: Context-specific independence in bayesian networks. In: Proceedings of the Twelfth Conference on Uncertainty in Artificial Intelligence, pp. 115–123. Citeseer (1996)Google Scholar
  6. 6.
    Fierens, D., Ramon, J., Blockeel, H., Bruynooghe, M.: A comparison of pruning criteria for probability trees. Machine Learning 78, 251–285 (2010)CrossRefGoogle Scholar
  7. 7.
    Kok, S., Domingos, P.: Learning markov logic networks using structural motifs. In: Fürnkranz, J., Joachims, T. (eds.) ICML, pp. 551–558. Omni Press (2010)Google Scholar
  8. 8.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann (1988)Google Scholar
  9. 9.
    Provost, F.J., Domingos, P.: Tree induction for probability-based ranking. Machine Learning 52, 199–215 (2003)zbMATHCrossRefGoogle Scholar
  10. 10.
    Zhang, H., Su, J.: Conditional Independence Trees. In: Boulicaut, J.-F., Esposito, F., Giannotti, F., Pedreschi, D. (eds.) ECML 2004. LNCS (LNAI), vol. 3201, pp. 513–524. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  11. 11.
    Kohavi, R.: Scaling up the accuracy of naive-bayes classifiers: A decision-tree hybrid. In: Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, vol. 7. AAAI Press, Menlo Park (1996)Google Scholar
  12. 12.
    Dzeroski, S.: Inductive logic programming in a nutshell. In: [16]Google Scholar
  13. 13.
    Blockeel, H., Raedt, L.D.: Top-down induction of first-order logical decision trees. Artificial Intelligence 101, 285–297 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Neville, J., Jensen, D.: Relational dependency networks. Journal of Machine Learning Research 8, 653–692 (2007)zbMATHGoogle Scholar
  15. 15.
    Schulte, O.: A tractable pseudo-likelihood function for bayes nets applied to relational data. In: Proceedings of SIAM Conference on Data Mining (SIAM SDM), pp. 462–473 (2011)Google Scholar
  16. 16.
    Getoor, L., Tasker, B.: Introduction to statistical relational learning. MIT Press (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Hassan Khosravi
    • 1
  • Oliver Schulte
    • 1
  • Jianfeng Hu
    • 1
  • Tianxiang Gao
    • 1
  1. 1.School of Computing ScienceSimon Fraser UniversityVancouver-BurnabyCanada

Personalised recommendations