Skip to main content
SpringerLink
Log in

Towards Multistrategic Statistical Relational Learning

  • Chapter
Advances in Machine Learning II

Part of the book series: Studies in Computational Intelligence ((SCI,volume 263))

  • 2086 Accesses

Abstract

Statistical Relational Learning (SRL) is a growing field in Machine Learning that aims at the integration of logic-based learning approaches with probabilistic graphical models. Markov Logic Networks (MLNs) are one of the state-of-the-art SRL models that combine first-order logic and Markov networks (MNs) by attaching weights to first-order formulas and viewing these as templates for features of MNs. Learning models in SRL consists in learning the structure (logical clauses in MLNs) and the parameters (weights for each clause in MLNs). Structure learning of MLNs is performed by maximizing a likelihood function (or a function thereof) over relational databases and MLNs have been successfully applied to problems in relational and uncertain domains. However, most complex domains are characterized by incomplete data. Until now SRL models have mostly used Expectation-Maximization (EM) for learning statistical parameters under missing values. Multistrategic learning in the relational setting has been a successful approach to dealing with complex problems where multiple inference mechanisms can help solve different subproblems. Abduction is an inference strategy that has been proven useful for completing missing values in observations. In this paper we propose two frameworks for integrating abduction in SRL models. The first tightly integrates logical abduction with structure and parameter learning of MLNs in a single step. During structure search guided by conditional likelihood, clause evaluation is performed by first trying to logically abduce missing values in the data and then by learning optimal pseudo-likelihood parameters using the completed data. The second approach integrates abduction with Structural EM of [17] by performing logical abductive inference in the E-step and then by trying to maximize parameters in the M-step.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Arvanitis, A., Muggleton, S.H., Chen, J., Watanabe, H.: Abduction with stochastic logic programs based on a possible worlds semantics. Short Paper Proceedings of the 16th International Conference on Inductive Logic Programming, University of Corunna (2006)

    Google Scholar 

  2. Bacchus, F.: Representing and Reasoning with Probabilistic Knowledge. MIT Press, Cambridge (1990)

    Google Scholar 

  3. Besag, J.: Statistical analysis of non-lattice data. Statistician 24, 179–195 (1975)

    Article  Google Scholar 

  4. Biba, M., Ferilli, S., Esposito, F.: Discriminative structure learning of markov logic networks. In: Železný, F., Lavrač, N. (eds.) ILP 2008. LNCS (LNAI), vol. 5194, pp. 59–76. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  5. Biba, M., Ferilli, S., Esposito, F.: Structure learning of markov logic networks through iterated local search. In: Proceedings of 18th European Conference on Artificial Intelligence (ECAI). Frontiers in Artificial Intelligence and Applications, vol. 178, pp. 361–365 (2008)

    Google Scholar 

  6. Chen, J., Muggleton, S., Santos, J.: Abductive stochastic logic programs for metabolic network inhibition learning. In: Proceedings of Workshop Mining and Learning with Graphs, MLG 2007 (2007)

    Google Scholar 

  7. Clark, K.: Negation as failure. In: Gallaire, H., Minker, J. (eds.) Logic and databases, pp. 293–322. Plenum Press, New York (1978)

    Google Scholar 

  8. Cumby, C., Roth, D.: Feature extraction languages for propositionalized relational learning. In: Proceedings of the IJCAI 2003 Workshop on Learning Statistical Models from Relational Data, Acapulco, Mexico, IJCAII, pp. 24–31 (2003)

    Google Scholar 

  9. Cussens, J.: Parameter estimation in stochastic logic programs. Machine Learning 44(3), 245–271 (2001)

    Article  MATH  Google Scholar 

  10. De Raedt, L., Dehaspe, L.: Clausal discovery. Machine Learning 26, 99–146 (1997)

    Article  MATH  Google Scholar 

  11. De Raedt, L., Frasconi, P., Kersting, K., Muggleton, S. (eds.): Probabilistic Inductive Logic Programming - Theory and Applications. Springer, Heidelberg (2008)

    MATH  Google Scholar 

  12. Della Pietra, S., Della Pietra, V., Laferty, J.: Inducing features of random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence 19, 380–392 (1997)

    Article  Google Scholar 

  13. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society, Series B 39, 1–38 (1977)

    MATH  MathSciNet  Google Scholar 

  14. Eshghi, K., Kowalski, R.: Abduction compared to negation by failure. In: Levi, G., Martelli, M. (eds.) Proceedings of the 6th international conference on logic programming, pp. 234–255. The MIT Press, Cambridge (1989)

    Google Scholar 

  15. Esposito, F., Lamma, E., Malerba, P., Mello, D., Milano, M., Riguzzi, F., Semeraro, G.: Learning abductive logic programs. In: Proceedings of the ECAI 1996 workshop on abductive and inductive reasoning, Budapest, pp. 23–30 (1996)

    Google Scholar 

  16. Esposito, F., Semeraro, G., Fanizzi, N., Ferilli, S.: Multistrategy theory revision: induction and abduction in inthelex. Machine Learning 38(1-2), 133–156 (2000)

    Article  MATH  Google Scholar 

  17. Friedman, N.: Learning belief networks in the presence of missing values and hidden variables. In: Fourteenth Inter. Conf. on Machine Learning, ICML 1997 (1997)

    Google Scholar 

  18. Furnkranz, J.: Separate-and-conquer rule learning. Artificial Intelligence Review 13(1), 3–54 (1999)

    Article  Google Scholar 

  19. Genesereth, M.R., Nilsson, N.J.: Logical foundations of artificial intelligence. Morgan Kaufmann, San Mateo (1987)

    MATH  Google Scholar 

  20. Getoor, L., Taskar, B.: Introduction to Statistical Relational Learning. MIT, Cambridge (2007)

    MATH  Google Scholar 

  21. Geyer, C.J., Thompson, E.A.: Constrained monte carlo maximum likelihood for dependent data. Journal of the Royal Statistical Society, Series B 54, 657–699 (1992)

    MathSciNet  Google Scholar 

  22. Halpern, J.: An analysis of first-order logics of probability. Artificial Intelligence 46, 311–350 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  23. Hoos, H.H., Stutzle, T.: Stochastic Local Search: Foundations and Applications. Morgan Kaufmann, San Francisco (2005)

    MATH  Google Scholar 

  24. Huynh, T.N., Mooney, R.J.: Discriminative structure and parameter learning for markov logic networks. In: Proc. of the 25th International Conference on Machine Learning, ICML (2008)

    Google Scholar 

  25. Kakas, A., Mancarella, P.: On the relation of truth maintenance and abduction. In: Proc. 1st Pacific Rim International Conference on Artificial Intelligence (1990)

    Google Scholar 

  26. Kakas, A., Riguzzi, F.: Learning with abduction. New Generation Computing 18(3), 243–294 (2000)

    Article  Google Scholar 

  27. Kakas, M., Kowalski, R., Toni, F.: Abductive logic programming. J. Logic. Comput., 718–770 (1993)

    Google Scholar 

  28. Kersting, K., De Raedt, L.: Towards combining inductive logic programming with bayesian networks. In: Rouveirol, C., Sebag, M. (eds.) ILP 2001. LNCS (LNAI), vol. 2157, pp. 118–131. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  29. Kok, S., Domingos, P.: Learning the structure of markov logic networks. In: Proc. 22nd Int’l Conf. on Machine Learning, pp. 441–448 (2005)

    Google Scholar 

  30. Koller, D., Levy, A., Pfeffer, A.: P-classic: A tractable probabilistic description logic. In: Proc. of NCAI 1997, pp. 360–397 (1997)

    Google Scholar 

  31. Lamma, E., Mello, P., Milano, M., Riguzzi, F., Esposito, F., Ferilli, S., Semeraro, G.: Cooperation of abduction and induction in logic programming. In: Abductive and inductive reasoning: essays on their relation and integration. Kluwer, Dordrecht (2000)

    Google Scholar 

  32. Landwehr, N., Kersting, K., De Raedt, L.: Integrating naive bayes and foil. Journal of Machine Learning Research, 481–507 (2007)

    Google Scholar 

  33. Lavrac, N., Dzeroski, S.: Inductive Logic Programming: Techniques and applications. UK, Ellis Horwood, Chichester (1994)

    Google Scholar 

  34. Liu, D.C., Nocedal, J.: On the limited memory bfgs method for large scale optimization. Mathematical Programming 45, 503–528 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  35. Loureno, H.R., Martin, O., Stutzle, T.: Iterated local search. In: Glover, F., Kochenberger, G. (eds.) Handbook of Metaheuristics, pp. 321–353. Kluwer Academic Publishers, Norwell (2002)

    Google Scholar 

  36. Lowd, D., Domingos, P.: Efficient weight learning for markov logic networks. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) PKDD 2007. LNCS (LNAI), vol. 4702, pp. 200–211. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  37. McCallum, A.: Efficiently inducing features of conditional random fields. In: Proc. UAI 2003, pp. 403–410 (2003)

    Google Scholar 

  38. Michalski, R.S.: Inferential theory of learning. developing foundations for multistrategy learning. In: Michalski, R.S., Tecuci, G. (eds.) Machine Learning. A Multistrategy Approach, vol. IV, pp. 3–61. Morgan Kaufmann, San Francisco

    Google Scholar 

  39. Mihalkova, L., Mooney, R.J.: Bottom-up learning of markov logic network structure. In: Proc. 24th Int’l Conf. on Machine Learning, pp. 625–632 (2007)

    Google Scholar 

  40. Mitchell, T.M.: Machine Learning. The McGraw-Hill Companies, Inc., New York (1997)

    MATH  Google Scholar 

  41. Muggleton, S.: Stochastic logic programs. In: De Raedt, L. (ed.) Advances in inductive logic programming. IOS Press, Amsterdam (1996)

    Google Scholar 

  42. Muggleton, S.H.: Inverse entailment and progol. New Generation Computing Journal, 245–286 (1995)

    Google Scholar 

  43. Koller, D., Friedman, N., Getoor, L., Pfeffer, A.: Learning probabilistic relational models. In: Proc. 16th Int’l Joint Conf. on AI (IJCAI), pp. 1300–1307. Morgan Kaufmann, San Francisco (1999)

    Google Scholar 

  44. Ngo, L., Haddawy, P.: Answering queries from context-sensitive probabilistic knowledge bases. Theoretical Computer Science 171, 147–177 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  45. Nienhuys-Cheng, S.-H., de Wolf, R.: Foundations of Inductive Logic Programming. Springer, Heidelberg (1997)

    Google Scholar 

  46. Nilsson, N.: Probabilistic logic. Artificial Intelligence 28, 71–87 (1986)

    Article  MATH  MathSciNet  Google Scholar 

  47. Pasula, H., Russell, S.: Approximate inference for first-order probabilistic languages. In: Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence, pp. 741–748. Morgan Kaufmann, Seattle (2001)

    Google Scholar 

  48. Pearl, J.: Probabilistic reasoning in intelligent systems: Networks of plausible inference. Morgan Kaufmann, San Francisco (1988)

    Google Scholar 

  49. Plotkin, G.D.: A note on inductive generalization. Machine Intelligence 5, 153–163 (1970)

    MathSciNet  Google Scholar 

  50. Poole, D.: A logical framework for default reasoning. Artif. Intell. 36, 27–47 (1988)

    Article  MATH  MathSciNet  Google Scholar 

  51. Poole, D.: Probabilistic horn abduction and bayesian networks. Artificial Intelligence 64, 81–129 (1993)

    Article  MATH  Google Scholar 

  52. Poon, H., Domingos, P.: Sound and efficient inference with probabilistic and deterministic dependencies. In: Proc. 21st Nat’l Conf. on AI (AAAI), pp. 458–463. AAAI Press, Menlo Park (2006)

    Google Scholar 

  53. Poon, H., Domingos, P., Sumner, M.: A general method for reducing the complexity of relational inference and its application to mcmc. In: Proc. 23rd Nat’l Conf. on Artificial Intelligence. AAAI Press, Chicago (2008)

    Google Scholar 

  54. Popescul, A., Ungar, L.H.: Structural logistic regression for link analysis. In: Proceedings of the Second International Workshop on Multi-Relational Data Mining, pp. 92–106. ACM Press, Washington (2003)

    Google Scholar 

  55. Quinlan, J.R.: Learning logical definitions from relations. Machine Learning 5, 239–266 (1990)

    Google Scholar 

  56. De Raedt, L.: Logical settings for concept-learning. Artificial Intelligence 95(1), 197–201 (1997)

    Google Scholar 

  57. Reiter, R.: A logic for default reasoning. J. Artif. Intell. (13), 81–132 (1980)

    Google Scholar 

  58. Richardson, M., Domingos, P.: Markov logic networks. Machine Learning 62, 107–236 (2006)

    Article  Google Scholar 

  59. Santos Costa, V., Page, D., Qazi, M., Cussens, J.: Clp(bn): Constraint logic programming for probabilistic knowledge. In: Proceedings of the Nineteenth Conference on Uncertainty in Artificial Intelligence, pp. 517–524. Morgan Kaufmann, Acapulco (2003)

    Google Scholar 

  60. Sato, T., Kameya, Y.: Prism: A symbolic-statistical modeling language. In: Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence, pp. 1330–1335. Morgan Kaufmann, Nagoya (1997)

    Google Scholar 

  61. Sato, T., Kameya, Y.: A viterbi-like algorithm and em learning for statistical abduction. In: Proceedings of UAI 2000 Workshop on Fusion of Domain Knowledge with Data for Decision Support (2000)

    Google Scholar 

  62. Sha, F., Pereira, F.: Shallow parsing with conditional random fields. In: Proc. HLT-NAACL 2003, pp. 134–141 (2003)

    Google Scholar 

  63. Shapiro, E.: Algorithmic Program Debugging. MIT Press, Cambridge (1983)

    Google Scholar 

  64. Singla, P., Domingos, P.: Discriminative training of markov logic networks. In: Proc. 20th Nat’l Conf. on AI (AAAI), pp. 868–873. AAAI Press, Menlo Park (2005)

    Google Scholar 

  65. Singla, P., Domingos, P.: Markov logic in infinite domains. In: Proc. 23rd UAI, pp. 368–375. AUAI Press (2007)

    Google Scholar 

  66. Srinivasan, A.: The Aleph Manual, http://www.comlab.ox.ac.uk/oucl/~esearch/areas/machlearn/Aleph/

  67. Taskar, B., Abbeel, P., Koller, D.: Discriminative probabilistic models for relational data. In: Proceedings of the Eighteenth Conference on Uncertainty in Artificial Intelligence, pp. 485–492. Morgan Kaufmann, Edmonton (2002)

    Google Scholar 

  68. Wellman, J.S., Breese, M., Goldman, R.P.: From knowledge bases to decision models. Knowledge Engineering Review 7 (1992)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Bari, Via E. Orabona 4, 70124, Italy

    Marenglen Biba, Stefano Ferilli & Floriana Esposito

Authors
  1. Marenglen Biba
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stefano Ferilli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Floriana Esposito
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institute of Computer Science, Polish Academy of Sciences, ul. Ordona 21, 01-237, Warsaw, Poland

    Jacek Koronacki  & Sławomir T. Wierzchoń  & 

  2. Woodward Hall 430C, University of North Carolina, 9201 University City Blvd., N.C. 28223, Charlotte, USA

    Zbigniew W. Raś

  3. Systems Research Institute, Polish Academy of Sciences, ul.Newelska 6, 01-447, Warsaw, Poland

    Janusz Kacprzyk

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Biba, M., Ferilli, S., Esposito, F. (2010). Towards Multistrategic Statistical Relational Learning. In: Koronacki, J., Raś, Z.W., Wierzchoń, S.T., Kacprzyk, J. (eds) Advances in Machine Learning II. Studies in Computational Intelligence, vol 263. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-05179-1_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-05179-1_6

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-05178-4

  • Online ISBN: 978-3-642-05179-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation