Empirical Study of Relational Learning Algorithms in the Phase Transition Framework

  • Erick Alphonse
  • Aomar Osmani
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5781)


Relational Learning (RL) has aroused interest to fill the gap between efficient attribute-value learners and growing applications stored in multi-relational databases. However, current systems use general- purpose problem solvers that do not scale-up well. This is in contrast with the past decade of success in combinatorics communities where studies of random problems, in the phase transition framework, allowed to evaluate and develop better specialised algorithms able to solve real-world applications up to millions of variables. A number of studies have been proposed in RL, like the analysis of the phase transition of a NP-complete sub-problem, the subsumption test, but none has directly studied the phase transition of RL. As RL, in general, is \({\it \Sigma}_2-hard\), we propose a first random problem generator, which exhibits the phase transition of its decision version, beyond NP. We study the learning cost of several learners on inherently easy and hard instances, and conclude on expected benefits of this new benchmarking tool for RL.


Predicate Symbol Inductive Logic Programming Hypothesis Space Hard Instance Polynomial Hierarchy 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Domingos, P.: Prospects and challenges for multi-relational data mining. SIGKDD Explorations 5(1), 80–83 (2003)CrossRefGoogle Scholar
  2. 2.
    Page, D., Srinivasan, A.: Ilp: a short look back and a longer look forward. J. Mach. Learn. Res. 4, 415–430 (2003)zbMATHGoogle Scholar
  3. 3.
    Mitchell, T.M.: Generalization as search. Artificial Intelligence 18, 203–226 (1982)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Newell, A., Simon, H.A.: Human Problem Solving. Prentice-Hall, Englewood Cliffs (1972)Google Scholar
  5. 5.
    Cheeseman, P., Kanefsky, B., Taylor, W.: Where the really hard problems are. In: Proc. of the 12th International Joint Conference on Artificial Intelligence, pp. 331–340. Morgan Kaufmann, San Francisco (1991)Google Scholar
  6. 6.
    Mitchell, D., Selman, B., Levesque, H.: Hard and easy distribution of SAT problems. In: Proceedings of the Tenth National Conference on Artificial Intelligence (AAAI 1992), pp. 440–446 (1992)Google Scholar
  7. 7.
    Monasson, R., Zecchina, R., Kirkpatrick, S., Selman, B., Troyansky, L.: Determining computational complexity from characteristic ’phase transitions. Nature 400, 133–137 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Smith, B.M., Dyer, M.E.: Locating the phase transition in binary constraint satisfaction problems. Artificial Intelligence 81(1-2), 155–181 (1996)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Hogg, T., Williams, C.: The hardest constraint problems: A double phase transition. Artificial Intelligence 69(1–2), 359–377 (1994)CrossRefzbMATHGoogle Scholar
  10. 10.
    Mammen, D.L., Hogg, T.: A new look at the easy-hard-easy pattern of combinatorial search difficulty. Journal of Artificial Intelligence Research 7, 47–66 (1997)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Gomes, C., Heny Kautz, A.S., Selman, B.: Satisfiability solvers. In: Handbook of Knowledge Representation (2007)Google Scholar
  12. 12.
    Haussler, D.: Learning conjunctive concepts in structural domains. Machine Learning 4(1), 7–40 (1989)Google Scholar
  13. 13.
    Kearns, M.J., Vazirani, U.V.: An Introduction to Computational Learning Theory. The MIT Press, Cambridge (1994)Google Scholar
  14. 14.
    En, N., Srensson, N.: Translating pseudo-boolean constraints into SAT. Journal on Satisfiability, Boolean Modeling and Computation 2, 1–26 (2006)Google Scholar
  15. 15.
    Fürnkranz, J.: Pruning algorithms for rule learning. Mach. Learn. 27(2), 139–172 (1997)CrossRefGoogle Scholar
  16. 16.
    Shim, G.M., Choi, M.Y., Kim, D.: Phase transitions in a dynamic model of neural networks. Physics Review A 43, 1079–1089 (1991)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Nagashino, H., Kelso, J.A.: Phase transitions in oscillatory neural networks. In: Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, vol. 1710, pp. 279–287 (1992)Google Scholar
  18. 18.
    Schottky, B.: Phase transitions in the generalization behaviour of multilayer neural networks. Journal of Physics A Mathematical General 28, 4515–4531 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Biehl, M., Ahr, M., Schlsser, E.: Statistical physics of learning: Phase transitions in multilayered neural networks. Advances in Solid State Physics 40/2000, 819–826 (2000)CrossRefGoogle Scholar
  20. 20.
    Botta, M., Giordana, A., Saitta, L., Sebag, M.: Relational learning as search in a critical region. Journal of Machine Learning Research 4, 431–463 (2003)MathSciNetzbMATHGoogle Scholar
  21. 21.
    Alphonse, E., Osmani, A.: A model to study phase transition and plateaus in relational learning. In: Proc. of Conf. on Inductive Logic Programming, pp. 6–23 (2008)Google Scholar
  22. 22.
    Rückert, U., Kramer, S., Raedt, L.D.: Phase transitions and stochastic local search in k-term DNF learning. In: Elomaa, T., Mannila, H., Toivonen, H. (eds.) ECML 2002. LNCS (LNAI), vol. 2430, pp. 405–417. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  23. 23.
    Gottlob, G., Leone, N., Scarcello, F.: On the complexity of some inductive logic programming problems. In: Džeroski, S., Lavrač, N. (eds.) ILP 1997. LNCS, vol. 1297, pp. 17–32. Springer, Heidelberg (1997)CrossRefGoogle Scholar
  24. 24.
    Bylander, T.: A probabilistic analysis of propositional strips planning. Artificial Intelligence 81(1-2), 241–271 (1996)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Gent, I.P., Walsh, T.: Beyond NP: the QSAT phase transition. In: AAAI 1999/IAAI 1999: Proceedings of the sixteenth national conference on Artificial intelligence and the eleventh Innovative applications of artificial intelligence conference, pp. 648–653 (1999)Google Scholar
  26. 26.
    Chen, H., Interian, Y.: A model for generating random quantified boolean formulas. In: Kaelbling, L.P., Saffiotti, A. (eds.) Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence, pp. 66–71. Professional Book Center (2005)Google Scholar
  27. 27.
    Srinivasan, A.: A learning engine for proposing hypotheses (Aleph) (1999),
  28. 28.
    Muggleton, S.: Inverse entailment and PROGOL. New Generation Computing 13, 245–286 (1995)CrossRefGoogle Scholar
  29. 29.
    Alphonse, É., Rouveirol, C.: Extension of the top-down data-driven strategy to ILP. In: Muggleton, S.H., Otero, R., Tamaddoni-Nezhad, A. (eds.) ILP 2006. LNCS (LNAI), vol. 4455, pp. 49–63. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  30. 30.
    Cook, S.A., Mitchell, D.G.: Finding hard instances of the satisfiability problem: A survey. In: DIMACS Series in Discrete Mathematics and Theoretical Computer Science, pp. 1–17. American Mathematical Society (1997)Google Scholar
  31. 31.
    Xu, K., Li, W.: Many hard examples in exact phase transitions. Theor. Comput. Sci. 355(3), 291–302 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Gottlob, G.: Subsumption and implication. Information Processing Letters 24(2), 109–111 (1987)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Fürnkranz, J.: A pathology of bottom-up hill-climbing in inductive rule learning. In: Cesa-Bianchi, N., Numao, M., Reischuk, R. (eds.) ALT 2002. LNCS (LNAI), vol. 2533, pp. 263–277. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  34. 34.
    Plotkin, G.: A note on inductive generalization. In: Machine Intelligence, pp. 153–163. Edinburgh University Press (1970)Google Scholar
  35. 35.
    Valiant, L.G.: A theory of the learnable. In: ACM Symposium on Theory of Computing (STOC 1984), Baltimore, USA, pp. 436–445. ACM Press, New York (1984)Google Scholar
  36. 36.
    Kietz, J.U.: A comparative study of structural most specific generalisations used in machine learning. In: Proc. Third Workshop on ILP, pp. 149–164 (1993)Google Scholar
  37. 37.
    Muggleton, S., Feng, C.: Efficient induction of logic programs. In: Proc. of the 1st Conference on Algorithmic Learning Theory, Ohmsma, Tokyo, Japan, pp. 368–381 (1990)Google Scholar
  38. 38.
    Paskal, Y.I.: The meaning of the terms phase and phase transition. Russian Physics Journal 31(8), 664–666 (1988)Google Scholar
  39. 39.
    Selman, B., Levesque, H.J., Mitchell, D.: A new method for solving hard satisfiability problems. In: Proc. of the Tenth National Conference on Artificial Intelligence, Menlo Park, California, pp. 440–446 (1992)Google Scholar
  40. 40.
    Gent, I.P., Walsh, T.: Easy problems are sometimes hard. Artificial Intelligence 70(1–2), 335–345 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  41. 41.
    Davenport, A.: A comparison of complete and incomplete algorithms in the easy and hard regions. In: Workshop on Studying and Solving Really Hard Problems, CP 1995, pp. 43–51 (1995)Google Scholar
  42. 42.
    Smith, B.M.: Constructing an asymptotic phase transition in random binary constraint satisfaction problems. Theoretical Computer Science 265(1–2), 265–283 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Xu, K., Boussemart, F., Hemery, F., Lecoutre, C.: Random constraint satisfaction: Easy generation of hard (satisfiable) instances. Artif. Intell. 171(8-9), 514–534 (2007)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Erick Alphonse
    • 1
  • Aomar Osmani
    • 1
  1. 1.LIPN-CNRS UMR 7030France

Personalised recommendations