# Predicate invention and learning from positive examples only

## Abstract

Previous bias shift approaches to predicate invention are not applicable to learning from positive examples only, if a complete hypothesis can be found in the given language, as negative examples are required to determine whether new predicates should be invented or not. One approach to this problem is presented, MERLIN 2.0, which is a successor of a system in which predicate invention is guided by sequences of input clauses in SLD-refutations of positive and negative examples w.r.t. an overly general theory. In contrast to its predecessor which searches for the minimal finite-state automaton that can generate all positive and no negative sequences, MERLIN 2.0 uses a technique for inducing Hidden Markov Models from positive sequences only. This enables the system to invent new predicates without being triggered by negative examples. Another advantage of using this induction technique is that it allows for incremental learning. Experimental results are presented comparing MERLIN 2.0 with the positive only learning framework of Progol 4.2 and comparing the original induction technique with a new version that produces deterministic Hidden Markov Models. The results show that predicate invention may indeed be both necessary and possible when learning from positive examples only as well as it can be beneficial to keep the induced model deterministic.

## Keywords

Hide Markov Model Logic Program Positive Sequence Inductive Logic Programming Negative Sequence## References

- 1.Bain M. and Muggleton S., “Non-Monotonic Learning”, in Muggleton S. (ed.),
*Inductive Logic Programming*, Academic Press (1992) 145–161Google Scholar - 2.Bar-Hillel Y., Perles M. and Shamir E., “On formal properties of simple phrase structure grammars”,
*Zeitschrift für Phonetik, Sprachwissenschaft und Kommunikationsforschung*,**14**, 1, Akademie Verlag, Berlin (1961) 143–172Google Scholar - 3.Baum L., Petrie T, Soules G. and Weiss N., “A maximization technique occurring in the statistical analysis of probabilistic functions in Markov chains”,
*The Annals of Mathematical Statistics***41**(1970) 164–171Google Scholar - 4.Biermann A. W. and Feldman J. A., “On the Synthesis of Finite-State Machines from Samples of Their Behavior”,
*IEEE Transactions on Computers*21 (1972) 592–597Google Scholar - 5.Boström H., “Theory-Guided Induction of Logic Programs by Inference of Regular Languages”,
*Proc. of the 13th International Conference on Machine Learning*, Morgan Kaufmann (1996) 46–53Google Scholar - 6.Kijsirikul B., Numao M. and Shimura M., “Discrimination-based constructive induction of logic programs”,
*Proceedings of the 10th National Conference on Artificial Intelligence*, Morgan Kaufmann (1992) 44–49Google Scholar - 7.Lewis H. R. and Papadimitriou C. H.,
*Elements of the Theory of Computation*, Prentice-Hall (1981)Google Scholar - 8.Lapointe S., Ling, C. and Matwin S., “Constructive Inductive Logic Programming”,
*Proceedings of the 13th International Joint Conference on Artificial Intelligence*, Morgan Kaufmann (1993) 1030–1036Google Scholar - 9.Lloyd J. W.,
*Foundations of Logic Programming*, (2nd edition), Springer-Verlag (1987)Google Scholar - 10.Muggleton S., “Inverse entailment and Progol”,
*New Generation Computing***13**(1995) 245–286Google Scholar - 11.Muggleton S., “Learning from positive data”,
*Proc. of the Sixth International Workshop on Inductive Logic Programming*(1996)Google Scholar - 12.Muggleton S., “Stochastic Logic Programs”,
*Advances in Inductive Logic Programming*(Ed. L. De Raedt), IOS Press (1996) 254–264Google Scholar - 13.Stahl I., “Predicate Invention in Inductive Logic Programming”,
*Advances in Inductive Logic Programming*(Ed. L. De Raedt), IOS Press (1996) 34–47Google Scholar - 14.Stolcke A. and Omohundro S., “Best-first Model Merging for Hidden Markov Model Induction”, TR-94-003, International Computer Science Institute, Berkeley, CA (1994)Google Scholar
- 15.Wirth R. and O'Rorke P., “Constraints on Predicate Invention”,
*Proceedings of the 8th International Workshop on Machine Learning*, Morgan Kaufmann (1991) 457–461Google Scholar - 16.Wrobel S., “Concept Formation During Interactive Theory Revision”,
*Machine Learning Journal***14**(1994) 169–192CrossRefGoogle Scholar