Inductive Inference Systems for Learning Classes of Algorithmically Generated Sets and Structures

Abstract

Computability theorists have extensively studied sets A the elements of which can be enumerated by Turing machines. These sets, also called computably enumerable sets, can be identified with their Gödel codes. Although each Turing machine has a unique Gödel code, different Turing machines can enumerate the same set. Thus, knowing a computably enumerable set means knowing one of its infinitely many Gödel codes. In the approach to learning theory stemming from E.M. Gold’s seminal paper [9], an inductive inference learner for a computably enumerable set A is a system or a device, usually algorithmic, which when successively (one by one) fed data for A outputs a sequence of Gödel codes (one by one) that at a certain point stabilize at codes correct for A. The convergence is called semantic or behaviorally correct, unless the same code for A is eventually output, in which case it is also called syntactic or explanatory. There are classes of sets that are semantically inferable, but not syntactically inferable.

Here, we are also concerned with generalizing inductive inference from sets, which are collections of distinct elements that are mutually independent, to mathematical structures in which various elements may be interrelated. This study was recently initiated by F. Stephan and Yu. Ventsov. For example, they systematically investigated inductive inference of the ideals of computable rings. With F. Stephan we continued this line of research by studying inductive inference of computably enumerable vector subspaces and other closure systems.

In particular, we showed how different convergence criteria interact with different ways of supplying data to the learner. Positive data for a set A are its elements, while negative data for A are the elements of its complement. Inference fromtext means that only positive data are supplied to the learner. Moreover, in the limit, all positive data are given. Inference fromswitching means that changes from positive to negative data or vice versa are allowed, but if there are only finitely many such changes, then in the limit all data of the eventually requested type (either positive or negative) are supplied. Inference from an informant means that positive and negative data are supplied alternately, but in the limit all data are supplied. For sets, inference from switching is more restrictive than inference from an informant, but more powerful than inference from text. On the other hand, for example, the class of computably enumerable vector spaces over an infinite field, which is syntactically inferable from text does not change if we allow semantic convergence, or inference from switching, but not both at the same time. While many classes of inferable algebraic structures have nice algebraic characterizations when learning from text or from switching is considered, we do not know of such characterizations for learning from an informant.