Machine induction without revolutionary paradigm shifts
This paper provides a beginning study of the effects on inductive inference of paradigm shifts whose absence is approximately modeled by various formal approaches to forbidding large changes in the size of programs conjectured.
One approach, called severe parsimony, requires all the programs conjectured on the way to success to be nearly (i.e., within a recursive function of) minimal size. It is shown that this very conservative constraint allows learning infinite classes of functions, but not infinite r.e. classes of functions.
Another approach, called non-revolutionary, requires all conjectures to be nearly the same size as one another. This quite conservative constraint is, nonetheless, shown to permit learning some infinite r.e. classes of functions. Allowing, up to one extra bounded size mind change towards a final program learned certainly doesn't appear revolutionary. However, somewhat surprisingly for scientific (inductive) inference, it is shown that there are classes learnable with the non-revolutionary constraint (respectively, with severe parsimony), up to (i}+1) mind changes, and no anomalies, which classes cannot be learned with no size constraint, an unbounded, finite number of anomalies in the final program, but with no more than i mind changes. Hence, in some cases, the possibility of one extra mind change is considerably more liberating than removal of very conservative size shift constraints. The proof of these results is also combinatorially interesting.
KeywordsParadigm Shift Initial Segment Recursive Function Computable Function Inductive Inference
Unable to display preview. Download preview PDF.
- 1.D. Angluin, W. Gasarch, and C. Smith. Training sequences. Theoretical Computer Science, 66(3):255–272, 1989.Google Scholar
- 2.G. Baliga, J. Case, S. Jain, and M. Suraj. Machine learning of higher order programs. Journal of Symbolic Logic, 59(2):486–500, 1994.Google Scholar
- 3.J. M. Barzdin and K. Podnieks. The theory of inductive inference. In Mathematical Foundations of Computer Science, High Tatras, Czechoslovakia, pages 9–15, 1973.Google Scholar
- 6.J. Case. The power of vacillation in language learning. Technical Report 93-08, University of Delaware, 1992. Expands on the article in Proceedings of the Workshop on Computational Learning Theory, Morgan Kauffman, 1988; journal article under revision.Google Scholar
- 7.J. Case, S. Jain, and S. Ngo Manguelle. Refinements of inductive inference by Popperian and reliable machines. Kybernetika, 30:23–52, 1994.Google Scholar
- 8.J. Case, S. Jain, and A. Sharma. On learning limiting programs. International Journal of Foundations of Computer Science, 3(1):93–115, 1992.Google Scholar
- 9.J. Case, S. Jain, and A. Sharma. Vacillatory learning of nearly minimal size grammers. Journal of Computer and System Sciences, 49(2):189–207, October 1994.Google Scholar
- 10.J. Case, S. Jain, and M. Suraj. Not-so-nearly-minimal-size program inference. In Klaus P. Jantke and Steffen Lange, editors, Algorithmic Learning for Knowledge-Based Systems, volume 961 of Lecture Notes in Artificial Intelligence, pages 77–96. Springer-Verlag, 1995.Google Scholar
- 13.R. Freivalds. Minimal Gödel numbers and their identification in the limit. Lecture Notes in Computer Science, 32:219–225, 1975.Google Scholar
- 14.M. A. Fulk and S. Jain. Approximate inference and scientific method. Information and Computation, 114(2):179–191, November 1994.Google Scholar
- 16.S. Jain. On a question about learning nearly minimal programs. Information Processing Letters, 53(1):1–4, January 1995.Google Scholar
- 17.S. Jain and A. Sharma. Restrictions on grammar size in language identification. In David Powers and Larry Reeker, editors, Proceedings MLNLO '91, Machine Learning of Natural Language and Ontology, Stanford University, California. Document D91-09, DFKI: Kaiserslautern FRG, 1991., pages 87–92, March 1991.Google Scholar
- 18.S. Jain and A. Sharma. Prudence in vacillatory language identification. Mathematical Systems Theory, 1994. To Appear.Google Scholar
- 19.Thomas Kuhn. The Structure of Scientific Revolutions. University of Chicago Press, Chicago, 1970.Google Scholar
- 20.S. Lange and P. Watson. Machine discovery in the presence of incomplete or ambiguous data. In K. Jantke and S. Arikawa, editors, Algorithmic Learning Theory, volume 872 of Lecture Notes in Artificial Intelligence, pages 438–452. Springer-Verlag, Berlin, Reinhardsbrunn Castle, Germany, October 1994.Google Scholar
- 21.M. Machtey and P. Young. An Introduction to the General Theory of Algorithms. North Holland, New York, 1978.Google Scholar
- 22.H. Putnam. Probability and confirmation. In Mathematics, Matter, and Method. Cambridge University Press, 1975.Google Scholar
- 23.G. Riccardi. The Independence of Control Structures in Abstract Programming Systems. PhD thesis, SUNY/ Buffalo, 1980.Google Scholar
- 24.G. Riccardi. The independence of control structures in abstract programming systems. Journal of Computer and System Sciences, 22:107–143, 1981.Google Scholar
- 25.H. Rogers. Gödel numberings of partial recursive functions. Journal of Symbolic Logic, 23:331–341, 1958.Google Scholar
- 26.H. Rogers. Theory of Recursive Functions and Effective Computability. McGraw Hill, New York, 1967. Reprinted, MIT Press 1987.Google Scholar
- 27.J. Royer. A Connotational Theory of Program Structure. Lecture Notes in Computer Science 273. Springer Verlag, 1987.Google Scholar
- 28.N. Shapiro. Review of “Limiting recursion” by E.M. Gold and “Trial and error predicates and the solution to a problem of Mostowski” by H. Putnam. Journal of Symbolic Logic, 36:342, 1971.Google Scholar