Learning without Coding

  • Samuel E. MoeliusIII
  • Sandra Zilles
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6331)


Iterative learning is a model of language learning from positive data, due to Wiehagen. When compared to a learner in Gold’s original model of language learning from positive data, an iterative learner can be thought of as memory-limited. However, an iterative learner can memorize some input elements by coding them into the syntax of its hypotheses. A main concern of this paper is: to what extent are such coding tricks necessary?

One means of preventing some such coding tricks is to require that the hypothesis space used be free of redundancy, i.e., that it be 1-1. By extending a result of Lange & Zeugmann, we show that many interesting and non-trivial classes of languages can be iteratively identified in this manner. On the other hand, we show that there exists a class of languages that cannot be iteratively identified using any 1-1 effective numbering as the hypothesis space.

We also consider an iterative-like learning model in which the computational component of the learner is modeled as an enumeration operator, as opposed to a partial computable function. In this new model, there are no hypotheses, and, thus, no syntax in which the learner can encode what elements it has or has not yet seen. We show that there exists a class of languages that can be identified under this new model, but that cannot be iteratively identified. On the other hand, we show that there exists a class of languages that cannot be identified under this new model, but that can be iteratively identified using a Friedberg numbering as the hypothesis space.


Coding tricks inductive inference iterative learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [Ang80]
    Angluin, D.: Finding patterns common to a set of strings. J. Comput. Syst. Sci. 21(1), 46–62 (1980)zbMATHCrossRefMathSciNetGoogle Scholar
  2. [BB75]
    Blum, L., Blum, M.: Toward a mathematical theory of inductive inference. Inform. Control 28(2), 125–155 (1975)zbMATHCrossRefGoogle Scholar
  3. [BBCJS10]
    Becerra-Bonache, L., Case, J., Jain, S., Stephan, F.: Iterative learning of simple external contextual languages. Theor. Comput. Sci. 411(29-30), 2741–2756 (2010)zbMATHCrossRefMathSciNetGoogle Scholar
  4. [Bei84]
    Beick, H.-R.: Induktive Inferenz mit höchster Konvergenzgeschwindigkeit. PhD thesis, Sektion Mathematik, Humboldt-Universität Berlin (1984)Google Scholar
  5. [Cas74]
    Case, J.: Periodicity in generations of automata. Math. Syst. Theory 8(1), 15–32 (1974)zbMATHCrossRefMathSciNetGoogle Scholar
  6. [Cas94]
    Case, J.: Infinitary self-reference in learning theory. J. Exp. Theor. Artif. In. 6(1), 3–16 (1994)zbMATHCrossRefMathSciNetGoogle Scholar
  7. [CCJS07]
    Carlucci, L., Case, J., Jain, S., Stephan, F.: Results on memory-limited U-shaped learning. Inform. Comput. 205(10), 1551–1573 (2007)zbMATHCrossRefMathSciNetGoogle Scholar
  8. [CK10]
    Case, J., Kötzing, T.: Strongly non-U-shaped learning results by general techniques. In: Proc. of COLT 2010, pp. 181–193 (2010)Google Scholar
  9. [CM08]
    Case, J., Moelius, S.: Optimal language learning. In: Freund, Y., Györfi, L., Turán, G., Zeugmann, T. (eds.) ALT 2008. LNCS (LNAI), vol. 5254, pp. 419–433. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  10. [dBY10]
    de Brecht, M., Yamamoto, A.: Topological properties of concept spaces (full version). Inform. Comput. 208(4), 327–340 (2010)zbMATHCrossRefGoogle Scholar
  11. [FKW82]
    Freivalds, R., Kinber, E., Wiehagen, R.: Inductive inference and computable one-one numberings. Z. Math. Logik 28(27), 463–479 (1982)zbMATHCrossRefMathSciNetGoogle Scholar
  12. [Fri58]
    Friedberg, R.: Three theorems on recursive enumeration. I. Decomposition. II. Maximal set. III. Enumeration without duplication. J. Symbolic Logic 23(3), 309–316 (1958)CrossRefMathSciNetGoogle Scholar
  13. [Ful90]
    Fulk, M.: Prudence and other conditions on formal language learning. Inform. Comput. 85(1), 1–11 (1990)zbMATHCrossRefMathSciNetGoogle Scholar
  14. [Gol67]
    Mark Gold, E.: Language identification in the limit. Inform. Control 10(5), 447–474 (1967)zbMATHCrossRefGoogle Scholar
  15. [Jai10]
    Jain, S.: Private communcation (2010)Google Scholar
  16. [JLMZ10]
    Jain, S., Lange, S., Moelius, S., Zilles, S.: Incremental learning with temporary memory. Theor. Comput. Sci. 411(29-30), 2757–2772 (2010)zbMATHCrossRefMathSciNetGoogle Scholar
  17. [JS08]
    Jain, S., Stephan, F.: Learning in Friedberg numberings. Inform. Comput. 206(6), 776–790 (2008)zbMATHCrossRefMathSciNetGoogle Scholar
  18. [Kum90]
    Kummer, M.: An easy priority-free proof of a theorem of Friedberg. Theor. Comput. Sci. 74(2), 249–251 (1990)zbMATHCrossRefMathSciNetGoogle Scholar
  19. [LW91]
    Lange, S., Wiehagen, R.: Polynomial time inference of arbitrary pattern languages. New Generat. Comput. 8(4), 361–370 (1991)zbMATHCrossRefGoogle Scholar
  20. [LZ96]
    Lange, S., Zeugmann, T.: Incremental learning from positive data. J. Comput. Syst. Sci. 53(1), 88–103 (1996)zbMATHCrossRefMathSciNetGoogle Scholar
  21. [LZZ08]
    Lange, S., Zeugmann, T., Zilles, S.: Learning indexed families of recursive languages from positive data: A survey. Theor. Comput. Sci. 397(1-3), 194–232 (2008)zbMATHCrossRefMathSciNetGoogle Scholar
  22. [MZ10]
    Moelius, S., Zilles, S.: Learning without coding (2010),(unpublished manuscript),
  23. [Rog67]
    Rogers, H.: Theory of Recursive Functions and Effective Computability. McGraw Hill, New York (1967); Reprinted, MIT Press (1987)zbMATHGoogle Scholar
  24. [SSS+94]
    Shimozono, S., Shinohara, A., Shinohara, T., Miyano, S., Kuhara, S., Arikawa, S.: Knowledge acquisition from amino acid sequences by machine learning system BONSAI. Trans. Inform. Process. Soc. Jpn. 35(10), 2009–2018 (1994)Google Scholar
  25. [Wie76]
    Wiehagen, R.: Limes-Erkennung rekursiver Funktionen durch spezielle Strategien. J. Inform. Process. Cybern. (EIK) 12(1/2), 93–99 (1976)zbMATHMathSciNetGoogle Scholar
  26. [Wie91]
    Wiehagen, R.: A thesis in inductive inference. In: Dix, J., Schmitt, P.H., Jantke, K.P. (eds.) NIL 1990. LNCS (LNAI), vol. 543, pp. 184–207. Springer, Heidelberg (1991)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Samuel E. MoeliusIII
    • 1
  • Sandra Zilles
    • 2
  1. 1.IDA Center for Computing SciencesBowie
  2. 2.Department of Computer ScienceUniversity of ReginaReginaCanada

Personalised recommendations