Advertisement

Solutions to Open Questions for Non-U-Shaped Learning with Memory Limitations

  • John Case
  • Timo Kötzing
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6331)

Abstract

A U-shape occurs when a learner first learns, then unlearns, and, finally, relearns, some target concept. Within the framework of Inductive Inference, previous results have shown, for example, that U-shapes are unnecessary for explanatory learning, but are necessary for behaviorally correct learning.

This paper solves the following two problems regarding non-U-shaped learning posed in the prior literature.

First, it is shown that there are classes learnable with three memory states that are not learnable non-U-shapedly with any finite number of memory states. This result is surprising, as for learning with one or two memory states, U-shapes are known to be unnecessary.

Secondly, it is shown that there is a class learnable memorylessly with a single feedback query such that this class is not learnable non-U-shapedly memorylessly with any finite number of feedback queries. This result is complemented by the result that any class of infinite languages learnable memorylessly with finitely many feedback queries is so learnable without U-shapes – in fact, all classes of infinite languages learnable with complete memory are learnable memorylessly with finitely many feedback queries and without U-shapes. On the other hand, we show that there is a class of infinite languages learnable memorylessly with a single feedback query, which is not learnable without U-shapes by any particular bounded number of feedback queries.

Keywords

Inductive Inference 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [BCM+08]
    G. Baliga, J. Case, W. Merkle, F. Stephan, and W. Wiehagen. When unlearning helps. Information and Computation, 206:694-709, 2008.zbMATHCrossRefMathSciNetGoogle Scholar
  2. [Car82]
    Carey, S.: Face perception: Anomalies of development. In: Strauss, S., Stavy, R. (eds.) U-Shaped Behavioral Growth. Developmental Psychology Series. Academic Press, NY (1982)Google Scholar
  3. [Cas74]
    Case, J.: Periodicity in generations of automata. Mathematical Systems Theory 8, 15–32 (1974)zbMATHCrossRefMathSciNetGoogle Scholar
  4. [Cas94]
    Case, J.: Infinitary self-reference in learning theory. Journal of Experimental and Theoretical Artificial Intelligence 6, 3–16 (1994)zbMATHCrossRefGoogle Scholar
  5. [Cas99]
    Case, J.: The power of vacillation in language learning. SIAM Journal on Computing 28, 1941–1969 (1999)zbMATHCrossRefMathSciNetGoogle Scholar
  6. [CCJS07a]
    Carlucci, L., Case, J., Jain, S., Stephan, F.: Non-U-shaped vacillatory and team learning. Journal of Computer and System Sciences, Special Issue in Memory of Carl Smith (2007)Google Scholar
  7. [CCJS07b]
    Carlucci, L., Case, J., Jain, S., Stephan, F.: Results on memory-limited U-shaped learning. Information and Computation 205, 1551–1573 (2007)zbMATHCrossRefMathSciNetGoogle Scholar
  8. [CJLZ99]
    Case, J., Jain, S., Lange, S., Zeugmann, T.: Incremental concept learning for bounded data mining. Information and Computation 152, 74–110 (1999)zbMATHCrossRefMathSciNetGoogle Scholar
  9. [CL82]
    Case, J., Lynes, C.: Machine inductive inference and language identification. In: Nielsen, M., Schmidt, E.M. (eds.) ICALP 1982. LNCS, vol. 140, pp. 107–115. Springer, Heidelberg (1982)CrossRefGoogle Scholar
  10. [FJO94]
    Fulk, M., Jain, S., Osherson, D.: Open problems in Systems That Learn. Journal of Computer and System Sciences 49, 589–604 (1994)CrossRefMathSciNetGoogle Scholar
  11. [FKS95]
    Freivalds, R., Kinber, E., Smith, C.: On the impact of forgetting on learning machines. Journal of the ACM 42, 1146–1168 (1995)zbMATHCrossRefMathSciNetGoogle Scholar
  12. [Gol67]
    Gold, E.: Language identification in the limit. Information and Control 10, 447–474 (1967)zbMATHCrossRefGoogle Scholar
  13. [JK09]
    Jain, S., Kinber, E.: Iterative learning from texts and counterexamples using additional information. In: Gavaldà, R., Lugosi, G., Zeugmann, T., Zilles, S. (eds.) ALT 2009. LNCS, vol. 5809, pp. 308–322. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  14. [JLMZ10]
    Jain, S., Lange, S., Moelius III, S.E., Zilles, S.: Incremental learning with temporary memory. Theoretical Computer Science 411, 2757–2772 (2010)zbMATHCrossRefMathSciNetGoogle Scholar
  15. [JORS99]
    Jain, S., Osherson, D., Royer, J., Sharma, A.: Systems that Learn: An Introduction to Learning Theory, 2nd edn. MIT Press, Cambridge (1999)Google Scholar
  16. [KS95]
    Kinber, E., Stephan, F.: Language learning from texts: Mind changes, limited memory and monotonicity. Information and Computation 123, 224–241 (1995)zbMATHCrossRefMathSciNetGoogle Scholar
  17. [LZ96]
    Lange, S., Zeugmann, T.: Incremental learning from positive data. Journal of Computer and System Sciences 53, 88–103 (1996)zbMATHCrossRefMathSciNetGoogle Scholar
  18. [MPU+92]
    Marcus, G., Pinker, S., Ullman, M., Hollander, M., Rosen, T.J., Xu, F.: Overregularization in Language Acquisition. In: Monographs of the Society for Research in Child Development, vol. 57(4). University of Chicago Press, Chicago (1992); Includes commentary by H. ClahsenGoogle Scholar
  19. [OSW86]
    Osherson, D., Stob, M., Weinstein, S.: Systems that Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists. MIT Press, Cambridge (1986)Google Scholar
  20. [OW82]
    Osherson, D., Weinstein, S.: Criteria of language learning. Information and Control 52, 123–138 (1982)zbMATHCrossRefMathSciNetGoogle Scholar
  21. [RC94]
    Royer, J., Case, J.: Subrecursive Programming Systems: Complexity and Succinctness. In: Research monograph in Progress in Theoretical Computer Science. Birkhäuser, Basel (1994)Google Scholar
  22. [Rog67]
    Rogers, H.: Theory of Recursive Functions and Effective Computability. McGraw Hill, New York (1967); Reprinted by MIT Press, Cambridge, Massachusetts (1987)zbMATHGoogle Scholar
  23. [SS82]
    Strauss, S., Stavy, R. (eds.): U-Shaped Behavioral Growth. Developmental Psychology Series. Academic Press, NY (1982)Google Scholar
  24. [TA02]
    Taatgen, N.A., Anderson, J.R.: Why do children learn to say broke? A model of learning the past tense without feedback. Cognition 86, 123–155 (2002)CrossRefGoogle Scholar
  25. [WC80]
    Wexler, K., Culicover, P.: Formal Principles of Language Acquisition. MIT Press, Cambridge (1980)Google Scholar
  26. [Wie76]
    Wiehagen, R.: Limes-Erkennung rekursiver Funktionen durch spezielle Strategien. Elektronische Informationverarbeitung und Kybernetik 12, 93–99 (1976)zbMATHMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • John Case
    • 1
  • Timo Kötzing
    • 2
  1. 1.Department of Computer and Information SciencesUniversity of DelawareNewarkUSA
  2. 2.Max-Planck-Institut für InformatikSaarbrückenGermany

Personalised recommendations