Abstract
In this paper we introduce a paradigm for learning in the limit of potentially infinite languages from all positive data and negative counterexamples provided in response to the conjectures made by the learner. Several variants of this paradigm are considered that reflect different conditions/constraints on the type and size of negative counterexamples and on the time for obtaining them. In particular, we consider the models where 1) a learner gets the least negative counterexample; 2) the size of a negative counterexample must be bounded by the size of the positive data seen so far; 3) a counterexample may be delayed. Learning power, limitations of these models, relationships between them, as well as their relationships with classical paradigms for learning languages in the limit (without negative counterexamples) are explored. Several surprising results are obtained. In particular, for Gold’s model of learning requiring a learner to syntactically stabilize on correct conjectures, learners getting negative counterexamples immediately turn out to be as powerful as the ones that do not get them for indefinitely (but finitely) long time (or are only told that their latest conjecture is not a subset of the target language, without any specific negative counterexample). Another result shows that for behaviourally correct learning (where semantic convergence is required from a learner) with negative counterexamples, a learner making just one error in almost all its conjectures has the “ultimate power”: it can learn the class of all recursively enumerable languages. Yet another result demonstrates that sometimes positive data and negative counterexamples provided by a teacher are not enough to compensate for full positive and negative data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Angluin, D.: Queries and concept learning. Machine Learning 2, 319–342 (1988)
Bārzdiņš, J.: Two theorems on the limiting synthesis of functions. Theory of Algorithms and Programs 1, 82–88 (1974) (in Russian)
Baliga, G., Case, J., Jain, S.: Language learning with some negative information. Journal of Computer and System Sciences 51(5), 273–285 (1995)
Brown, R., Hanlon, C.: Derivational complexity and the order of acquisition in child speech. In: Hayes, J.R. (ed.) Cognition and the Development of Language, Wiley, Chichester (1970)
Case, J., Lynes, C.: Machine inductive inference and language identification. In: Nielsen, M., Schmidt, E.M. (eds.) ICALP 1982. LNCS, vol. 140, pp. 107–115. Springer, Heidelberg (1982)
Case, J., Smith, C.: Comparison of identification criteria for machine inductive inference. Theoretical Computer Science 25, 193–220 (1983)
Demetras, M., Post, K., Snow, C.: Feedback to first language learners: The role of repetitions and clarification questions. Journal of Child Language 13, 275–292 (1986)
Fortnow, L., Gasarch, W., Jain, S., Kinber, E., Kummer, M., Kurtz, S., Pleszkoch, M., Slaman, T., Solovay, R., Stephan, F.: Extremes in the degrees of inferability. Annals of Pure and Applied Logic 66, 231–276 (1994)
Gasarch, W., Martin, G.: Bounded Queries in Recursion Theory. Birkhäuser, Basel (1998)
Gold, E.M.: Language identification in the limit. Information and Control 10, 447–474 (1967)
Gasarch, W., Pleszkoch, M.: Learning via queries to an oracle. In: Rivest, R., Haussler, D., Warmuth, M. (eds.) Proceedings of the Second Annual Workshop on Computational Learning Theory, pp. 214–229. Morgan Kaufmann, San Francisco (1989)
Hirsh-Pasek, K., Treiman, R., Schneiderman, M.: Brown and Hanlon revisited: Mothers’ sensitivity to ungrammatical forms. Journal of Child Language 11, 81–88 (1984)
Jain, S., Osherson, D., Royer, J., Sharma, A.: Systems that Learn: An Introduction to Learning Theory, 2nd edn. MIT Press, Cambridge (1999)
Lange, S., Nessel, J., Zilles, S.: Learning languages with queries. In: Proceedings of Treffen der GI-Fachgruppe Maschinelles Lernen (FGML), Learning Lab Lower Saxony, Hannover, Germany, pp. 92–99 (2002)
Motoki, T.: Inductive inference from all positive and some negative data (1992) (unpublished manuscript)
Rogers, H.: Theory of Recursive Functions and Effective Computability. McGraw-Hill, New York (1967); Reprinted by MIT Press in 1987
Shinohara, T.: Studies on Inductive Inference from Positive Data. PhD thesis, Kyushu University, Kyushu, Japan (1986)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jain, S., Kinber, E. (2004). Learning Languages from Positive Data and Negative Counterexamples. In: Ben-David, S., Case, J., Maruoka, A. (eds) Algorithmic Learning Theory. ALT 2004. Lecture Notes in Computer Science(), vol 3244. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30215-5_6
Download citation
DOI: https://doi.org/10.1007/978-3-540-30215-5_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23356-5
Online ISBN: 978-3-540-30215-5
eBook Packages: Springer Book Archive