Abstract
This paper studies efficient learning with respect to mind changes. Our starting point is the idea that a learner that is efficient with respect to mind changes minimizes mind changes not only globally in the entire learning problem, but also locally in subproblems after receiving some evidence. Formalizing this idea leads to the notion of uniform mind change optimality. We characterize the structure of language classes that can be identified with at most α mind changes by some learner (not necessarily effective): A language class \({\mathcal L}\) is identifiable with α mind changes iff the accumulation order of \({\mathcal L}\) is at most α. Accumulation order is a classic concept from point-set topology. To aid the construction of learning algorithms, we show that the characteristic property of uniformly mind change optimal learners is that they output conjectures (languages) with maximal accumulation order. We illustrate the theory by describing mind change optimal learners for various problems such as identifying linear subspaces and one-variable patterns.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ambainis, A., Jain, S., Sharma, A.: Ordinal mind change complexity of language identification. Theor. Comput. Sci. 220(2), 323–343 (1999)
Angluin, D.: Finding patterns common to a set of strings. J. Comput. Syst. Sci. 21(1), 46–62 (1980)
Angluin, D.: Inductive inference of formal languages from positive data. Information and Control 45(2), 117–135 (1980)
Apsitis, K.: Derived sets and inductive inference. In: Arikawa, S., Jantke, K.P. (eds.) AII 1994 and ALT 1994. LNCS, vol. 872, pp. 26–39. Springer, Heidelberg (1994)
Baliga, G., Case, J., Jain, S.: The synthesis of language learners. Information and Computation 152, 16–43 (1999)
Cantor, G.: Grundlagen einer allgemeinen Mannigfaltigkeitslehre. In: Ewald, W. (ed.) From Kant to Hilbert, vol. 2, pp. 878–920. Oxford Science Publications (1996)
Freivalds, R., Kinber, E., Smith, C.H.: On the intrinsic complexity of learning. Inf. Comput. 123(1), 64–71 (1995)
Freivalds, R., Smith, C.H.: On the role of procrastination in machine learning. Inf. Comput. 107(2), 237–271 (1993)
Gold, E.M.: Language identification in the limit. Information and Control 10(5), 447–474 (1967)
Jain, S., Osherson, D., Royer, J.S., Sharma, A.: Systems That Learn, 2nd edn. MIT Press, Cambridge (1999)
Jain, S., Sharma, A.: The intrinsic complexity of language identification. J. Comput. Syst. Sci. 52(3), 393–402 (1996)
Jain, S., Sharma, A.: Mind change complexity of learning logic programs. TCS 284(1), 143–160 (2002)
Jayanthan, A.J.: Derived length for arbitrary topological spaces. International Journal of Mathematics and Mathematical Sciences 15(2), 273–277 (1992)
Kelly, K.: The Logic of Reliable Inquiry. Oxford University Press, Oxford (1996)
Kelly, K.: Efficient convergence implies Ockham’s Razor. In: Proceedings of the 2002 International Workshop on Computation Models of Scientific Reasoning and Applications, pp. 24–27 (2002)
Kelly, K.: Justification as truth-finding efficiency: How ockham’s razor works. Minds and Machines 14(4), 485–505 (2004)
Kocabas, S.: Conflict resolution as discovery in particle physics. Machine Learning 6, 277–309 (1991)
Kuratowski, K.: Topology, vol. 1. Academic Press, London (1966); Translated by J. Jaworowski
Lange, S., Zeugmann, T.: Language learning with a bounded number of mind changes. In: Symposium on Theoretical Aspects of Computer Science, pp. 682–691 (1993)
Martin, E., Osherson, D.N.: Elements of Scientific Inquiry. The MIT Press, Cambridge (1998)
Martin, E., Sharma, A., Stephan, F.: Learning, logic, and topology in a common framework. In: Cesa-Bianchi, N., Numao, M., Reischuk, R. (eds.) ALT 2002. LNCS (LNAI), vol. 2533, pp. 248–262. Springer, Heidelberg (2002)
Motoki, T., Shinohara, T., Wright, K.: The correct definition of finite elasticity: corrigendum to identification of unions. In: Proceedings of COLT 1991, p. 375. Morgan Kaufmann, San Francisco (1991)
Mukouchi, Y.: Inductive inference with bounded mind changes. In: Doshita, S., Nishida, T., Furukawa, K., Jantke, K.P. (eds.) ALT 1992. LNCS, vol. 743, pp. 125–134. Springer, Heidelberg (1993)
Odifreddi, P.: Classical Recursion Theory. North-Holland, Amsterdam (1999)
Osherson, D.N., Stob, M., Weinstein, S.: Systems that learn: an introduction to learning theory for cognitive and computer scientists. MIT Press, Cambridge (1986)
Schulte, O.: Automated discovery of conservation principles and new particles in particle physics. Manuscript submitted to Machine Learning (2005)
Valdés-Pérez, R.: Algebraic reasoning about reactions: Discovery of conserved properties in particle physics. Machine Learning 17, 47–67 (1994)
Valdés-Pérez, R.: On the justification of multiple selection rules of conservation in particle physics phenomenology. Computer Physics Communications 94, 25–30 (1996)
Wright, K.: Identification of unions of languages drawn from an identifiable class. In: Proceedings of the second annual workshop on Computational learning theory, pp. 328–333. Morgan Kaufmann Publishers Inc., San Francisco (1989)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Luo, W., Schulte, O. (2005). Mind Change Efficient Learning. In: Auer, P., Meir, R. (eds) Learning Theory. COLT 2005. Lecture Notes in Computer Science(), vol 3559. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11503415_27
Download citation
DOI: https://doi.org/10.1007/11503415_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-26556-6
Online ISBN: 978-3-540-31892-7
eBook Packages: Computer ScienceComputer Science (R0)