Recursive Teaching Dimension, Learning Complexity, and Maximum Classes
- 843 Downloads
This paper is concerned with the combinatorial structure of concept classes that can be learned from a small number of examples. We show that the recently introduced notion of recursive teaching dimension (RTD, reflecting the complexity of teaching a concept class) is a relevant parameter in this context. Comparing the RTD to self-directed learning, we establish new lower bounds on the query complexity for a variety of query learning models and thus connect teaching to query learning.
For many general cases, the RTD is upper-bounded by the VC-dimension, e.g., classes of VC-dimension 1, (nested differences of) intersection-closed classes, “standard” boolean function classes, and finite maximum classes. The RTD thus is the first model to connect teaching to the VC-dimension.
The combinatorial structure defined by the RTD has a remarkable resemblance to the structure exploited by sample compression schemes and hence connects teaching to sample compression. Sequences of teaching sets defining the RTD coincide with unlabeled compression schemes both (i) resulting from Rubinstein and Rubinstein’s corner-peeling and (ii) resulting from Kuzmin and Warmuth’s Tail Matching algorithm.
KeywordsConcept Class Query Complexity Combinatorial Structure Compression Scheme Target Concept
Unable to display preview. Download preview PDF.
- 1.Angluin, D.: Queries and concept learning. Mach. Learn. 2, 319–342 (1988)Google Scholar
- 5.Floyd, S., Warmuth, M.: Sample compression, learnability, and the vapnik-chervonenkis dimension. Mach. Learn. 21(3), 269–304 (1995)Google Scholar
- 9.Helmbold, D., Sloan, R., Warmuth, M.: Learning nested differences of intersection-closed concept classes. Mach. Learn. 5, 165–196 (1990)Google Scholar
- 10.Jackson, J., Tomkins, A.: A computational model of teaching. In: 5th Annl. Workshop on Computational Learning Theory, pp. 319–326 (1992)Google Scholar
- 13.Littlestone, N.: Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Mach. Learn. 2(4), 285–318 (1988)Google Scholar
- 14.Littlestone, N., Warmuth, M.: Relating data compression and learnability. Technical report, UC Santa Cruz (1986)Google Scholar
- 16.Natarajan, B.: On learning boolean functions. In: 19th Annl. Symp. Theory of Computing, pp. 296–304 (1987)Google Scholar
- 17.Rubinstein, B., Rubinstein, J.: A geometric approach to sample compression (2009) (unpublished manuscript)Google Scholar
- 21.Welzl, E.: Complete range spaces (1987) (unpublished notes)Google Scholar
- 22.Zilles, S., Lange, S., Holte, R., Zinkevich, M.: Teaching dimensions based on cooperative learning. In: 21st Annl. Conf. Learning Theory, pp. 135–146 (2008)Google Scholar