Abstract
It has been recently shown that sharp generalization bounds can be obtained when the function class from which the algorithm choo-ses its hypotheses is “small” in the sense that the Rademacher averages of this function class are small [8,9]. Seemingly based on different arguments, generalization bounds were obtained in the compression scheme [7], luckiness [13], and algorithmic luckiness [6] frameworks in which the “size” of the function class is not specified a priori.
We show that the bounds obtained in all these frameworks follow from the same general principle, namely that coordinate projections of this function subclass evaluated on random samples are “small” with high probability.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bartlett, P.L., Bousquet, O., Mendelson, S.: Local Rademacher Complexities (preprint)
Cannon, A.H., Ettinger, J.M., Hush, D.R., Scovel, J.C.: Machine learning with data dependent hypothesis classes. JMLR 2, 335–358 (2002)
Floyd, S., Warmuth, M.: Sample compression, learnability, and the Vapnik- Chervonenkis dimension. Machine Learning 21(3), 269–304 (1995)
Gat, Y.: A bound concerning the generalization ability of a certain class of learning algorithms, Tech. Rep. No. 548, Univ. of California, Berkeley (March 1999)
Graepel, T., Herbrich, R., Shawe-Taylor, J.: Generalisation Error Bounds for Sparse Linear Classifiers. In: Proc. 13th Annu. Conf. on Comput. Learning Theory (2000)
Herbrich, R., Williamson, R.C.: Algorithmic luckiness. JMLR 3, 175–212 (2002)
Littlestone, N., Warmuth, M.: Relating Data Compression and Learnability. Univ. of California, Santa Cruz (1986) (unpublished manuscript)
Mendelson, S.: Rademacher averages and phase transitions in Glivenko-Cantelli class. IEEE Trans. on Inform. Th. 48(1), 251–263 (2002)
Mendelson, S.: Improving the sample complexity using global data. IEEE Trans. on Inform. Th. 48(7), 1977–1991 (2002)
Mendelson, S.: A few notes on Statistical Learning Theory. In: Mendelson, S., Smola, A.J. (eds.) Advanced Lectures on Machine Learning. LNCS, vol. 2600, pp. 1–40. Springer, Heidelberg (2003)
Mendelson, S., Philips, P.: On the importance of small coordinate projections. JMLR (to appear)
McDiarmid, C.: On the method of bounded differences. In: Surveys in Combinatorics. London Math. Soc. Lect. Note Series, vol. 141, pp. 148–188 (1989)
Shawe-Taylor, J., Bartlett, P.L., Williamson, R.C., Anthony, M.: Structural risk minimization over data-dependent hierarchies. IEEE Trans. on Inform. Th. 44(5), 1926–1940 (1998)
Talagrand, M.: Majorizing measures: The generic chaining. Ann. Probab. 24, 1049–1103 (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mendelson, S., Philips, P. (2003). Random Subclass Bounds. In: Schölkopf, B., Warmuth, M.K. (eds) Learning Theory and Kernel Machines. Lecture Notes in Computer Science(), vol 2777. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45167-9_25
Download citation
DOI: https://doi.org/10.1007/978-3-540-45167-9_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40720-1
Online ISBN: 978-3-540-45167-9
eBook Packages: Springer Book Archive