Abstract
The regularization functional induced by the graph Laplacian of a random neighborhood graph based on the data is adaptive in two ways. First it adapts to an underlying manifold structure and second to the density of the data-generating probability measure. We identify in this paper the limit of the regularizer and show uniform convergence over the space of Hölder functions. As an intermediate step we derive upper bounds on the covering numbers of Hölder functions on compact Riemannian manifolds, which are of independent interest for the theoretical analysis of manifold-based learning methods.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comp. 15(6), 1373–1396 (2003)
Belkin, M., Niyogi, P.: Semi-supervised learning on manifolds. Machine Learning 56, 209–239 (2004)
Belkin, M., Niyogi, P.: Towards a theoretical foundation for laplacian-based manifold methods. In: Auer, P., Meir, R. (eds.) COLT 2005. LNCS, vol. 3559, pp. 486–500. Springer, Heidelberg (2005)
Bousquet, O., Chapelle, O., Hein, M.: Measure based regularization. In: Thrun, S., Saul, L., Schölkopf, B. (eds.) Adv. in Neur. Inf. Proc. Syst. (NIPS), vol. 16, MIT Press, Cambridge (2004)
Canu, S., Elisseeff, A.: Regularization, kernels and sigmoid net (unpublished, 1999)
Coifman, S., Lafon, S.: Diffusion maps. Appl. and Comp. Harm. Anal. (January 2005) (preprint) (to appear)
Hebey, E.: Nonlinear analysis on manifolds: Sobolev spaces and inequalities. Courant Institute of Mathematical Sciences, New York (1998)
Hein, M.: Geometrical aspects of statistical learning theory. PhD thesis, MPI für biologische Kybernetik/Technische Universität Darmstadt (2005), http://www.kyb.mpg.de/publication.html?user=mh
Hein, M., Audibert, J.-Y., von Luxburg, U.: From graphs to manifolds – weak and strong pointwise consistency of graph laplacians. In: Auer, P., Meir, R. (eds.) COLT 2005. LNCS, vol. 3559, pp. 470–485. Springer, Heidelberg (2005)
Hendriks, H., Janssen, J.H.M., Ruymgaart, F.H.: Strong uniform convergence of density estimators on compact Euclidean manifolds. Statist. Prob. Lett. 16, 305–311 (1993)
Hoeffding, W.: Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58, 13–30 (1963)
Kolmogorov, A.N., Tihomirov, V.M.: ε-entropy and ε-capacity of sets in functional spaces. Amer. Math. Soc. Transl. 17, 277–364 (1961)
Schick, T.: Manifolds with boundary of bounded geometry. Math. Nachr. 223, 103–120 (2001)
van der Vaart, A.W., Wellner, J.A.: Weak convergence and empirical processes, 2nd edn. Springer, New-York (2001)
Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schölkopf, B.: Learning with local and global consistency. In: Thrun, S., Saul, L., Schölkopf, B. (eds.) Adv. in Neur. Inf. Proc. Syst. (NIPS), vol. 16. MIT Press, Cambridge (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hein, M. (2006). Uniform Convergence of Adaptive Graph-Based Regularization. In: Lugosi, G., Simon, H.U. (eds) Learning Theory. COLT 2006. Lecture Notes in Computer Science(), vol 4005. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11776420_7
Download citation
DOI: https://doi.org/10.1007/11776420_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-35294-5
Online ISBN: 978-3-540-35296-9
eBook Packages: Computer ScienceComputer Science (R0)