Abstract
The contraction inequality for Rademacher averages is extended to Lipschitz functions with vector-valued domains, and it is also shown that in the bounding expression the Rademacher variables can be replaced by arbitrary iid symmetric and sub-gaussian variables. Example applications are given for multi-category learning, K-means clustering and learning-to-learn.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bartlett, P.L., Mendelson, S.: Rademacher and Gaussian complexities: risk bounds and structural results. J. Mach. Learn. Res. 3, 463–482 (2002)
Baxter, J.: A model of inductive bias learning. J. Artif. Intell. Res. 12, 149–198 (2000)
Biau, G., Devroye, L., Lugosi, G.: On the performance of clustering in Hilbert spaces. IEEE Trans. Inf. Theory 54(2), 781–790 (2008)
Boucheron, S., Lugosi, G., Massart, P.: Concentration Inequalities. Oxford University Press, Oxford (2013)
Caponnetto, A., De Vito, E.: Optimal rates for regularized least-squares algorithm. Found. Comput. Math. 7, 331–368 (2007)
Chapelle, O., Wu, M.: Gradient descent optimization of smoothed information retrieval metrics. Inf. Retr. 13(3), 216–235 (2010)
Chaudhuri, S., Tewari, A.: Generalization bounds for learning to rank: does the length of document lists matter? In: ICML 2015 (2015)
Ciliberto, C., Poggio, T., Rosasco, L.: Convex learning of multiple tasks and their structure (2015). arXiv preprint: arXiv:1504.03101
Crammer, K., Singer, Y.: On the algorithmic implementation of multiclass kernel-based vector machines. J. Mach. Learn. Res. 2, 265–292 (2002)
Kakade, S.M., Shalev-Shwartz, S., Tewari, A.: Regularization techniques for learning with matrices. J. Mach. Learn. Res. 13, 1865–1890 (2012)
Koltchinskii, V., Panchenko, D.: Empirical margin distributions and bounding the generalization error of combined classifiers. Ann. Stat. 30(1), 1–50 (2002)
Ledoux, M., Talagrand, M.: Probability in Banach Spaces: Isoperimetry and Processes. Springer, Berlin (1991)
Lei, Y., Dogan, U., Binder, A., Kloft, M.: Multi-class SVMs: from tighter data-dependent generalization bounds to novel algorithms. In: Advances in Neural Information Processing Systems, pp. 2026–2034 (2015)
Maurer, A.: Transfer bounds for linear feature learning. Mach. Learn. 75(3), 327–350 (2009)
Maurer, A., Pontil, M.: K-dimensional coding schemes in Hilbert spaces. IEEE Trans. Inf. Theory 56(11), 5839–5846 (2010)
Maurer, A., Pontil, M., Romera-Paredes, B.: The benefit of multitask representation learning. J. Mach. Learn. Res. 17(81), 1–32 (2016)
McDonald, D.J., Shalizi, C.R., Schervish, M.: Generalization error bounds for stationary autoregressive models (2011). arXiv preprint: arXiv:1103.0942
Meir, R., Zhang, T.: Generalization error bounds for Bayesian mixture algorithms. J. Mach. Learn. Res. 4, 839–860 (2003)
Michelli, C.A., Pontil, M.: On learning vector-valued functions. J. Mach. Learn. Res. 6, 615–637 (2005)
Mroueh, Y., Poggio, T., Rosasco, L., Slotine, J.J.: Multiclass learning with simplex coding. In: Advances in Neural Information Processing Systems, pp. 2789–2797 (2012)
Slepian, D.: The one-sided barrier problem for Gaussian noise. Bell Syst. Tech. J. 41, 463–501 (1962)
Szarek, S.: On the best constants in the Khintchine inequality. Stud. Math. 58, 197–208 (1976)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Maurer, A. (2016). A Vector-Contraction Inequality for Rademacher Complexities. In: Ortner, R., Simon, H., Zilles, S. (eds) Algorithmic Learning Theory. ALT 2016. Lecture Notes in Computer Science(), vol 9925. Springer, Cham. https://doi.org/10.1007/978-3-319-46379-7_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-46379-7_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-46378-0
Online ISBN: 978-3-319-46379-7
eBook Packages: Computer ScienceComputer Science (R0)