Abstract
We give a bound on the expected reconstruction error for a general coding method where data in a Hilbert space are represented by finite dimensional coding vectors. The result can be specialized to K-means clustering, nonnegative matrix factorization and the sparse coding techniques introduced by Olshausen and Field.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bartlett, P.L., Mendelson, S.: Rademacher and Gaussian Complexities: Risk Bounds and Structural Results. Journal of Machine Learning Research 3, 463–482 (2002)
Bartlett, P., Linder, T., Lugosi, G.: The minimax distortion redundancy in empirical quantizer design. IEEE Transactions on Information Theory 44, 1802–1813 (1998)
Biau, G., Devroye, L., Lugosi, G.: On the performance of clustering in Hilbert spaces. IEEE Transactions on Information Theory 54, 781–790 (2008)
Cucker, F., Smale, S.: On the mathematical foundations of learning. Bulletin of the American Mathematical Society 39(1), 1–49 (2001)
Hoeffding, W.: Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association 58, 13–30 (1963)
Hoyer, P.O.: Non-negative matrix factorization with sparseness constraints. Journal of Machine Learning Research 5, 1457–1469 (2004)
Koltchinskii, V., Panchenko, D.: Empirical margin distributions and bounding the generalization error of combined classifiers. The Annals of Statistics 30(1), 1–50 (2002)
Ledoux, M., Talagrand, M.: Probability in Banach Spaces. Springer, Heidelberg (1991)
Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)
Li, S.Z., Hou, X., Zhang, H., Cheng, Q.: Learning spatially localized parts-based representations. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Hawaii, USA, vol. I, pp. 207–212 (2001)
McDiarmid, C.: Concentration. In: Probabilistic Methods of Algorithmic Discrete Mathematics, pp. 195–248. Springer, Berlin (1998)
Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996)
Shawe-Taylor, J., Williams, C.K.I., Cristianini, N., Kandola, J.S.: On the eigenspectrum of the Gram matrix and the generalization error of kernel-PCA. IEEE Transactions on Information Theory 51(7), 2510–2522 (2005)
Wigelius, O., Ambroladze, A., Shawe-Taylor, J.: Statistical analysis of clustering with applications (preprint, 2007)
Slepian, D.: The one-sided barrier problem for Gaussian noise. Bell System Tech. J. 41, 463–501 (1962)
van der Vaart, A.W., Wallner, J.A.: Weak Convergence and Empirical Processes. Springer, Heidelberg (1996)
Zwald, L., Bousquet, O., Blanchart, G.: Statistical properties of kernel principal component analysis. Machine Learning 66(2-3), 259–294 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Maurer, A., Pontil, M. (2008). Generalization Bounds for K-Dimensional Coding Schemes in Hilbert Spaces. In: Freund, Y., Györfi, L., Turán, G., Zeugmann, T. (eds) Algorithmic Learning Theory. ALT 2008. Lecture Notes in Computer Science(), vol 5254. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87987-9_11
Download citation
DOI: https://doi.org/10.1007/978-3-540-87987-9_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87986-2
Online ISBN: 978-3-540-87987-9
eBook Packages: Computer ScienceComputer Science (R0)