Abstract
We develop a new error bound for transductive learning algorithms. The slack term in the new bound is a function of a relaxed notion of transductive stability, which measures the sensitivity of the algorithm to most pairwise exchanges of training and test set points. Our bound is based on a novel concentration inequality for symmetric functions of permutations. We also present a simple sampling technique that can estimate, with high probability, the weak stability of transductive learning algorithms with respect to a given dataset. We demonstrate the usefulness of our estimation technique on a well known transductive learning algorithm.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Azuma, K.: Weighted sums of certain dependent random variables. Tohoku Mathematical Journal 19, 357–367 (1967)
Belkin, M., Matveeva, I., Niyogi, P.: Regularization and semi-supervised learning on large graphs. In: Shawe-Taylor, J., Singer, Y. (eds.) COLT 2004. LNCS, vol. 3120, pp. 624–638. Springer, Heidelberg (2004)
Blum, A., Langford, J.: PAC-MDL bounds. In: Schölkopf, B., Warmuth, M.K. (eds.) COLT/Kernel 2003. LNCS, vol. 2777, pp. 344–357. Springer, Heidelberg (2003)
Bousquet, O., Elisseeff, A.: Stability and generalization. Journal of Machine Learning Research 2, 499–526 (2002)
Derbeko, P., El-Yaniv, R., Meir, R.: Explicit learning curves for transduction and application to clustering and compression algorithms. Journal of Artificial Intelligence Research 22, 117–142 (2004)
Devroye, L., Györfi, L., Lugosi, G.: A Probabilistic Theory of Pattern Recognition. Springer, New York (1996)
Grimmett, G.R., Stirzaker, D.R.: Probability and Random Processes, 2nd edn. Oxford Science Publications (1995)
Huang, T.M., Kecman, V.: Performance comparisons of semi-supervised learning algorithms. In: ICML Workshop, Learning with Partially Classified Training Data, pp. 45–49 (2005)
Hush, D., Scovel, C., Steinwart, I.: Stability of unstable learning algorithms. Technical Report LA-UR-03-4845, Los Alamos National Laboratory (2003)
Kearns, M., Ron, D.: Algorithmic stability and sanity-check bounds for leave-one-out cross-validation. Neural Computation 11(6), 1427–1453 (1999)
Kutin, S.: Extensions to McDiarmid’s inequality when differences are bounded with high probability. Technical Report TR-2002-04, University of Chicago (2002)
Kutin, S., Niyogi, P.: Almost-everywhere algorithmic stability and generalization error. In: UAI, pp. 275–282 (2002)
Ledoux, M.: The concentration of measure phenomenon. Mathematical Surveys and Monographs, vol. 89. American Mathematical Society (2001)
Manku, G.S., Rajagopalan, S., Lindsay, B.G.: Approximate medians and other quantiles in one pass and with limited memory. In: SIGMOD, vol. 28, pp. 426–435 (1998)
Mukherjee, S., Niyogi, P., Poggio, T., Rifkin, R.: Statistical learning: Stability is sufficient for generalization and necessary and sufficient for consistency of empirical risk minimization. Technical Report AI Memo 2002–024, MIT (2004)
Rakhlin, A., Mukherjee, S., Poggio, T.: Stability results in learning theory. Analysis and Applications 3(4), 397–419 (2005)
Talagrand, M.: Majorizing measures: the generic chaining. Springer, Heidelberg (2005)
Vapnik, V.N.: Estimation of Dependences Based on Empirical Data. Springer Verlag, New York (1982)
Vapnik, V.N.: Statistical Learning Theory. Wiley Interscience, New York (1998)
Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Scholkopf, B.: Learning with local and global consistency. In: NIPS, pp. 321–328 (2003)
Zhu, X., Ghahramani, Z., Lafferty, J.D.: Semi-supervised learning using gaussian fields and harmonic functions. In: ICML, pp. 912–919 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
El-Yaniv, R., Pechyony, D. (2006). Stable Transductive Learning. In: Lugosi, G., Simon, H.U. (eds) Learning Theory. COLT 2006. Lecture Notes in Computer Science(), vol 4005. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11776420_6
Download citation
DOI: https://doi.org/10.1007/11776420_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-35294-5
Online ISBN: 978-3-540-35296-9
eBook Packages: Computer ScienceComputer Science (R0)