Almost Sure Convergence Properties of Nadaraya-Watson Regression Estimates

  • Harro Walk
Part of the International Series in Operations Research & Management Science book series (ISOR, volume 46)


For Nadaraya-Watson regression estimates with window kernel self-contained proofs of strong universal consistency for special bandwidths and of the corresponding Cesàro summability for general bandwidths are given.


Modeling Uncertainty Convergence Property Nonparametric Regression General Kernel Exponential Inequality 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Aizerman, M.A., E. M. Braverman, and L. I. Rozonoer. (1964). The probability problem of pattern recognition learning and the method of potential functions. Automation and Remote Control 25, 1175–1190.zbMATHGoogle Scholar
  2. Bauer, H. (1991). Wahrscheinlichkeitstheorie, 4th ed. W. de Gruyter, Berlin.zbMATHGoogle Scholar
  3. Devroye, L. (1991). Exponential inequalities in nonparametric estimation. In: G. Roussas, Ed., Nonparametric Functional Estimation and Related Topics. NATO ASI Ser. C, Kluwer Acad. Publ., Dordrecht, 31–44.CrossRefGoogle Scholar
  4. Devroye, L., L. Gyöfi, A. Krzyzak, and G. Lugosi. (1994). On the strong universal consistency of nearest neighbor regression function estimates. Ann. Statist. 22, 1371–1385.MathSciNetCrossRefGoogle Scholar
  5. Devroye, L., L. Gyöfi, and G. Lugosi. (1996). A Probabilistic Theory of Pattern Recognition. Springer, New York, Berlin, Heidelberg.CrossRefGoogle Scholar
  6. Devroye, L. and A. Krzyzak. (1989). An equivalence theorem for L1 convergence of the kernel regression estimate. J. Statist. Plann. Inference 23, 71–82.MathSciNetCrossRefGoogle Scholar
  7. Devroye, L. and T. J. Wagner. (1980). Distribution-free consistency results in nonparametric discrimination and regression function estimation. Ann. Statist. 8, 231–239.MathSciNetCrossRefGoogle Scholar
  8. Devroye, L. and T. J. Wagner. (1980). On the L1 convergence of kernel estimators of regression functions with applications in discrimination. Z. Wahrscheinlichkeitstheorie Verw. Gebiete 51, 15–25.MathSciNetCrossRefGoogle Scholar
  9. Dunford, N. and J. Schwartz. (1958). Linear Operators, Part I. Interscience Publ., New York.zbMATHGoogle Scholar
  10. Efron, B. and C. Stein. (1981). The jackknife estimate of variance. Ann. Statist. 9, 586–596.MathSciNetCrossRefGoogle Scholar
  11. Etemadi, N. (1981). An elementary proof of the strong law of large numbers. Z. Wahrscheinlichkeitstheorie Verw. Gebiete 55, 119–122.MathSciNetCrossRefGoogle Scholar
  12. Gladyshev, E.G. (1965). On stochastic approximation. Theor. Probab. Appl. 10, 275–278.CrossRefGoogle Scholar
  13. Gyöfi, L. (1991). Universal consistencies of a regression estimate for unbounded regression functions. In: G. Roussas, Ed., Nonparametric Functional Estimation and Related Topics. NATO ASI Ser. C, Kluwer Acad. Publ., Dordrecht, 329–338.Google Scholar
  14. Gyöfi, L., G. Morvai, and S. Yakowitz. (1998). Limits to consistent on-line forecasting for ergodic time series. IEEE Trans. Information Theory, 44, 886–892.MathSciNetCrossRefGoogle Scholar
  15. Hädle, W. (1990). Applied Nonparametric Regression. Cambridge University Press, Cambridge.CrossRefGoogle Scholar
  16. Hardy, G. H. (1949). Divergent Series. Clarendon Press, Oxford.zbMATHGoogle Scholar
  17. Loève, M. (1977). Probability Theory II, 4th ed. Springer, New York, Heidelberg, Berlin.zbMATHGoogle Scholar
  18. MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations. In: J. Neyman, Ed., Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability. University of California Press, Berkeley and Los Angeles, 281–297.Google Scholar
  19. McDiarmid, C. (1989). On the method of bounded differences. In: Surveys in Combinatorics 1989. Cambridge University Press, Cambridge, 148–188.CrossRefGoogle Scholar
  20. Nadaraya, E.A. (1964). On estimating regression. Theory Probab. Appl. 9, 141–142.CrossRefGoogle Scholar
  21. Nobel, A.B. (1999). Limits to classification and regression estimation from ergodic processes. Ann. Statist. 27, 262–273.MathSciNetCrossRefGoogle Scholar
  22. Robbins, H. and D. Siegmund. (1971). A convergence theorem for nonnegative almost supermartingales and some applications. In: J.S. Rustagi, Ed., Optimizing Methods of Statistics. Academic Press, New York, 233–257.Google Scholar
  23. Spiegelman, C. and J. Sacks. (1980). Consistent window estimation in nonparametric regression. Ann. Statist. 8, 240–246.MathSciNetCrossRefGoogle Scholar
  24. Steele, J.M. (1986). An Efron-Stein inequality for nonsymmetric statistics. Ann. Statist. 14, 753–758.MathSciNetCrossRefGoogle Scholar
  25. Stone, C.J. (1977). Consistent nonparametric regression. Ann. Statist. 5, 595–645.MathSciNetCrossRefGoogle Scholar
  26. Van Ryzin, J. (1969). On strong consistency of density estimates. Ann. Math. Statist. 40, 1765–1772.MathSciNetCrossRefGoogle Scholar
  27. Walk, H. (1997). Strong universal consistency of kernel and partitioning regression estimates. Preprint 97-1, Math. Inst. A, Univ. Stuttgart.Google Scholar
  28. Watson, G.S. (1964). Smooth regression analysis. Ser. A 26, 359–372.MathSciNetzbMATHGoogle Scholar
  29. Wheeden, R.L. and A. Zygmund. (1977). Measure and Integral. Marcel Dekker, New York.CrossRefGoogle Scholar
  30. Yakowitz, S. and C. C. Heyde. (1997). Long-range dependent effects with implications for forecasting and queueing inference. Preprint.Google Scholar
  31. Zeller, K. and W. Beekmann. (1970). Theorie der Limitierungsverfahren, 2. Aufl. Springer, Berlin, Heidelberg, New York.CrossRefGoogle Scholar

Copyright information

© Springer Science + Business Media, Inc. 2002

Authors and Affiliations

  • Harro Walk
    • 1
  1. 1.Mathematisches Institut AUniversitä StuttgartStuttgartGermany

Personalised recommendations