Strong Laws of Large Numbers and Nonparametric Estimation

  • Harro Walk


Elementary approaches to classic strong laws of large numbers use a monotonicity argument or a Tauberian argument of summability theory. Together with results on variance of sums of dependent random variables they allow to establish various strong laws of large numbers in case of dependence, especially under mixing conditions. Strong consistency of nonparametric regression estimates of local averaging type (kernel and nearest neighbor estimates), pointwise as well as in L 2, can be considered as a generalization of strong laws of large numbers. Both approaches can be used to establish strong universal consistency in the case of independence and, mostly by sharpened integrability assumptions, consistency under ρ-mixing or α-mixing. In a similar way Rosenblatt-Parzen kernel density estimates are treated.


Regression Estimate Nonparametric Estimation Strong Consistency Dependent Random Variable Real Random Variable 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Chandra, T.K., Goswami, A.: Cesàro uniform integrability and the strong law of large numbers. Sankhyā, Ser. A 54, 215–231 (1992) MathSciNetMATHGoogle Scholar
  2. Chandra, T.K., Goswami, A.: Corrigendum: Cesàro uniform integrability and the strong law of large numbers. Sankhyā, Ser. A 55, 327–328 (1993) MathSciNetGoogle Scholar
  3. Csörgő, S., Tandori, K., Totik, V.: On the strong law of large numbers for pairwise independent random variables. Acta Math. Hung. 42, 319–330 (1983) CrossRefGoogle Scholar
  4. Dehling, H., Philipp, W.: Almost sure invariance principles for weakly dependent vector-valued random variables. Ann. Probab. 10, 689–701 (1982) CrossRefMathSciNetMATHGoogle Scholar
  5. Devroye, L.: On the almost everywhere convergence of nonparametric regression function estimates. Ann. Stat. 9, 1310–1319 (1981) CrossRefMathSciNetMATHGoogle Scholar
  6. Devroye, L., Györfi, L.: Nonparametric Density Estimation: The L 1-View. Wiley, New York (1985) Google Scholar
  7. Devroye, L., Krzyżak, A.: An equivalence theorem for L 1 convergence of the kernel regression estimate. J. Stat. Plan. Inference 23, 71–82 (1989) CrossRefMATHGoogle Scholar
  8. Devroye, L., Wagner, T.J.: Distribution-free consistency results in nonparametric discrimination and regression function estimation. Ann. Stat. 8, 231–239 (1980a) CrossRefMathSciNetMATHGoogle Scholar
  9. Devroye, L., Wagner, T.J.: On the L 1-convergence of kernel estimators of regression functions with applications in discrimination. Z. Wahrscheinlichkeitstheor. Verw. Geb. 51, 15–25 (1980b) CrossRefMathSciNetMATHGoogle Scholar
  10. Devroye, L., Györfi, L., Krzyżak, A., Lugosi, G.: On the strong universal consistency of nearest neighbor regression function estimates. Ann. Stat. 22, 1371–1385 (1994) CrossRefMATHGoogle Scholar
  11. Doukhan, P.: Mixing: Properties and Examples. Springer, New York (1994) MATHGoogle Scholar
  12. Etemadi, N.: An elementary proof of the strong law of large numbers. Z. Wahrscheinlichkeitstheor. Verw. Geb. 55, 119–122 (1981) CrossRefMathSciNetMATHGoogle Scholar
  13. Etemadi, N.: On the law of large numbers for nonnegative random variables. J. Multivar. Anal. 13, 187–193 (1983) CrossRefMathSciNetMATHGoogle Scholar
  14. Ferrario, P.: The Strong law of large numbers and applications. Laurea Thesis, Politecnico di Milano (2004) Google Scholar
  15. Frey, S.: Konvergenzverhalten von Regressionsschätzungen unter Mischungsbedingungen. Wissenschaftliche Arbeit, Universität Stuttgart (2007) Google Scholar
  16. Gaposhkin, V.F.: Criteria for the strong law of large numbers for some classes of second-order stationary processes and homogeneous random fields. Theory Probab. Appl. 22, 286–310 (1977) CrossRefMATHGoogle Scholar
  17. Greblicki, W., Krzyżak, A., Pawlak, M.: Distribution-free pointwise consistency of kernel regression estimate. Ann. Stat. 12, 1570–1575 (1984) CrossRefMATHGoogle Scholar
  18. Györfi, L.: Universal consistencies of a regression estimate for unbounded regression functions. In: Roussas, G. (ed.) Nonparametric Functional Estimation and Related Topics. NATO ASI Series, pp. 329–338. Kluwer Academic, Dordrecht (1991) Google Scholar
  19. Györfi, L., Härdle, W., Sarda, P., Vieu, P.: Nonparametric Curve Estimation for Mixing Time Series. Springer, New York (1989) Google Scholar
  20. Györfi, L., Masry, E.: The L 1 and L 2 strong consistency of recursive kernel density estimation from dependent samples. IEEE Trans. Inf. Theory IT 36, 531–539 (1990) CrossRefGoogle Scholar
  21. Györfi, L., Kohler, M., Walk, H.: Weak and strong universal consistency of semi-recursive partitioning and kernel regression estimates. Stat. Decis. 16, 1–18 (1998) MATHGoogle Scholar
  22. Györfi, L., Kohler, M., Krzyżak, A., Walk, H.: A Distribution-Free Theory of Nonparametric Regression. Springer Series in Statistics. Springer, New York (2002) MATHGoogle Scholar
  23. Hardly, G.H.: Divergent Series. Oxford University Press, London (1949) Google Scholar
  24. Irle, A.: On consistency in nonparametric estimation under mixing conditions. J. Multivar. Anal. 60, 123–147 (1997) CrossRefMathSciNetMATHGoogle Scholar
  25. Karamata, J.: Einige Sätze über die Rieszschen Mittel. Acad. R. Serbe, Bull. Acad. Sci. Math. Nat.(Belgrade) A 4, 121–137 (1938) Google Scholar
  26. Korevaar, J.: Tauberian Theory: A Century of Developments. Springer, Berlin (2004) MATHGoogle Scholar
  27. Kozek, A.S., Leslie, J.R., Schuster, E.F.: On a univeral strong law of large numbers for conditional expectations. Bernoulli 4, 143–165 (1998) CrossRefMathSciNetMATHGoogle Scholar
  28. Landau, E.: Über die Bedeutung einiger neuer Grenzwertsätze der Herren Hardy und Axer. Pr. Mat.-Fiz. 21, 91–177 (1910) Google Scholar
  29. Liebscher, E.: Strong convergence of sums of α-mixing random variables with applications to density estimation. Stoch. Process. Appl. 65, 69–80 (1996) CrossRefMathSciNetMATHGoogle Scholar
  30. Peligrad, M.: Properties of uniform consistency of the kernel estimators of density and of regression functions under dependence assumptions. Stoch. Stoch. Rep. 40, 147–168 (1992) MathSciNetMATHGoogle Scholar
  31. Rhomari, N.: Approximation et inégalités exponentielles pour les sommes de vecteurs aléatoires dépendants. C. R. Acad. Sci. Paris, Ser. I 334, 149–154 (2002) MathSciNetMATHGoogle Scholar
  32. Rio, E.: Covariance inequalities for strongly mixing processes. Ann. Inst. H. Poincaré, Sect. B 29, 587–597 (1993) MathSciNetMATHGoogle Scholar
  33. Schmidt, R.: Über divergente Folgen und lineare Mittelbildungen. Math. Z. 22, 89–152 (1925) CrossRefMathSciNetGoogle Scholar
  34. Serfling, R.J.: Moment inequalities for the maximum cumulative sum. Ann. Math. Stat. 41, 1227–1234 (1970a) CrossRefMathSciNetMATHGoogle Scholar
  35. Serfling, R.J.: Convergence properties of S n under moment restrictions. Ann. Math. Stat. 41, 1235–1248 (1970b) CrossRefMathSciNetMATHGoogle Scholar
  36. Spiegelman, C., Sacks, J.: Consistent window estimation in nonparametric regression. Ann. Stat. 8, 240–246 (1980) CrossRefMathSciNetMATHGoogle Scholar
  37. Steele, J.M.: An Efron-Stein inequality for nonsymmetric statistics. Ann. Stat. 14, 753–758 (1986) CrossRefMathSciNetMATHGoogle Scholar
  38. Stone, C.J.: Consistent nonparametric regression. Ann. Stat. 5, 595–645 (1977) CrossRefMATHGoogle Scholar
  39. Szász, O.: Verallgemeinerung und neuer Beweis einiger Sätze Tauberscher Art. Sitzungsber. Bayer. Akad. Wiss. München, Math.-Phys. Kl. 59, 325–340 (1929) MATHGoogle Scholar
  40. Walk, H.: Strong universal pointwise consistency of recursive regression estimates. Ann. Inst. Stat. Math. 53, 691–707 (2001) CrossRefMathSciNetMATHGoogle Scholar
  41. Walk, H.: Strong universal consistency of smooth kernel regression estimates. Ann. Inst. Stat. Math. 57, 665–685 (2005a) CrossRefMathSciNetMATHGoogle Scholar
  42. Walk, H.: Strong laws of large numbers by elementary Tauberian arguments. Monatshefte Math. 144, 329–346 (2005b) CrossRefMathSciNetMATHGoogle Scholar
  43. Walk, H.: Almost sure Cesàro and Euler summability of sequences of dependent random variables. Arch. Math. 89, 466–480 (2007) CrossRefMathSciNetMATHGoogle Scholar
  44. Walk, H.: A universal strong law of large numbers for conditional expectations via nearest neighbors. J. Multivar. Anal. 99, 1035–1050 (2008a) CrossRefMathSciNetMATHGoogle Scholar
  45. Walk, H.: Strong consistency of kernel estimates under dependence. Stat. Probab. Lett. (2008b). doi: 10.1016/j.spl.2010.03.010 Google Scholar
  46. Zeller, K., Beekmann, W.: Theorie der Limitierungsverfahren, 2 Aufl. Springer, Berlin (1970) MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  1. 1.Department of MathematicsUniversität StuttgartStuttgartGermany

Personalised recommendations