Nonparametric Regression Estimation

  • L. Györfi
  • M. Kohler
Part of the International Centre for Mechanical Sciences book series (CISM, volume 434)


The basic aim of mathematical statistics is to learn a probability law or its characteristics from data.


Regression Function Nonparametric Regression Piecewise Polynomial Spline Space Covering Number 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Antos, A., Györfi, L. and Kohler, M. (2000). Lower bounds on the rate of convergence of nonparametric regression estimates. J. Statistical Planning and Inference, 83, pp. 91–100.CrossRefMATHGoogle Scholar
  2. Barron, A. R., Birgé, L. and Massart, P. (1999). Risk bounds for model selection via penalization. Probability Theory and Related Fields, 113, pp. 301–413.MathSciNetCrossRefMATHGoogle Scholar
  3. Beirlant, J. and Györfi, L. (1998). On the asymptotic L2-error in partitioning regression estimation. J. Statistical Planning and Inference, 71, pp. 93–107.CrossRefMATHGoogle Scholar
  4. Birgé, L. (1986). On estimating a density using Hellinger distance and some other strange facts. Probability Theory and Related Fields, 71, pp. 271–291.MathSciNetCrossRefMATHGoogle Scholar
  5. Ibragimov, I. A. and Khasminskii, R. Z. (1980). On nonparametric estimation of regression. Doklady Acad. Nauk SSSR 252, 780–784.MathSciNetGoogle Scholar
  6. de Boor, C. (1978). A Practical Guide to Splines. Springer, New York.CrossRefMATHGoogle Scholar
  7. Devroye, L. and Györfi, L. (1983). Distribution-free exponential upper bound on the L1 error of partitioning estimates of a regression function. In Proceedings of the Fourth Pannonian Symposium on Mathematical Statistics, Konecny F., Mogyor6di, J. and Wetz, W. Eds., pp. 67–76, Budapest, Akadémiai Kiadô.Google Scholar
  8. Devroye, L., Györfi, L. and Lugosi, G. (1996). A Probabilistic Theory of Pattern Recognition. Springer Verlag.Google Scholar
  9. Devroye, L., Györfi, L., Krzyzak, A. and Lugosi, G. (1994). On the strong universal consistency of nearest neighbor regression function estimates. Annals of Statistics, 22, pp. 13711385.Google Scholar
  10. Devroye, L. and Krzyzak, A. (1989). An equivalence theorem for L1 convergence of the kernel regression estimate. Journal of Statistical Planning and Inference, 23, pp. 71–82.MathSciNetCrossRefMATHGoogle Scholar
  11. Devroye, L. and Wagner, T. J. (1980). Distribution-free consistency results in nonparamet- ric discrimination and regression function estimation. Annals of Statistics, 8, pp. 231–239.MathSciNetCrossRefMATHGoogle Scholar
  12. Dudley, R. (1978). Central limit theorems for empirical measures. Annals of Probability 6 899–929.MathSciNetCrossRefMATHGoogle Scholar
  13. Györfi, L. (1991). Universal consistencies of regression estimate for unbounded regression functions. in Nonparametric functional estimation and related topics, ed. G. Roussas, pp. 329–338. Dordrecht: Kluwer Academic Publishers.Google Scholar
  14. Györfi, L., Kohler, M., Krzyzak, A. and Walk, H. (2002). A Distribution-Free Theory of Nonparametric Regression. Springer, (book under preparation )Google Scholar
  15. Györfi, L., Kohler, M. and Walk, H. (1998). Weak and strong universal consistency of semi-recursive partitioning and kernel regression estimates. Statistics and Decisions, 16, pp. 1–18.MathSciNetMATHGoogle Scholar
  16. Györfi, L. and Walk, H. (1996). On the strong universal consistency of a series type regression estimate. Mathematical Methods in Statistics, 5, pp. 332–342.MATHGoogle Scholar
  17. Györfi, L. and Walk, H. (1997). Consistency of a recursive regression estimate by Pal Révész. Statistics and Probability Letters, 31, pp. 177–183.MathSciNetCrossRefMATHGoogle Scholar
  18. Tukey, J. W. (1947). Nonparametric estimation II. Statistically equivalent blocks and tolerance regions. Annals of Mathematical Statistics, 18, pp. 529–539.MathSciNetCrossRefMATHGoogle Scholar
  19. Ibragimov, I. A. and Khasminskii, R. Z. (1981). Statistical Estimation: Asymptotic Theory. Springer Verlag, New York.MATHGoogle Scholar
  20. Ibragimov, I. A. and Khasminskii, R. Z. (1982). On the bounds for quality of nonparametric regression function estimation. Theory of Probability and its Applications 27, 81–94.MathSciNetCrossRefMATHGoogle Scholar
  21. Kohler, M. (2000). Inequalities for uniform deviations of averages from expectations with applications to nonparamtric regression. Journal of Statistical Planning and Inference 89, 1–23.MathSciNetCrossRefMATHGoogle Scholar
  22. Kohler, M. and Krzyzak, A. (2001). Nonparametric regression estimation using penalized least squares. To appear in IEEE Trans. Inf. Theory.Google Scholar
  23. Korostelev, A. P. and Tsybakov, A. B. (1993). Minimax Theory of Image Reconstruction. Springer Verlag, Berlin.MATHGoogle Scholar
  24. Lee, W. S., Bartlett, R L., Williamson, R. C. (1996). Efficient agnostic learning of neural networks with bounded fan—in. IEEE Trans. Inform. Theory 42, 2118–2132.MathSciNetCrossRefMATHGoogle Scholar
  25. Nadaraya, E. A. (1964). On estimating regression. Theory of Probability and its Applications, 9, pp. 141–142.CrossRefGoogle Scholar
  26. Nadaraya, E. A. (1970). Remarks on nonparametric estimates for density functions and regression curves. Theory of Probability and its Applications, 15, pp. 134–137.CrossRefMATHGoogle Scholar
  27. Pollard, D. (1984). Convergence of Stochastic Processes. Springer, New York. Schumaker, L. (1981). Spline functions: Basic Theory. Wiley, New York.Google Scholar
  28. Spiegelman, C. and Sacks, J. (1980). Consistent window estimation in nonparametric regression. Annals of Statistics, 8, pp. 240–246.MathSciNetCrossRefMATHGoogle Scholar
  29. Steele, J. (1975) Combinatorial entropy and uniform limit laws. PhD Thesis, Stanford University, Stanford, CA.Google Scholar
  30. Stone, C. J. (1977). Consistent nonparametric regression. Annals of Statistics, 5, pp. 595645.Google Scholar
  31. Stone, C. J. (1982). Optimal global rates of convergence for nonparametric regression. Annals of Statistics 10, 1040–1053.MATHGoogle Scholar
  32. Tukey, J. W. (1961). Curves as parameters and touch estimation Proceedings of the Fourth Berkeley Symposium, pp. 681–694.Google Scholar
  33. van de Vaart, A. W. and Wellner, J. A. (1996). Weak convergence and empirical processes. Springer, New York.CrossRefMATHGoogle Scholar
  34. Vapnik, V. N. and Chervonenkis, A. Y. (1974). Theory of Pattern Recognition. Nauka, Moscow. (in Russian); German translation: Theorie der Zeichenerkennung, Akademie Verlag, Berlin, 1979.Google Scholar
  35. Vajda, I. (1989). Theory of Statistical Inference and Information. Kluwer Academic Publishers.Google Scholar
  36. van de Geer, S. (1987). A new approach to least squares estimation, with applications. Ann. Statist. 15, 587–602.MathSciNetCrossRefMATHGoogle Scholar
  37. van de Geer, S. (1988). Regression Analysis and Empirical Processes. CWI-tract 45, Centre for Mathematics and Computer Science, Amsterdam.Google Scholar
  38. van de Geer, S. (1990). Estimating a regression function. Ann. Statist. 18, 907–924.MathSciNetCrossRefMATHGoogle Scholar
  39. van de Geer, S. (2000). Empirical Processes in M-Estimation. Cambridge University Press, Cambridge.Google Scholar
  40. Walk, H. (2000). Strong universal consistency of kernel and partitioning regression esti-mates. (submitted for publications)Google Scholar
  41. Watson, G. S. (1964). Smooth regression analysis. Sankhya Series A, 26, pp. 359–372.MATHGoogle Scholar

Copyright information

© Springer-Verlag Wien 2002

Authors and Affiliations

  • L. Györfi
    • 1
  • M. Kohler
    • 2
  1. 1.Budapest University of Technology and EconomicsBudapestHungary
  2. 2.Universität StuttgartStuttgartGermany

Personalised recommendations