Bandwidth Selection for Kernel Regression: a Survey

Conference paper
Part of the Statistics and Computing book series (SCO)


This paper is concerned with nonparametric estimation of a regression function. The behaviour of kernel estimates depends on a smoothing parameter (i.e. the bandwidth). Bandwidth choice turns out to be of particular importance as well for practical use as to insure good asymptotic properties of the estimate. Various techniques have been proposed in the past ten last years to select optimal values of this parameter. This paper presents a survey on theoretical results concerned with bandwidth selection.

Key Words

Asymptotic optimality Bandwidth choice Bibliography Bootstrap Cross-validation Kernel regression Plug-in. 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Akaike, H. (1974). Statistical Predictor Identification. A. Inst. Statist. Math., 22, 203–217.Google Scholar
  2. Akaike, H. (1974). A new look at the statistical model identification. I.E.E.E. Trans. Auto. Control, 19, 716–723.MathSciNetCrossRefzbMATHGoogle Scholar
  3. Akaike, H. (1970). Statistical predictor identification. Ann. Inst. Statist. Math., 22, 203217.Google Scholar
  4. Altman, N. (1990). Kernel smoothing of data with correlated errors. J. Amer. Statist. Assoc., 85, 749–759.MathSciNetCrossRefGoogle Scholar
  5. Burman, P. and C Hen, K.W. (1989). Nonparametric estimation of regression function. Ann. Statist., 17, 1567–1596.MathSciNetCrossRefzbMATHGoogle Scholar
  6. Boularan, J., Ferre, L. and Vieu, P. (1991). Growth curves: a two-stage nonparametric approach. Preprint.Google Scholar
  7. Cao-Abad, R. (1991). Rate of convergence for the wild bootstrap in nonparametric regression. Ann. Statist., 19, 2226–2231.MathSciNetCrossRefzbMATHGoogle Scholar
  8. Chiu, S.T. (1989). Bandwidth selection for kernel estimate with correlated noise. Statist. Proba. Let., 8, 347–353.MathSciNetCrossRefzbMATHGoogle Scholar
  9. Chiu, S.T. (1990). On the asymptotic distribution of bandwidth estimates. Ann. of Statist., 18, 1696–1711.MathSciNetCrossRefzbMATHGoogle Scholar
  10. Chiu, C. and Marron, J.S. (1990). The negative correlation between data determined bandwidth and the optimal bandwidth. Statist. Proba. Let., 10, 173–180.MathSciNetCrossRefzbMATHGoogle Scholar
  11. Chu, C. and Marron, J.S. (1991a). Comparison of two bandwidth selectors for dependent errors. Ann. Statist., 19, 1906–1918.MathSciNetCrossRefzbMATHGoogle Scholar
  12. Chu, C. and Marron, J.S. (1991b). Choosing a kernel regression estimator. Statist. Science., 6, 404–436.MathSciNetCrossRefzbMATHGoogle Scholar
  13. Collomb, G. (1976). Estimation non paramétrique de la régression par la méthode du noyau. Thesis at Univ. P. Sabatier, Toulouse, France.Google Scholar
  14. Collomb, G. (1981). Estimation non paramétrique de la régression: revue bibliographique. Inter. Statist. Review, 49, 75–93.MathSciNetCrossRefzbMATHGoogle Scholar
  15. Collomb, G. (1985). Nonparametric regression an up-to-date bibliopgraphy. Statistics, 2, 309–324.MathSciNetCrossRefGoogle Scholar
  16. Collomb, G., Sarda, P. and Vieu, P. (1986). Weak pointwise consistency of the cross cvalidatory window estimate in non parametric regression. Comm. Math. Univ. Carlova, 26, 789–798.MathSciNetGoogle Scholar
  17. Diggle, P. and Marron, J.S. (1988) Equivalence of smoothing parameter selectors in density and intensity. J. Amer. Statist. Assoc., 83, 793–800.MathSciNetCrossRefzbMATHGoogle Scholar
  18. Eubank, R.L. and Schucany, W. (1990). Adaptive bandwidth choice for kernel regression. Preprint.Google Scholar
  19. Fan, J. (1992a). Design adaptive regression. J. Amer. Statist. Ass., 87, in print.Google Scholar
  20. Fan, J. (1992b). Variable bandwidth and local linear regression smoothers. Preprint. FAN, J. (1992c). Local linear regression smoothers. Preprint.Google Scholar
  21. Gasser, T. and Herrmann, E. (1991). Data-adaptive kernel estimation. in Nonparametric functional Estimation and related topics. Ed G.G. Roussas. Kluwer Academic Publishers, 6780.Google Scholar
  22. Gasser, T., Knew, A. and Kohler, W. (1991). A flexible and fast method for automatic smoothing. J. Amer. Statist. Assoc., 86, 643–653.MathSciNetCrossRefzbMATHGoogle Scholar
  23. Gasser, T. and M. Üller, H.G. (1984). Nonparametric estimation of regression functions and their derivatives by the kernel method. Scand. J. of Statist., 11, 171–185.zbMATHGoogle Scholar
  24. Goldstein, L. and Messer, K. (1990). Optimal plug-in estimators for nonparametric functional estimation. Preprint.Google Scholar
  25. Gonzalez-Manteiga, W. and Cadarso-Suarez, M. (1991). A generalized Kaplan Meier estimator with applications. Preprint.Google Scholar
  26. Györfi, L., Härdle, W., Sarda, P. and Vieu, P. (1989). Nonparametric curve estimation from time series. Springer, Lectures notes in Statistics, vol 60.zbMATHGoogle Scholar
  27. Hall, P. (1984a). Asymptotic properties of integrated squared errors and cross-validation for kernel estimation of a regression function. Z.f.W.u.v.G., 67, 175–196.zbMATHGoogle Scholar
  28. Hall, P. (1984b). Integrated squared errors properties of kernel estimation of a regression function. Ann. Statist., 12, 241–260.MathSciNetCrossRefzbMATHGoogle Scholar
  29. Hall, P. (1989). On convergence rates in nonparametric problems. Int. Statist. Rev., 57, 47–58.CrossRefGoogle Scholar
  30. Hall, P. and Jones, C. (1990). Adaptive M-estimation in nonparametric regression. Ann. Statist., B, 53, 245–252.Google Scholar
  31. Hall, P. and Marron, J.S. (1991). Local minima in Cross-Validation. J. Roy. Statist. Soc., B, 53, 245–252.MathSciNetzbMATHGoogle Scholar
  32. Härdle, W. (1990). Applied nonparametric regression. Oxford University Press, Boston. HÄRDLE, W. and BOWMAN, A. (1987). Bootstrapping in nonparametric regression. J.A.S.A., 83, 127–141.Google Scholar
  33. HÄrdle, W., Hall, P. and Marron, J.S. (1988). How far are automatically chosen regression smoothing parameters from their optimum? J. Amer. Statst. Assoc., 83, 86–101. With discussions.Google Scholar
  34. Härdle, W., Hall, P. and Marron, J.S. (1992a). Regression smoothing parameters that are not far from their optimum? J. Amer. Statst. Assoc.,In print.Google Scholar
  35. Härdle, W., Hart, J., Marron, J.S. and Tsybakov, A. (1992b). Bandwidth choice for average derivative estimation. J. Amer. Statst. Assoc., 87, 218–226.zbMATHGoogle Scholar
  36. Härdle, W. and Kelly, G. (1987). Nonparametric kernel regression estimation. Optimal bandwidth choice. Statistics, 18, 21–35.MathSciNetCrossRefzbMATHGoogle Scholar
  37. Härdle, W. and Mammen, E. (1990). Bootstrap methods in nonparametric regression. Proccedings of NATO advanced study on Nonparametric Functional Estimation, Kluwer Dordrecht (ed G.G. ROUSSAS), 111–123.Google Scholar
  38. Härdle, W. and Marron, J.S. (1985a). Optimal bandwidth selection rule in nonparametric regression. Ann. Statist.13, 4, 1465–1481.CrossRefGoogle Scholar
  39. Härdle, W. and Marron, J.S. (1985b). Asymptotic nonequivalence of some bandwidth selectors in nonparametric regression. Biometrika, 72, 481–484.MathSciNetzbMATHGoogle Scholar
  40. Härdle, W. and Marron, J.S. (1990). Semiparanetric comparison of regression curves. Ann. Statist. 18, 63–89.MathSciNetCrossRefzbMATHGoogle Scholar
  41. Härdle, W. and Vieu, P. (1991). Kernel regression smoothing of time series. J. of Time Series Anal., in print.Google Scholar
  42. Hart, J. and Vieu, P. (1990). Data-driven bandwidth choice for density estimation based on dependent data. Ann. Statist., 18, 873–890.MathSciNetCrossRefzbMATHGoogle Scholar
  43. Hart, J. and Wehrly, T, (1991). Consistency of cross-validation when data are curves. Accepted for publication in Stoch. Proc. Appl..Google Scholar
  44. Herrmann, E., Gasser, T. and Kneip, A. (1991). Choice of bandwith for kernel regression when residuals are correlated. Preprint.Google Scholar
  45. Johnston, G.J. (1982). Probabilities of maximal deviations for nonparametric regression function estimates. J. Mult. Anal.,. 12, 402–444.MathSciNetCrossRefzbMATHGoogle Scholar
  46. Jones, M.C. (1990). Prospects for automatic bandwidth selection in extensions to basic kernel estimation. Proccedings of NATO advanced study on Nonparametric Functional Estimation, Kluwer Dordrecht (ed G.G. ROUSSAS), 241–250.Google Scholar
  47. Jones, M.C. and Davies, S. (1991). Versions of kernel regression estimator. Preprint.Google Scholar
  48. Konakov, V.D. and Piterbarg, V.I. (1984). On the convergence rate of maximal deviation distribution for kernel regression estimates. Journ. Mult. Anal., 15, 279–294.MathSciNetCrossRefzbMATHGoogle Scholar
  49. Mack, Y. and Wulwick, M.C. (1991). Nonparametric regression analysis of some economic data. Proccedings of NATO advanced study on Nonparametric Functional Estimation, Kluwer Dordrecht (ed G.G. ROUSSAS), 361–374.Google Scholar
  50. Mack, Y. and Müller, H.G. (1989a). Derivative estimation in nonparametric regression with random predictor variable. Sankhya, A, 51, 59–72.zbMATHGoogle Scholar
  51. Mack, Y. and Müller, H.G. (1989b). Convolution type estimators for nonparametric regression. Statist. Proba. Let., 7, 229–239.CrossRefGoogle Scholar
  52. Marron, J.S. (1987). Partitioned cross-validation. Econometric Reviews, 6, 271–284.MathSciNetCrossRefGoogle Scholar
  53. Marron, J.S. (1988). Automatic smoothing parameter selection: a survey. Empirical Economics, 13, 187–208.CrossRefGoogle Scholar
  54. Marron, J.S. (1991). Bootstrap bandwidth selection. Preprint.Google Scholar
  55. Marron. J.S. and Härdle, W. (1986). Random approximations of some measures of accuracy in nonparametric curve estimation. J. Multiv. Anal., 20, 91–113.CrossRefGoogle Scholar
  56. Marron, J.S. and Wand, M. (1991). Exact mean integrated squared error. Preprint.Google Scholar
  57. Müller, H.G. (1985). Empirical bandwidth choice for nonparametric regression by means of pilot estimators. Statist. and Decisions, 2, 193–206.Google Scholar
  58. Müller, H.G. and Wang, J.L. (1990). Locally adaptive hazard smoothing. Prob. Th. Rel. Fields, 85, 523–538.CrossRefzbMATHGoogle Scholar
  59. Nadaraya, E. (1964). On estimating regression. Theory Proba. and App1., 9, 141–142. PATIL, P. ( 1991 ). Bandwidth choice for non parametric hazard rate estimation. Preprint.Google Scholar
  60. Park, B. and Marron, J.S. (1990). Comparison of data-driven bandwidth selectors. J. Amer. Statist. Ass., 85, 66–72.CrossRefGoogle Scholar
  61. Parzen, E. (1962). On estimating a probability density function and the mode. Ann. Math. Statist., 33, 1065–1076.MathSciNetCrossRefzbMATHGoogle Scholar
  62. Priestley, M.B. and Chao, M.T. (1972). Non-parametric function fitting. J. Roy. Statist. Ass., ser. B, 34, 385–392.MathSciNetzbMATHGoogle Scholar
  63. Rice, J. (1984). Bandwidth choice for nonparametric regression. Ann. Statist., 33, 1215 1230.Google Scholar
  64. Sarda, P. (1991). Estimating smooth distribution functions. Proceedings of NATO advanced study on Nonparametric Functional Estimation, Kluwer Dordrecht (ed G.G. ROUSSAS), 261270.Google Scholar
  65. Sarda, P. and Vieu, P. (1991). Smoothing parameter selection in hazard estimation. Statist. Proba. Let., 11, 429–434.MathSciNetCrossRefzbMATHGoogle Scholar
  66. Shibata, R. (1981). An optimal selection of regression variables. Biometrika, 68, 45–54. STANISWALLIS, J.G. (1989). Local bandwidth selection for kernel estimate. J. Amer. Statist. Ass., 84.Google Scholar
  67. Stone, C. (1982). Optimal global rates of convergence in nonparametric regression. Ann. Statist., 10, 1040–1053.MathSciNetCrossRefzbMATHGoogle Scholar
  68. Vieu, P. (1991a). Quadratic errors for nonparametric estimates under dependence. J. Multivariate Anal., 39, 324–347.MathSciNetCrossRefzbMATHGoogle Scholar
  69. Vieu, P. (1991b). Smoothing techniques in Time Series Analysis. Proceedings of NATO advanced study on Nonparametric Functional Estimation, Kluwer Dordrecht (ed G.G. ROUSSAS), 271–283.Google Scholar
  70. Vieu, P. (1991c). Nonparametric regression: local optimal bandwidth choice. J. Roy. Statist. Soc. B, 53, 453–464.MathSciNetzbMATHGoogle Scholar
  71. Vieu, P. (1991d). On nonparametric autoregression order choice. Preprint.Google Scholar
  72. Watson, G.S. (1964). Smooth regression analysis. Sankhya, A, 26, 359–372.zbMATHGoogle Scholar
  73. Wehrly, T, and Hart, J. (1988). Bandwidth selectors for kernel estimators of growth curves with correlated errors (abstract). Inst. Math. Statist. Bull., 17.Google Scholar
  74. Wong, W. (1982). On the consistency of cross validation in kernel non-parametric regression. Ann. Statist., 11, 1136–1141.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1993

Authors and Affiliations

  • P. Vieu
    • 1
  1. 1.Laboratoire de Statistique et ProbabilitésUniversité Paul SabatierToulouse CedexFrance

Personalised recommendations