Universal weighted kernel-type estimators for some class of regression models

Abstract

For a wide class of nonparametric regression models with random design, we suggest consistent weighted least square estimators, asymptotic properties of which do not depend on correlation of the design points. In contrast to the predecessors’ results, the design is not required to be fixed or to consist of independent or weakly dependent random variables under the classical stationarity or ergodicity conditions; the only requirement being that the maximal spacing statistic of the design tends to zero almost surely (a.s.). Explicit upper bounds are obtained for the rate of uniform convergence in probability of these estimators to an unknown estimated random function which is assumed to lie in a Hölder space a.s. A Wiener process is considered as an example of such a random regression function. In the case of i.i.d. design points, we compare our estimators with the Nadaraya–Watson ones.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  1. Benhenni K, Hedli-Griche S, Rachdi M (2010) Estimation of the regression operator from functional fixed-design with correlated errors. J Multivar Anal 101:476–490

    MathSciNet  Article  Google Scholar 

  2. Brown LD, Levine M (2007) Variance estimation in nonparametric regression via the difference sequence method. Ann Stat 35:2219–2232

    MathSciNet  Article  Google Scholar 

  3. Einmahl U, Mason DM (2005) Uniform in bandwidth consistency of kernel-type function estimators. Ann Stat 33:1380–1403

    MathSciNet  Article  Google Scholar 

  4. Fan J, Gijbels I (1996) Local polynomial modelling and its applications. Chapman and Hall, London

    Google Scholar 

  5. Gasser T, Engel J (1990) The choice of weights in kernel regression estimation. Biometrica 77:277–381

    MathSciNet  Article  Google Scholar 

  6. Györfi L, Kohler M, Krzyzak A, Walk H (2002) A distribution-free theory of nonparametric regression. Springer, New York

    Google Scholar 

  7. Hall P, Müller H-G, Wang J-L (2006) Properties of principal component methods for functional and longitudinal data analysis. Ann Stat 34:1493–1517

    MathSciNet  Article  Google Scholar 

  8. Hansen BE (2008) Uniform convergence rates for kernel estimation with dependent data. Econom Theory 24:726–748

    MathSciNet  Article  Google Scholar 

  9. Härdle W (1990) Applied nonparametric regression. Cambridge University Press, New York

    Google Scholar 

  10. Honda T (2010) Nonparametric regression for dependent data in the errors-in-variables problem. Institute of economic research, Hitotsubashi University, Global COE Hi-Stat discussion paper series

    Google Scholar 

  11. Jennen-Steinmetz C, Gasser T (1989) A unifying approach for nonparametric regression estimation. J Am Stat Assoc 83:1084–1089

    MathSciNet  Article  Google Scholar 

  12. Jones MC, Davies SJ, Park BU (1994) Versions of kernel-type regression estimators. J Am Stat Assoc 89:825–832

    MathSciNet  Article  Google Scholar 

  13. Kulik R, Lorek P (2011) Some results on random design regression with long memory errors and predictors. J Stat Plan Inference 141:508–523

    MathSciNet  Article  Google Scholar 

  14. Kulik R, Wichelhaus C (2011) Nonparametric conditional variance and error density estimation in regrssion models with dependent errors and predictors. Electron J Stat 5:856–898

    MathSciNet  Article  Google Scholar 

  15. Laib N, Louani D (2010) Nonparametric kernel regression estimation for stationary ergodic data: asymptotic properties. J Multivar Anal 101:2266–2281

    MathSciNet  Article  Google Scholar 

  16. Li Y, Hsing T (2010) Uniform convergence rates for nonparametric regression and principal component analysis in functional/longitudinal data. Ann Stat 38:3321–3351

    MathSciNet  Article  Google Scholar 

  17. Linke YY, Borisov IS (2017) Constructing initial estimators in one-step estimation procedures of nonlinear regression. Stat Probab Lett 120:87–94

    MathSciNet  Article  Google Scholar 

  18. Linke YY, Borisov IS (2018) Constructing explicit estimators in nonlinear regression problems. Theory Probab Appl 63:22–44

    MathSciNet  Article  Google Scholar 

  19. Mack YP, Müller H-G (1987) Adaptive nonparametric estimation of a multivariate regression function. J Multivar Anal 23:169–182

    MathSciNet  Article  Google Scholar 

  20. Mack YP, Müller H-G (1988) Convolution type estimators for nonparametric regression. Stat Probab Lett 7:229–239

    MathSciNet  Article  Google Scholar 

  21. Mack YP, Müller H-G (1989) Erratum. Stat Probab Lett 8:195

  22. Masry E (2005) Nonparametric regression estimation for dependent functional data. Stoch Proc Appl 115:155–177

    MathSciNet  Article  Google Scholar 

  23. Müller H-G (1988) Nonparametric regression analysis of longitudinal data. Springer, New York

    Google Scholar 

  24. Petrov VV (1975) Sums of independent random variables. Springer, New York

    Google Scholar 

  25. Priestley MB, Chao MT (1972) Non-parametric function fitting. J R Stat Soc Ser B 34:385–392

    MathSciNet  MATH  Google Scholar 

  26. Rosenblatt M (1969) Conditional probability density and regression estimates. In: Krishaiah PR (ed) Multivariate analysis II. Wiley, New York, pp 25–31

    Google Scholar 

  27. Roussas GG (1990) Nonparametric regression estimation under mixing conditions. Stoch Proc Appl 36:107–116

    MathSciNet  Article  Google Scholar 

  28. Roussas GG (1991) Kernel estimates under association: strong uniform consistency. Stat Probab Lett 12:393–403

    MathSciNet  Article  Google Scholar 

  29. Wand MP, Jones MC (1995) Kernel smoothing. Chapman and Hall, London

    Google Scholar 

  30. Wu JS, Chu CK (1994) Nonparametric estimation of a regression function with dependent observations. Stoch Proc Appl 50:149–160

    MathSciNet  Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Pavel S. Ruzankin.

Ethics declarations

Conflict of interest

On behalf of all authors, P.S. Ruzankin states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported by the Russian Foundation for Basic Research, under Grant 18-01-00074; and by the Program for Fundamental Scientific Research of the SB RAS, No. I.1.3, under Grant 0314-2019-0008.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Borisov, I.S., Linke, Y.Y. & Ruzankin, P.S. Universal weighted kernel-type estimators for some class of regression models. Metrika 84, 141–166 (2021). https://doi.org/10.1007/s00184-020-00768-0

Download citation

Keywords

  • Nonparametric regression
  • Uniform consistency
  • Kernel-type estimator

Mathematics Subject Classification

  • 62G08