Regression function estimation as a partly inverse problem

Abstract

This paper is about nonparametric regression function estimation. Our estimator is a one-step projection estimator obtained by least-squares contrast minimization. The specificity of our work is to consider a new model selection procedure including a cutoff for the underlying matrix inversion, and to provide theoretical risk bounds that apply to non-compactly supported bases, a case which was specifically excluded of most previous results. Upper and lower bounds for resulting rates are provided.

This is a preview of subscription content, log in to check access.

Notes

  1. 1.

    If \(b_A\) is a combination of \( \Gamma \)-type functions, then the bias term \(\inf _{t\in S_m}\Vert b_A-t\Vert ^2\) is much smaller (exponentially decreasing) and the rate \(\log (n)/n\) can be reached by the adaptive estimator (see e.g. Mabon 2017).

  2. 2.

    In Cohen et al. (2013), the condition \(K(m)<+\infty \) is not clearly stated; it is implicit as the result does not hold otherwise. Actually all examples of the paper are for A compact, in which case \(K(m)<+\infty \). If A is not compact, then K(m) may be \(+\infty \). Therefore, our condition (7) and Lemma 4 clarify Cohen et al.’s result.

References

  1. Abramowitz, M., Stegun, I. A. (1964). Handbook of mathematical functions with formulas, graphs, and mathematical tables, ninth dover printing, tenth gpo printing edn. New York: Dover.

  2. Askey, R., Wainger, S. (1965). Mean convergence of expansions in Laguerre and Hermite series. American Journal of Mathematics, 87, 695–708.

    MathSciNet  Article  Google Scholar 

  3. Baraud, Y. (2000). Model selection for regression on a fixed design. Probability Theory and Related Fields, 117, 467–493.

    MathSciNet  Article  Google Scholar 

  4. Baraud, Y. (2002). Model selection for regression on a random design. ESAIM Probability and Statistics, 6, 127–146.

    MathSciNet  Article  Google Scholar 

  5. Barron, A., Birgé, L., Massart, P. (1999). Risk bounds for model selection via penalization. Probability Theory and Related Fields, 113, 301–413.

    MathSciNet  Article  Google Scholar 

  6. Belomestny, D., Comte, F., Genon-Catalot, V. (2016). Nonparametric Laguerre estimation in the multiplicative censoring model. Electronic Journal of Statistics, 10(2), 3114–3152.

    MathSciNet  Article  Google Scholar 

  7. Belomestny, D., Comte, F., Genon-Catalot, V. (2019). Sobolev-Hermite versus Sobolev nonparametric density estimation on R. The Annals of the Institute of Statistical Mathematics, 71, 29–62.

    MathSciNet  Article  Google Scholar 

  8. Birgé, L., Massart, P. (1998). Minimum contrast estimators on sieves: Exponential bounds and rates of convergence. Bernoulli, 4, 329–375.

    MathSciNet  Article  Google Scholar 

  9. Bouaziz, O., Brunel, E., Comte, F. (2018). Nonparametric survival function estimation for data subject to interval censoring case 2. Preprint hal-01766456.

  10. Brunel, E., Comte, F. (2009). Cumulative distribution function estimation under interval censoring case 1. Electrononic Journal of Statistics, 3, 1–24.

    MathSciNet  Article  Google Scholar 

  11. Cohen, A., Davenport, M. A., Leviatan, D. (2013). On the stability and accuracy of least squares approximations. Foundations of Computational Mathematics, 13, 819–834.

    MathSciNet  Article  Google Scholar 

  12. Comte, F., Genon-Catalot, V. (2015). Adaptive Laguerre density estimation for mixed Poisson models. Electronic Journal of Statistics, 9, 1112–1148.

    MathSciNet  MATH  Google Scholar 

  13. Comte, F., Genon-Catalot, V. (2018). Laguerre and Hermite bases for inverse problems. Journal of the Korean Statistical Society, 47, 273–296.

    MathSciNet  Article  Google Scholar 

  14. Comte, F., Cuenod, C.-A., Pensky, M., Rozenholc, Y. (2017). Laplace deconvolution and its application to dynamic contrast enhanced imaging. Journal of the Royal Statistical Society, Series B, 79, 69–94.

    Article  Google Scholar 

  15. DeVore, R. A., Lorentz, G. G. (1993). Constructive approximation. Berlin: Springer.

    Google Scholar 

  16. Efromovich, S. (1999). Nonparametric curve estimation. Methods, theory, and applications. Springer Series in Statistics. New York: Springer.

  17. Klein, T., Rio, E. (2005). Concentration around the mean for maxima of empirical processes. Annals of Probability, 33(3), 1060–1077.

    MathSciNet  Article  Google Scholar 

  18. Mabon, G. (2017). Adaptive deconvolution on the nonnegative real line. Scandinavian Journal of Statistics, 44, 707–740.

    MathSciNet  Article  Google Scholar 

  19. Nadaraya, E. A. (1964). On estimating regression. Theory of Probability and Its Applications, 9, 141–142.

    Article  Google Scholar 

  20. Plancade, S. (2011). Model selection for hazard rate estimation in presence of censoring. Metrika, 74, 313–347.

    MathSciNet  Article  Google Scholar 

  21. Stewart, G. W., Sun, J.-G. (1990). Matrix perturbation theory. Boston: Academic Press, Inc.

    Google Scholar 

  22. Szegö, G. (1975). Orthogonal polynomials (4 Ed.). American Mathematical Society, Colloquium Publications, Vol. XXIII. Providence, RI: American mathematical Society.

  23. Tropp, J. A. (2012). User-friendly tail bounds for sums of random matrices. Foundations of Computational Mathematics, 12(4), 389–434.

    MathSciNet  Article  Google Scholar 

  24. Tsybakov, A. B. (2009). Introduction to nonparametric estimation. Springer Series in Statistics. New York: Springer.

    Google Scholar 

  25. Watson, G. S. (1964). Smooth regression analysis. Sankhy \(\bar{a}\) Series A, 26, 359–372.

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to F. Comte.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 938 KB)

About this article

Verify currency and authenticity via CrossMark

Cite this article

Comte, F., Genon-Catalot, V. Regression function estimation as a partly inverse problem. Ann Inst Stat Math 72, 1023–1054 (2020). https://doi.org/10.1007/s10463-019-00718-2

Download citation

Keywords

  • Hermite basis
  • Laguerre basis
  • Model selection
  • Nonparametric estimation
  • Regression function