Advertisement

Robust Estimators in Non-linear Regression Models with Long-Range Dependence

  • A. Ivanov
  • N. Leonenko
Chapter
Part of the Springer Optimization and Its Applications book series (SOIA, volume 28)

Summary

We present the asymptotic distribution theory for M-estimators in non-linear regression model with long-range dependence (LRD) for a general class of covariance functions in discrete and continuous time. Our limiting distributions are not always Gaussian and they have second moments. We present non-Gaussian distributions in terms of characteristic functions rather then the multiple Ito–Wiener integrals. These results are some variants of the non-central limit theorems of Taqqu (1979), however the normalizing factors and limiting distributions are of more general type. Beran (1991) observed, in the case of a Gaussian sample with LRD, that the M-estimators and the least-squares estimator of the location parameter have equal asymptotic variances. We present a similar phenomenon for a general non-linear regression model with LRD in discrete and continuous time (see Corollary 1), that is in the case of Gaussian non-linear regression models with LRD errors the M-estimates (for smooth score functions) of the regression parameters are asymptotically equivalent in the first order to the least-squares estimator.

Keywords

Linear Regression Model Asymptotic Normality Little Square Estimate Robust Estimator Nonlinear Regression Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Adenstedt, R.K. (1974). On large-sample estimation of the mean of a stationary random sequence. The Annals of Statistics, 2, 1095-1107.MATHCrossRefMathSciNetGoogle Scholar
  2. Albin, J.M.P. (1998). A note on Rosenblatt distribution. Statistics & Probability Letters, 40, 83-91.MATHCrossRefMathSciNetGoogle Scholar
  3. Anh, V.V., Leonenko, N.N., and McVinish, R. (2001). Models for fractional Riesz-Bessel motion and related processes. Fractals, 9, N3, 329-346.CrossRefGoogle Scholar
  4. Anh, V.V., Knopova, V.P., and Leonenko, N.N. (2004). Continuous-time stochastic processes with cyclical long-range dependence. Australian and New Zealand Journal of Statistics, 46, N2, 275-296.MATHCrossRefMathSciNetGoogle Scholar
  5. Arcones, M.A. (2001).Asymptotic distribution of regression M-estimators. Journal of Statistical Planning and Inference, 97, 235-261.MATHCrossRefMathSciNetGoogle Scholar
  6. Bardadym, T.A. and Ivanov, A.V. (1999). On asymptotic normality of lα estimates of parameter of non-linear regression. Theory of Probability and Mathematical Statistics, 60, 1-10.MATHGoogle Scholar
  7. Barndorff-Nielsen, O.E. and Shephard, N. (2001). Non-Gaussian Ornstein-Uhlenbeck-based models and some of their uses in financial econometrics (with discussion). Journal of the Royal Statistical Society B, 63, 167-241.MATHCrossRefMathSciNetGoogle Scholar
  8. Barndorff-Nielsen, O.E. and Leonenko, N.N. (2005). Spectral properties of superpositions of Ornstein-Uhlenbeck type processes. Methodology and Computing in Applied Probability, 7, 335-352.MATHCrossRefMathSciNetGoogle Scholar
  9. Beran, J. (1991). M-estimators of location for data with slowly decaying correlations. Journal of the American Statistical Association, 86, 704-708.MATHCrossRefMathSciNetGoogle Scholar
  10. Beran J. (1994). Statistics for Long-Memory Processes. Chapman and Hall, New York.Google Scholar
  11. Beran, J. and Ghosh, S. (1998). Root-n-consistent estimation in partial linear models with long-memory errors. Scandinavian Journal of Statistics, 25, 345-357.MATHCrossRefMathSciNetGoogle Scholar
  12. Braiman R., Yohai, V.J., and Zamar, R.H. (2001). Optimal robust M-estimates of location. The Annals of Statistics, 29, 194-223.CrossRefMathSciNetGoogle Scholar
  13. Cantoni, E. and Ronchetti, E. (2001). Robust inference for generalized linear models. Journal of the American Statistical Association, 96, 1022-1030.MATHCrossRefMathSciNetGoogle Scholar
  14. Choy, K. and Taniguchi, M. (2001). Stochastic regression model with dependent disturbances. Journal of Time Series Analysis, 22, 175-196.MATHCrossRefMathSciNetGoogle Scholar
  15. Dahlhaus, R. (1995). Efficient location and regression estimation for long-range dependent regression models. The Annals of Statistics, 23, 1029-1047.MATHCrossRefMathSciNetGoogle Scholar
  16. Deo, R.S. (1997). Asymptotic theory for certain regression models with long memory errors. Journal of Time Series Analysis, 18, 385-393.MATHCrossRefMathSciNetGoogle Scholar
  17. Deo, R.S. and Hurvich, C.M. (1998). Linear trend with fractional integrated errors. Journal of Time Series Analysis, 19, 379-397.MATHCrossRefMathSciNetGoogle Scholar
  18. Dobrushin, R.L. and Major, P. (1979). Non-central limit theorems for non-linear functionals of Gaussian fields. Z. Wahrsch. verw. Gebiete., 50, 27-52.MATHCrossRefMathSciNetGoogle Scholar
  19. Doukhan, P., Oppenheim, G., and Taqqu, M.S. (2003). Theory and Applications of Long-Range Dependence. Birkhauser, Boston.Google Scholar
  20. Feller, W. (1971). An Introduction to Probability Theory and Its Applications, v. 2. John Willey, New York.Google Scholar
  21. Giraitis L. and Koul H. (1997). Estimation of the dependence parameter in linear regression with long-range dependent errors. Stochastic Processes and their Applications, 71, 207-224.MATHCrossRefMathSciNetGoogle Scholar
  22. Giraitis L., Koul H., and Surgailis, D. (1996). Asymptotic normality of regression estimators with long memory errors. Statistics & Probability Letters, 29, 317-335.MATHCrossRefMathSciNetGoogle Scholar
  23. Grenander, U. and Rosenblatt, M. (1984). Statistical Analysis of Stationary Time Series. Chelsea Publ. Company, New York.Google Scholar
  24. Hall, P., Koul, H.L., and Turlach, A. (1997). Note on the convergence rates of semiparametric estimators of dependence index. The Annals of Statistics, 25, 1725-1739.MATHCrossRefMathSciNetGoogle Scholar
  25. Hannan, E. (1973). The asymptotic theory of linear time-series models. Journal of Applied Probability, 10, 130-145.MATHCrossRefMathSciNetGoogle Scholar
  26. Hampel, F.R., Ronchetti, E.M., Rousseeuw, P.J., and Stahel, W.A. (1986). Robust Statistics. The Approach Based on Influence Functions. Wiley, New York.Google Scholar
  27. Heyde, C.C. and Leonenko, N.N. (2005). Student processes. Advances in Applied Probability, 37, 342-365.MATHCrossRefMathSciNetGoogle Scholar
  28. Holevo, A.S. (1976). On the asymptotic efficient regression estimates in the case of degenerate spectrum. Theory of Probability and Its Applications, 21, 324-333.Google Scholar
  29. Huber, P.J. (1981). Robust Statistics. Wiley, New York.Google Scholar
  30. Ibragimov, I.A. (1963). On estimation of the spectral function of a stationary Gaussian process. Theory of Probability and Its Applications, 8, 366-401.MATHCrossRefGoogle Scholar
  31. Ibragimov, I.A. and Rozanov, Yu.A. (1970). Gaussian Random Processes. Nauka. Moscow (in Russian).Google Scholar
  32. Ivanov, A.V. (1997). Asymptotic Theory of Nonlinear Regression. Kluwer Academic Publishers. Dordrecht.Google Scholar
  33. Ivanov, A.V. and Leonenko, N.N. (1989). Statistical Analysis of Random Fields. Kluwer Academic Publishers. Dordrecht.Google Scholar
  34. Ivanov, A.V. and Leonenko, N.N. (2000). Asymptotic inference for non-linear regression with long-range dependence. Theory of Probability and Mathematical Statistics, 63 61-79.MATHMathSciNetGoogle Scholar
  35. Ivanov, A.V. and Leonenko, N.N. (2002). Asymptotic behavior of M-estimators in continuous-time non-linear regression with long-range dependent errors. Random Operators and Stochastic Equations, 10, N 3, 201-222.MATHCrossRefMathSciNetGoogle Scholar
  36. Ivanov, A.V. and Leonenko, N.N. (2004). Asymptotic theory for non-linear regression with long-range dependence. Mathematical Methods of Statistics, 13, 153-178.MATHMathSciNetGoogle Scholar
  37. Ivanov, A.V. and Leonenko, N.N. (2008). Semiparametric analysis of long-range dependence in nonlinear regression. Journal of Statistical Planning and Inference, 138, 1733–1753MATHCrossRefMathSciNetGoogle Scholar
  38. Ivanov, A.V. and Orlovsky, I.V. (2001). Lp-estimates in nonlinear regression with long-range dependence, Theory of Stochastic Processes, 7(23), N 3-4, 38-49.Google Scholar
  39. Kato, T. and Masry, E. (2003). A time-domain semi-parametric estimate for strong dependent continuous-time stationary processes. Journal of Time Series Analysis, 24, N 6, 679-703.MATHCrossRefMathSciNetGoogle Scholar
  40. Koul, H.L. (1992). M-estimators in linear models with long range dependent errors. Statistics & Probability Letters, 14, 153-164.MATHCrossRefMathSciNetGoogle Scholar
  41. Koul, H.L. (1996). Asymptotics of M-estimations in non-linear regression with long-range dependence errors. In: Proceedings of the Athens Conference on Applied Probability and Time Series Analysis (P.M. Robinson and M. Rosenblatt, Eds.). Springer Verlag Lecture Notes in Statistics, II, 272-291.Google Scholar
  42. Koul, H.L. (2002). Asymptotic distributions of some scale estimators in nonlinear regression models. Metrika, 55, 75-90.CrossRefMathSciNetGoogle Scholar
  43. Koul, H.L. and Baillie, R. T. (2003). Asymptotics of M-estimators in non-linear regression models with long-memory designs. Statistics & Probability Letters, 61, 237-252.MATHCrossRefMathSciNetGoogle Scholar
  44. Koul, H.L. and Mukherjee, K. (1993). Asymptotics of R-, MD- and LAD-estimators in linear regression models with long-range dependent errors. Probability Theory and Related Fields, 95, 535-553.MATHCrossRefMathSciNetGoogle Scholar
  45. Koul, H.L. and Mukherjee, K. (1994). Regression quantiles and related processes under long range dependent errors. Journal of Multivariate Analysis, 51, 318-337.MATHCrossRefMathSciNetGoogle Scholar
  46. Koul, H.L. and Surgailis, D. (2000a). Second-order behavior of M-estimators in linear regression with long-memory errors. Journal of Statistical Planning and Inference, 91, 399-412.MATHCrossRefMathSciNetGoogle Scholar
  47. Koul, H.L. and Surgailis, D. (2000b). Asymptotic normality of the Whittle estimator in linear regression models with long-memory errors. Statistical Inference for Stochastic Processes, 3, 129-147.MATHCrossRefMathSciNetGoogle Scholar
  48. Koul, H.L. and Surgailis, D. (2003). Robust estimators in regression models with long memory errors. In: Theory and Applications of Long-Range Dependence (P. Doukhan, G. Oppenheim and M.S, Taqqu, Eds.). Birkhäuser, Boston, 339–353.Google Scholar
  49. Koul, H.L., Baillie, R.T., and Surgailis, D. (2004). Regression model fitting with a long memory covariance process. Economic Theory, 20, 485-512.MATHMathSciNetGoogle Scholar
  50. Künsch, H.R., Beran, J., and Hampel, F. (1993). Contrasts under long-range correlations. The Annals of Statistics, 21, 943-964.MATHCrossRefMathSciNetGoogle Scholar
  51. Leonenko, N.N. (1999). Limit Theorems for Random Fields with Singular Spectrum. Kluwer Academic Publishers. Dordrecht.Google Scholar
  52. Leonenko, N.N. and Anh, V.V. (2001). Rate of convergence to the Rosenblatt distribution for additive functionals of stochastic processes with long-range dependence. Journal of Applied Mathematics and Stochastic Analysis, 14, 27-46.MATHCrossRefMathSciNetGoogle Scholar
  53. Leonenko, N.N. and Taufer, E. (2006). Weak convergence of weighted quadratic functionals of stationary long memory processes to Rosenblatt-type distributions. Journal of Statistical Planning and Inference, 136, 1220-1236.MATHCrossRefMathSciNetGoogle Scholar
  54. Leonenko, N.N. and Sakhno, L.M. (2006). On the Whittle estimators for some classes of continuos-parameter random processes and fields. Statistics & Probability Letters, 76, 781-795.MATHCrossRefMathSciNetGoogle Scholar
  55. Leonenko, N.N., Sharapov, M.M., and El-Bassiouny, A.H. (2000). On the exactness of normal approximation of LSE of regression coefficient of long-memory random fields. Statistics & Probability Letters, 48, 121-130.MATHCrossRefMathSciNetGoogle Scholar
  56. Leonenko, N.N. and Woyczynski, W.A. (1999). Parameter identification for singular random fields arising in Burgers' turbulence. Journal of Statistical Planning and Inference, 80, 1-13.MATHCrossRefMathSciNetGoogle Scholar
  57. Lobato, I. and Robinson, P.M. (1996). Average periodogramm estimation of long memory. Journal of Econometrics, 73, 303-324.MATHCrossRefMathSciNetGoogle Scholar
  58. Ma, C. (2003). Long-memory continuous-time correlation models. Journal of Applied Probability, 40, 113-1146.Google Scholar
  59. Milnor, J.W. (1965). Topology from the Differentiable Viewpoint. Princeton University Press, Princeton, NJ.Google Scholar
  60. Mukherjee, K. (2000). Linearization of randomly weighted empiricals under long range dependence with applications to nonlinear regression quantiles. Economic Theory, 16, 301-323.MATHGoogle Scholar
  61. Pázman, A. and Pronzato, L. (2006). On the irregular behavior of LS estimates for asymptotially singular designs. Statistics & Probability Letters, 76, 1089-1096.MATHCrossRefMathSciNetGoogle Scholar
  62. Pollard, D. and Radchenko, P. (2006). Nonlinear least-squares estimation. Journal of Multivariate Analysis, 97, 548-562.MATHCrossRefMathSciNetGoogle Scholar
  63. Qian, G. and Künsch, H.R. (1998). On model selection via stochastic complexity in robust linear regression. Journal of Statistical Planning and Inference, 75, 91-116.MATHCrossRefMathSciNetGoogle Scholar
  64. Robinson, P.M. (1994a). Rate of convergence and optimal spectral bandwidth for long range dependence. Probability Theory and Related Fields, 99, 443-473.MATHCrossRefGoogle Scholar
  65. Robinson, P.M. (1994b). Semiparametric analysis of long-memory time series. The Annals of Statistics, 22, 515-539.MATHCrossRefGoogle Scholar
  66. Robinson, P.M. (1995a). Long-periodogram regression of time series with long-range dependence. The Annals of Statistics, 23, 1048-1072.MATHCrossRefGoogle Scholar
  67. Robinson, P.M. (1995b). Gaussian semiparametric estimation of long-range dependence. The Annals of Statistics, 23, 1630-1661.MATHCrossRefGoogle Scholar
  68. Robinson, P.M. and Hidalgo, F.J. (1997). Time series regression with long-range dependence. The Annals of Statistics, 25, 77-104.MATHCrossRefMathSciNetGoogle Scholar
  69. Rosenblatt, M. (1961). Independence and dependence. Proceedings of the Fourth Berkeley Symposium on Mathematics Statistics and Probability. University of California Press, California, 431-443.Google Scholar
  70. Ronner, A.E. (1984). Asymptotic normality of p-norm estimators in multiple regression. Z. Wahrsch.verw. Gebiete, 66, 613-620.MATHCrossRefMathSciNetGoogle Scholar
  71. Samarov, A. and Taqqu, M.S. (1988). On the efficiency of the sample mean in long memory noise. Journal of Time Series Analysis, 9, 191-200.MATHCrossRefMathSciNetGoogle Scholar
  72. Seneta, E. (1976). Regularly Varying Functions. Lecture Notes in Mathematics, 508. Springer-Verlag, Berlin-New York.Google Scholar
  73. Sibbersten, P. (2003). Log-periodogram estimation of the memory parameter of a long-memory process under trend. Statistics & Probability Letters, 61, 261-268.CrossRefMathSciNetGoogle Scholar
  74. Skouras, K. (2000). Strong consistency in nonlinear regression models. The Annals of Statistics, 28, 871-879.MATHCrossRefMathSciNetGoogle Scholar
  75. Taqqu, M.S. (1979). Convergence of iterated processes of arbitrary Hermite rank. Z. Wahrsch. verw. Gebiete., 50, 53-58.MATHCrossRefMathSciNetGoogle Scholar
  76. Yajima, Y. (1988). On estimation of a regression model with long-memory stationary errors. The Annals of Statistics, 16, 791-807.MATHCrossRefMathSciNetGoogle Scholar
  77. Yajima, Y. (1991). Asymptotic properties of the LSE in a regression model with long-memory stationary errors. The Annals of Statistics, 19, 158-177.MATHCrossRefMathSciNetGoogle Scholar
  78. Wu, Y. and Zen, M.M. (1999). A strongly consistent information criterion for linear model selection based on M-estimation. Probability Theory and Related Fields, 113, 599-625.MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media LLC 2009

Authors and Affiliations

  1. 1.National Technical University, Kyiv Polytechnic InstituteKyivUkraine

Personalised recommendations