Skip to main content
SpringerLink
Log in

Experimental and analytic comparison of the accuracy of different estimates of parameters in a linear regression model

  • Stochastic Systems
  • Published:
Automation and Remote Control Aims and scope Submit manuscript

Abstract

We consider LS-, LAD-, R-, M-, S-, LMS-, LTS-, MM-, and HBR-estimates for the parameters of a linear regression model with unknown noise distribution. With computer modeling for medium sized samples, we compare the accuracy of the considered estimates for the most popular probability distributions of noise in a regression model. For different noise distributions, we analytically compute asymptotic efficiencies of LS-, LAD-, R-, M-, S-, and LTS- estimates. We give recommendations for practical applications of these methods for different noise distributions in the model. We show examples on real datasets that support the advantages of robust estimates.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Deniel, C., Applications of Statistics to Industrial Experimentation, New York: Wiley, 1976. Translated under the title Primenenie statistiki v promyshlennom eksperimente, Moscow: Mir, 1979.

    Book  Google Scholar 

  2. Huber, P.J., Robust Regression: Asymptotics, Conjectures and Monte Carlo, Ann. Statist., 1973, vol. 1, no. 5, pp. 799–821.

    Article  MathSciNet  MATH  Google Scholar 

  3. Jaeckel, L.A., Estimating Regression Coefficients by Minimizing the Dispersion of the Residuals, Ann. Statist., 1972, vol. 43, no. 5, pp. 1449–1458.

    Article  MathSciNet  MATH  Google Scholar 

  4. Jureckova, J., Nonparametric Estimate of Regression Coefficients, Ann. Math. Statist., 1971, vol. 42, no. 4, pp. 1328–1338.

    Article  MathSciNet  MATH  Google Scholar 

  5. Rousseeuw, P.J. and Leroy, A.M., Robust Regression and Outlier Detection, Chichester: Wiley, 1987.

    Book  MATH  Google Scholar 

  6. Rousseeuw, P.J., Least Median of Squares Regression, J. Am. Statist. Assoc., 1984, vol. 79, no. 388, pp. 871–880.

    Article  MathSciNet  MATH  Google Scholar 

  7. Rousseeuw, P.J. and Yohai, V.J., Robust Regression by Means of S-Estimators, in Robust and Nonlinear Time Series, Lecture Notes in Statistics, 1984, vol. 26, pp. 256–272.

    Article  MathSciNet  MATH  Google Scholar 

  8. Yohai, V.J., High Breakdown-Point and High Efficiency Robust Estimates for Regression, Ann. Statist., 1987, vol. 15, no. 2. pp. 642–656.

    Article  MathSciNet  MATH  Google Scholar 

  9. Chang, W.H., Mckean, J.W., Naranjo, J.D., and Sheather, S.J., High-Breakdown Rank Regression, J. Am. Statist. Assoc., 1999, vol. 94, no. 445, pp. 205–219.

    Article  MathSciNet  MATH  Google Scholar 

  10. Hettmansperger, T.P. and McKean, J.W., Robust Nonparametric Statistical Methods, Boca Raton: CRC Press, 2011.

    MATH  Google Scholar 

  11. Aleskerov, F.T. and Subochev, A.N., On Stable Solutions to the Ordinal Social Choice Problem, Dokl. Math., 2009, vol.79, no. 3, pp. 437–439.

    Article  MathSciNet  MATH  Google Scholar 

  12. Zhang, H., Liu, C.-T., and Wang, X., An Association Test for Multiple Traits Based on the Generalized Kendall’s Tau, JASA, 2010, vol. 105, no. 490, pp. 473–481.

    Article  MathSciNet  MATH  Google Scholar 

  13. Maronna, R.A., Martin, D., and Yohai, V., Robust Statistics: Theory and Methods, Chichester: Wiley, 2006.

    Book  MATH  Google Scholar 

  14. Salibian-Barrera, M. and Yohai, V.J., A Fast Algorithm for S-Regression Estimates, J. Comput. Graphical Statist., 1999, vol. 15, no. 2, pp. 414–427.

    Article  MathSciNet  Google Scholar 

  15. Agull´o, J., Exact Algorithms for Computing the Least Median of Squares Estimate in Multiple Linear Regression, J. Lecture Notes-Monograph, L1-Statist. Proc. Related Topics, 1997, vol. 31, pp. 133–146.

    Google Scholar 

  16. Rousseeuw, P.J. and van Driessen, K., Computing LTS Regression for Large Datasets, Data Mining Knowledge Discovery, 2006, vol. 12, no. 1, pp. 29–45.

    Article  MathSciNet  Google Scholar 

  17. Seber, G., Linear Regression Analysis, New York: Wiley, 1977. Translated under the title Lineinyi regressionnyi analiz, Moscow: Mir, 1980.

    MATH  Google Scholar 

  18. Basset, G. and Koenker, R., Asymptotic Theory of Least Absolute Error Regression, JASA, 1978, vol. 73, no. 363, pp. 618–622.

    Article  MathSciNet  MATH  Google Scholar 

  19. Pollard, D., Asymptotics for Least Absolute Deviation Regression Estimators, Econometric Theory, 1991, vol. 7, pp. 186–199.

    Article  MathSciNet  Google Scholar 

  20. Boldin, M.V., Simonova, G.I., and Tyurin, Yu.N., Znakovyi statisticheskii analiz lineinykh modelei (Signed Statistical Analysis of Linear Models), Moscow: Nauka, 1997.

    Google Scholar 

  21. Davies, P.L., The Asymptotics of S-Estimators in the Linear Regression Model, Ann. Statist., 1990, vol. 18, no. 4, pp. 1651–1675.

    Article  MathSciNet  MATH  Google Scholar 

  22. Wilks, S., Mathematical Statistics, New York: Wiley, 1962. Translated under the title Matematicheskaya statistika, Moscow: Nauka, 1967.

    MATH  Google Scholar 

  23. Hampel, F.R., Ronchetti, E.M., Rousseeuw, P.J., and Stahel, W.A., Robust Statistics: The Approach Based on Influence Functions, New York: Wiley, 1986. Translated under the title Robastnost’ v statistike. Podkhod na osnove vliyaniya, Moscow: Mir, 1989.

    MATH  Google Scholar 

  24. Maronna, R.A. and Yohai, V.J., Correcting MM Estimates for “Fat” Data Sets, Computat. Statist. Data Anal., 2010, vol. 54, no. 12, pp. 3168–3173.

    Article  MathSciNet  MATH  Google Scholar 

  25. Rousseeuw, P.J. and van Driessen, K., A Fast Algorithm for the Minimum Covariance Determinant Estimator, Technometrics, 1999, vol. 41, no. 3, pp. 212–223.

    Article  Google Scholar 

  26. http://data.worldbank.org/indicator/NY.GDP.PCAP.CD

  27. http://data.worldbank.org/indicator/EN.ATM.CO2E.PC

Download references

Author information

Authors and Affiliations

  1. National Research University Higher School of Economics, Moscow, Russia

    E. R. Goryainova

  2. SJC “Europlan,”, Moscow, Russia

    E. A. Botvinkin

Authors
  1. E. R. Goryainova
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. E. A. Botvinkin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to E. R. Goryainova.

Additional information

Original Russian Text © E.R. Goryainova, E.A. Botvinkin, 2017, published in Avtomatika i Telemekhanika, 2017, No. 10, pp. 109–129.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Goryainova, E.R., Botvinkin, E.A. Experimental and analytic comparison of the accuracy of different estimates of parameters in a linear regression model. Autom Remote Control 78, 1819–1836 (2017). https://doi.org/10.1134/S000511791710006X

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S000511791710006X

Keywords

Search

Navigation