Statistical Papers

, Volume 60, Issue 6, pp 1849–1882 | Cite as

Preliminary test and Stein-type shrinkage ridge estimators in robust regression

  • M. Norouzirad
  • M. ArashiEmail author
Regular Article


A statistician may face with a dataset that suffers from multicollinearity and outliers, simultaneously. The Huberized ridge (HR) estimator is a technique that can be used here. On the other hand, an expert may claim that some/all the variables should be removed from the analysis, due to inappropriateness, that imposes a prior information that all coefficients equal to zero (in the form of a restriction) to the analysis. In such situations, one may consider the HR estimation under the subspace restriction. In this paper, we introduce some improved estimators for verifying this claim. They are employed to improve the performance of the HR estimator in the multiple regression model. Advantages of the proposed estimators over the usual HR estimator are demonstrated through a Monte Carlo simulation as well as two real data examples.


M-estimation Multicollinearity Outliers Preliminary test Ridge regression Stein-type Shrinkage 

Supplementary material

362_2017_899_MOESM1_ESM.pdf (1.2 mb)
Supplementary material 1 (pdf 1182 KB)


  1. Akdeniz F (2002) More on the pre-test estimator in ridge regression. Commun Stat Theory Methods 31:987–994MathSciNetzbMATHGoogle Scholar
  2. Akdeniz F, Kaciranlar S (2001) More on the new biased estimator in linear regression. Sankhy B 63:321–325MathSciNetzbMATHGoogle Scholar
  3. Alheety MI, Kibria BMG (2014) A generalized stochastic restricted ridge regression estimator. Commun Stat Theory Methods 43:4415–4427MathSciNetzbMATHGoogle Scholar
  4. Alpu O, Samkar H (2010) Liu estimator based on an m estimator. Turk Klin J Biostat 2:49–53Google Scholar
  5. Arslan O, Billor N (2000) Robust liu estimator for regression based on an M-estimator. J Appl Stat 27:39–47zbMATHGoogle Scholar
  6. Askin RG, Montgomery DC (1980) Augmented robust estimation. Technometrics 22:333–341zbMATHGoogle Scholar
  7. Askin RG, Montgomery DC (1984) An analysis of condtrained robust regression estimators. Navoal Logist Q 32:283–296zbMATHGoogle Scholar
  8. El-Salam M (2013) The efficiency of some robust ridge regression for handling multicollinearity and non-normals errors problems. Appl Math Sci 7:3831–3846MathSciNetGoogle Scholar
  9. Gibbons DGA (1981) Simulation study of some ridge estimators. J Am Stat Assoc 76:131–139zbMATHGoogle Scholar
  10. Gruber MHJ (1986) Improving efficiency by shrinkage the James-Stein and ridge regression estimators. Springer, New YorkzbMATHGoogle Scholar
  11. Hoerl AE, Kennard RW (1970) Ridge regression: biased estimation for non-orthogonal problems. Technometrics 12:55–67zbMATHGoogle Scholar
  12. Huber PJ (1981) Robust statistics. Wiley, HobokenzbMATHGoogle Scholar
  13. Jadhav NH, Kashid DN (2011) A jackknifed ridge M-estimator for regression model with multicollinearity and outliers. J Stat Theory Pract 5:659–673MathSciNetzbMATHGoogle Scholar
  14. Jadhav NH, Kashid DN (2014) Robust winsorized shrinkage estimators for linear regression models. J Modern Appl Stat Methods 13:131–150Google Scholar
  15. James W, Stein C (1961) Estimation with quadratic loss. In: Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics, pp. 361–379, University of California Press, Berkeley.
  16. Kan B, Alpu O, Yazici B (2013) Robust ridge and robust liu estimator for regression based on the lts estimator. J Appl Stat 40:644–655MathSciNetGoogle Scholar
  17. Kibria BMG (1996) On preliminary test ridge regression estimators for linear restrictions in a regression model with non-normal disturbances. Commun Stat Theory Methods 25:2349–2369MathSciNetzbMATHGoogle Scholar
  18. Kibria BMG (2003) Performance of some new ridge regression estimators. Commun Stat Simul Comput 32:419–435MathSciNetzbMATHGoogle Scholar
  19. Kibria BMG (2004a) One some ridge regression estimators under possible stochastic constraints. Pak J Stat All Ser 20:1–24zbMATHGoogle Scholar
  20. Kibria BMG (2004b) Performance of the shrinkage preliminary test ridge regression estimators based on the conflicting of w, lr and lm tests. J Stat Comput Simul 74:793–810MathSciNetzbMATHGoogle Scholar
  21. Kibria BMG (2012) Some liu and ridge-type estimators and their properties under the ill-conditioned gaussian linear regression model. J Stat Comput Simul 82:1–17MathSciNetzbMATHGoogle Scholar
  22. Kibria BMG, Saleh AKME (2003) Effect of w, lr, and lm tests on the performance of preliminary test ridge regression estimators. J Jpn Stat Soc 33:119–136MathSciNetzbMATHGoogle Scholar
  23. Kibria BMG, Saleh AKME (2004a) Performance of positive rule estimator in the ill-conditioned gaussian regression model. Calcutta Stat Assoc Bull 55:209–239MathSciNetzbMATHGoogle Scholar
  24. Kibria BMG, Saleh AKME (2004b) Preliminary test ridge regression estimators with students t errors and conflicting test-statistics. Metrika 59:105–124MathSciNetzbMATHGoogle Scholar
  25. Lawrence KD, Marsh LC (1984) Robust ridge estimation methods for predicting US coal mining fatalities. Commun Stat Theory Methods 13:139–149Google Scholar
  26. McDonald GC, Galarneau DI (1975) A monte carlo evaluation of some ridge-type estimators. J Am Stat Assoc 70:407–416zbMATHGoogle Scholar
  27. Montgomery DC, Askin RG (1981) Problems of nonnormality and multicollinearity for forecasting methods based on least squares. AIIE Trans 13:102–115MathSciNetGoogle Scholar
  28. Montgomery DC, Peck EA (1992) Introduction to linear regression analysis, 2nd edn. Wiley, HobokenzbMATHGoogle Scholar
  29. Muniz G, Kibria BMG (2010) On some ridge regression estimators: an empirical comparisons. Commun Stat Simul Comput 38:621–630MathSciNetzbMATHGoogle Scholar
  30. Muniz G, Kibria BMG, Mansson KM, Shukur G (2012) On developing ridge regression parameters: a graphical investigation. SORT 36:115–138MathSciNetzbMATHGoogle Scholar
  31. Norouzirad M, Arashi M (2017) Supplemental materials: The necessity of using shrinkage ridge M-estimator when multicollinearity and outliers are present in a datasetGoogle Scholar
  32. Ozturk F, Akdeniz F (2000) Ill-conditioning and multicollinearity. Linear Algebra Appl 321:295–305MathSciNetzbMATHGoogle Scholar
  33. Pati KD, Adnan R, Rasheed BA (2014) Ridge least trimmed squares estimators in presence of multicollinearity and outliers. Nat Sci 12:1–8Google Scholar
  34. Penrose K, Nelson A, Fisher A (1985) Generalized body composition prediction equation for men using simple measurement techniques. Med Sci Sports Exerc 17:189Google Scholar
  35. Pfaffenberger RC, Dielman TE (1984) A modified ridge regression estimator using the least absolute value criterion in the multiple linear regression model. In: Proceedings of the American Institute for Decision Sciences, pp. 791–793Google Scholar
  36. Pfaffenberger RC, Dielman TE (1985) A comparison of robust ridge estimators. In: Proceedings of the American Statistical Association Business and Economic Statistics Section, pp. 631–635Google Scholar
  37. Saleh AKME (2006) Theory of preliminary test and Stein-type estimation with applications. Wiley, HobokenzbMATHGoogle Scholar
  38. Saleh AKME, Kibria BMG (1993) Performance of some new preliminary test ridge regression estimators and their properties. Commun Stat Theory Methods 22:2747–2764MathSciNetzbMATHGoogle Scholar
  39. Saleh AKME, Shiraishi T (1989) On some r and m estimators of regression parameters under uncertain restriction. J Jpn Stat Soc 19:129–137MathSciNetzbMATHGoogle Scholar
  40. Samkar H, Alpu O (2010) Ridge regression based on some robust estimators. J Modern Appl Stat Methods 9:495–501Google Scholar
  41. Sarker N (1992) A new estimator combining the ridge regression and the restricted least squares method of estimation. Commun Stat Theory Methods 21:1987–2000MathSciNetzbMATHGoogle Scholar
  42. Sengupta D, Jammalamadaka SR (2003) Linear models: an integrated approach. World Scientific Publishing Company, SingaporezbMATHGoogle Scholar
  43. Silvapull MJ (1991) Robust ridge regression based on an M estimator. Aust N Z J Stat 33:319–333MathSciNetGoogle Scholar
  44. Susanti Y, Pratiwi H, Sulistijowati S, Liana T (2014) M estimation, S estimation, and MM estimation in robust regression. Int J Pure Appl Math 91:349–360zbMATHGoogle Scholar
  45. Tabakan G, Akdeniz F (2010) Difference-based ridge estimator of parameters in partial linear model. Stat Papers 51:357–368MathSciNetzbMATHGoogle Scholar
  46. Tabatabaey SMM, Kibria BGM, Saleh AKMdE (2004a) Estimation strategies for parameters of the linear regression models with spherically symmetric distributions. J Stat Res 38:13–31MathSciNetGoogle Scholar
  47. Tabatabaey SMM, Saleh AKME, Kibria BGM (2004b) Simultaneous estimation of regression parameters with spherically symmetric errors under possible stochastic constraints. Int J Stat Sci 3:1–20Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  1. 1.Department of Statistics, School of Mathematical SciencesShahrood University of TechnologyShahroodIran

Personalised recommendations