Statistical Papers

, Volume 60, Issue 3, pp 667–686 | Cite as

Some improved estimation strategies in high-dimensional semiparametric regression models with application to riboflavin production data

  • M. Arashi
  • Mahdi RoozbehEmail author
Regular Article


Due to advances in technologies, modern statistical studies often encounter linear models with high-dimension, where the number of explanatory variables is larger than the sample size. Estimation in these high-dimensional problems with deterministic covariates or designs is very different from those in the case of random covariates, due to the identifiability of the high-dimensional semiparametric regression parameters. In this paper, we consider ridge estimators and propose preliminary test, shrinkage and its positive rule ridge estimators in the restricted semiparametric regression model when the errors are dependent under a multicollinear setting, in high-dimension. The asymptotic risk expressions in addition to biases are exactly derived for the estimators under study. For our proposal, a real data analysis about production of vitamin B2 and a Monté–Carlo simulation study are considered to illustrate the efficiency of the proposed estimators. In this regard, kernel smoothing and cross-validation methods for estimating the optimum ridge parameter and nonparametric function are used.


Generalized restricted ridge estimator High-dimension Kernel smoothing Linear restriction Multicollinearity Semiparametric regression model Shrinkage estimator Sparsity 

Mathematics Subject Classification

Primary: 62G08 62J05 Secondary: 62J07 62G20 



We would like to thank two anonymous reviewers for their valuable comments and suggestions on the earlier version of this article which significantly improved the presentation. First author’s research is supported by Shahrood University of Technology (Grant No. 23088), Iran. The second author’s research is supported in part by a grant 94811069 from the Iran National Science Foundation (INSF) and Research Council of Semnan University.


  1. Akdenïz F, Tabakan G (2009) Restricted ridge estimators of the parameters in semiparametric regression model. Commun Stat Theor Methods 38:1852–1869MathSciNetCrossRefzbMATHGoogle Scholar
  2. Akdenïz Duran E, Härdle WK, Osipenko M (2012) Difference based ridge and Liu type estimators in semiparametric regression models. J Multivar Anal 105:164–175MathSciNetCrossRefzbMATHGoogle Scholar
  3. Amini M, Roozbeh M (2015) Optimal partial ridge estimation in restricted semiparametric regression models. J Multivar Anal 136:26–40MathSciNetCrossRefzbMATHGoogle Scholar
  4. Arashi M, Kibria BMG, Norouzirad M, Nadarajah S (2014) Improved preliminary test and Stein-Rule Liu estimators for the Ill-conditioned elliptical linear regression model. J Multivar Anal 124:53–74MathSciNetCrossRefzbMATHGoogle Scholar
  5. Arashi M, Saleh AKMdE, Tabatabaey SMM (2010) Estimation of parameters of parallelism model with elliptically distributed errors. Metrika 71:79–100MathSciNetCrossRefzbMATHGoogle Scholar
  6. Arashi M, Tabatabaey SMM (2009) Improved variance estimation under sub-space restriction. J Multivar Anal 100:1752–1760MathSciNetCrossRefzbMATHGoogle Scholar
  7. Arashi M, Tabatabaey SMM, Soleimani H (2012) Simple regression in view of elliptical models. Linear Algebra Appl 437:1675–1691MathSciNetCrossRefzbMATHGoogle Scholar
  8. Arashi M, Valizadeh T (2015) Performance of Kibria’s methods in partial linear ridge regression model. Stat Pap 56:231–246MathSciNetCrossRefzbMATHGoogle Scholar
  9. Bunea F (2004) Consistent covariate selection and post model selection inference in semiparametric regression. Ann Stat 32:898–927MathSciNetCrossRefzbMATHGoogle Scholar
  10. Bühlmann P (2013) Statistical significance in high-dimensional linear models. Bernoulli 19(4):1212–1242MathSciNetCrossRefzbMATHGoogle Scholar
  11. Bühlmann P, Kalisch M, Meier L (2014) High-dimensional statistics with a view towards applications in biology. Ann Rev Stat Appl 1:255–278CrossRefGoogle Scholar
  12. Bühlmann P, van de Geer S (2011) Statistics for high-dimensional data: methods, theory and applications. Springer, HeidelbergCrossRefzbMATHGoogle Scholar
  13. Chandra S, Sarkar N (2016) A restricted r-k class estimator in the mixed regression model with autocorrelated disturbances. Stat Pap 57:429–449MathSciNetCrossRefzbMATHGoogle Scholar
  14. Fallahpour S, Ahmed SE, Doksum KA (2012) L1 penalty and shrinkage estimation in partially linear models with random coefficient autoregressive errors. Appl Stoch Model Bus Ind 28:236–250CrossRefzbMATHGoogle Scholar
  15. Fan J, Lv J (2010) A selective overview of variable selection in high dimensional feature space. Stat Sin 20:101–148MathSciNetzbMATHGoogle Scholar
  16. Ghapani F, Rasekh AR, Babadi B (2016) The weighted ridge estimator in stochastic restricted linear measurement error models. Pap Stat. doi: 10.1007/s00362-016-0786-3
  17. Gibbons DG (1981) A simulation study of some ridge estimators. J Am Stat Assoc 76:131–139CrossRefzbMATHGoogle Scholar
  18. Gisela M, Kibria BMG (2009) One some ridge regression estimators: an empirical comparisons. Commun Stat. Simul Comput 38(3):621–630CrossRefzbMATHGoogle Scholar
  19. Härdle W, Liang H, Gao J (2000) Partially linear models. Physika Verlag, HeidelbergCrossRefzbMATHGoogle Scholar
  20. Hastie T, Tibshirani R, Friedman J (2009) The elements of statistical learning: data mining, inference, and prediction, 2nd edn. Springer, New YorkCrossRefzbMATHGoogle Scholar
  21. Hu H (2005) Ridge estimation of a semiparametric regression model. J Comput Appl Math 176:215–222MathSciNetCrossRefzbMATHGoogle Scholar
  22. Hoerl AE, Kennard RW (1970) Ridge regression: biased estimation for non-orthogonal problems. Technometrics 12:55–67CrossRefzbMATHGoogle Scholar
  23. Kibria BMG (2012) Some Liu and Ridge type estimators and their properties under the ill-conditioned Gaussian linear regression model. J Stat Comput Simul 82:1–17MathSciNetCrossRefzbMATHGoogle Scholar
  24. Kibria BMG, Saleh AKMdE (2004) Preliminary test ridge regression estimators with student’s t errors and conflicting test-statistics. Metrika 59:105–124MathSciNetCrossRefzbMATHGoogle Scholar
  25. Kibria BMG, Saleh AKMdE (2011) Improving the estimators of the parameters of a probit regression model: a ridge regression approach. J Stat Plan Infererence 142(6):1421–1435MathSciNetCrossRefzbMATHGoogle Scholar
  26. Knight K, Fu W (2000) Asymptotic for lasso-type estimators. Ann Stat 28:1356–1378MathSciNetCrossRefzbMATHGoogle Scholar
  27. Li J, Palta M (2009) Bandwidth selection through cross-validation for semi-parametric varying-coefficient partially linear models. J Stat Comput Simul 79:1277–1286MathSciNetCrossRefzbMATHGoogle Scholar
  28. Li J, Zhang W, Wu Z (2011) Optimal zone for bandwidth selection in semiparametric models. J Nonparametric Stat 23:701–717MathSciNetCrossRefzbMATHGoogle Scholar
  29. McDonald GC, Galarneau DI (1975) A monte carlo evaluation of some ridge-type estimators. J Am Stat Assoc 70:407–416CrossRefzbMATHGoogle Scholar
  30. Müller M (2000) Semiparametric extensions to generalized linear models. Habilitationsschrift, BerlinGoogle Scholar
  31. Roozbeh M (2015) Shrinkage ridge estimators in semiparametric regression models. J Multivar Anal 136:56–74MathSciNetCrossRefzbMATHGoogle Scholar
  32. Roozbeh M, Arashi M (2013) Feasible ridge estimator in partially linear models. J Multivar Anal 116:35–44MathSciNetCrossRefzbMATHGoogle Scholar
  33. Saleh AKMdE (2006) Theory of preliminary test and stein-type estimation with applications. Wiley, New YorkCrossRefzbMATHGoogle Scholar
  34. Saleh AKMdE, Kibria BMG (1993) Performances of some new preliminary test ridge regression estimators and their properties. Commun Stat Ser A 22:2747–2764MathSciNetCrossRefzbMATHGoogle Scholar
  35. Saleh AKMdE, Kibria BMG (2011) On some ridge regression estimators: a nonparametric approach. J Nonparametric Stat 23:819–851MathSciNetCrossRefzbMATHGoogle Scholar
  36. Shao J, Deng X (2012) Estimation in high-dimensional linear models with deterministic design matrices. Ann Stat 40:812–831MathSciNetCrossRefzbMATHGoogle Scholar
  37. Wang H (2010) Forward regression for ultra-high dimensional variable screening. J Am Stat Assoc 104:1512–1524MathSciNetCrossRefzbMATHGoogle Scholar
  38. Wu J, Yang H (2016) More on the unbiased ridge regression estimation. Stat Pap 57:31–42MathSciNetCrossRefzbMATHGoogle Scholar
  39. Yuzbashi, B. and Ahmed, S.E. (2015). Shrinkage ridge regression estimators in high-dimensional linear models. In: Proceedings of the 9th international conference on management science and engineering management. doi: 10.1007/978-3-662-47241-5_67
  40. Zhang CH, Huang J (2008) The sparsity and bias of the LASSO selection in high-dimensional linear regression. Ann Stat 36:1567–1594MathSciNetCrossRefzbMATHGoogle Scholar
  41. Zhang C, Li J, Menga J (2008) On Stein’s lemma, dependent covariates and functional monotonicity in multi-dimensional modeling. J Multivar Anal 99:2285–2303MathSciNetCrossRefzbMATHGoogle Scholar
  42. Zhong Z, Yang Hu (2007) Ridge estimation to the restricted linear model. Commun Stat Theor Methods 36(11):2099–2115MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.Department of Statistics, School of Mathematical SciencesShahrood University of TechnologyShahroodIran
  2. 2.Department of Statistics, Faculty of Mathematics, Statistics and Computer SciencesSemnan UniversitySemnanIran

Personalised recommendations