Statistics and Computing

, Volume 29, Issue 1, pp 33–42 | Cite as

Best linear estimation via minimization of relative mean squared error

  • Lin Su
  • Howard D. BondellEmail author


We propose methods to construct a biased linear estimator for linear regression which optimizes the relative mean squared error (MSE). Although there have been proposed biased estimators which are shown to have smaller MSE than the ordinary least squares estimator, our construction is based on the minimization of relative MSE directly. The performance of the proposed methods is illustrated by a simulation study and a real data example. The results show that our methods can improve on MSE, particularly when there exists correlation among the predictors.


Biased linear estimator Smallest relative mean squared error Ridge regression Ordinary least squares 


  1. Akaike, H.: A new look at the statistical model identification. IEEE Trans. Automat. Control 19(6), 716–723 (1974)MathSciNetCrossRefzbMATHGoogle Scholar
  2. Golub, G.H., Heath, M., Wahba, G.: Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21(2), 215–223 (1979)MathSciNetCrossRefzbMATHGoogle Scholar
  3. Grant, M., Boyd, S.: Graph implementations for nonsmooth convex programs. In Blondel, V., Boyd, S., Kimura, H., (eds.) Recent Advances in Learning and Control, Lecture Notes in Control and Information Sciences, pp. 95–110. Springer (2008).
  4. Grant, M., Boyd, S.: CVX: Matlab software for disciplined convex programming, version 2.1. (2014)
  5. Hirst, J.D., King, R.D., Sternberg, M.J.: Quantitative structure-activity relationships by neural networks and inductive logic programming. i. the inhibition of dihydrofolate reductase by pyrimidines. J. Comput. Aided Mol. Des. 8(4), 405–420 (1994)CrossRefGoogle Scholar
  6. Hoerl, A.E., Kennard, R.W.: Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12(1), 55–67 (1970)CrossRefzbMATHGoogle Scholar
  7. Janson, L., Fithian, W., Hastie, T.J.: Effective degrees of freedom: a flawed metaphor. Biometrika 102(2), 479–485 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  8. Liu, K.: A new class of blased estimate in linear regression. Commun. Stat. Theory Methods 22(2), 393–402 (1993)CrossRefGoogle Scholar
  9. Schwarz, G., et al.: Estimating the dimension of a model. Ann. Stat. 6(2), 461–464 (1978)MathSciNetCrossRefzbMATHGoogle Scholar
  10. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B (Methodological) 58(1), 267–288 (1996)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2017

Authors and Affiliations

  1. 1.NC State UniversityRaleighUSA

Personalised recommendations