Biased Estimation

Abstract

One purpose of variable selection is to reduce multicollinearity, although, as we noted in Section 11.2, reducing the number of independent variables can lead to bias. Obviously, the general principle is that it might be preferable to trade off a small amount of bias in order to substantially reduce the variances of the estimates of β. There are several other methods of estimation which are also based on trading off bias for variance. This chapter describes three of these: principal component regression, ridge regression and the shrinkage estimator.