Beyond ordinary regression analysis

Abstract

In response to the various problems that arise whenever the assumptions required by ordinary least-squares estimation procedures cannot be met, statisticians have developed a series of special estimation procedures. It will be recalled that most of the problems associated with violations of these assumptions involve biases in the standard errors of the parameters of the regression model. In other words, we can rely on ordinary least-squares estimation procedures to provide us with valuable information about the direction and magnitude of the parameters of the regression model, even if we cannot rely on the estimated standard errors of these parameter estimates for purposes of statistical inference. However, if we wish to make valid statistical inferences, based on unbiased estimates of the standard errors of these parameter estimates, we must be prepared to employ special estimation procedures that are appropriate to the problem at hand. Most of these special estimation procedures yield what are known as “consistent“ estimates of the parameters of the regression model. Strictly speaking, consistent estimates are not unbiased, but their bias is relatively small in large samples. Moreover, these consistent estimates have relatively large standard errors, even if these estimates of the standard errors are unbiased. The details of these special estimation procedures can be found in advanced statistical and econometric textbooks.