Linear Regression Models

Linear regression and the closely related linear prediction theory are widely used statistical tools in empirical finance and in many other fields. Because of the diversity of applications, introductory courses in linear regression usually focus on the mathematically simplest setting, which also occurs in many other applications. In this setting, the regressors are often assumed to be nonrandom vectors. In Sections 1.1–1.4, we follow this “standard” treatment of least squares estimates of the parameters in linear regression models with nonrandom regressors, for which the means, variances, and covariances of the least squares estimates can be easily derived by making use of matrix algebra. For nonrandom regressors, the sampling distribution of the least squares estimates can be derived by making use of linear transformations when the random errors in the regression model are independent normal, and application of the central limit theorem then yields asymptotic normality of the least squares estimates for more general independent random errors. These basic results on means, variances, and distribution theory lead to the standard procedures for constructing tests and confidence intervals described in Section 1.2 and variable selection methods and regression diagnostics in Sections 1.3 and 1.4. The assumption of nonrandom regressors, however, is violated in most applications to finance and other areas of economics, where the regressors are typically random variables some of which are sequentially determined input variables that depend on past outputs and inputs.