Review of Ordinary Least Squares and Generalized Least Squares

Abstract

The purpose of this chapter is to review the fundamentals of ordinary least squares and generalized least squares in the context of linear regression analysis. The presentation here is somewhat condensed given our objective of focusing on more advanced topics in econometrics. The results presented, though brief in form, are important and are the foundation for much to come. In the next section we present the assumptions of the classical linear regression model. In the following section the Gauss-Markov theorem is proved and the optimality of the ordinary least squares estimator is established. In Section 2.4 we introduce the large sample concepts of convergence in probability and consistency. It is shown that convergence in quadratic mean is a sufficient condition for consistency and that the ordinary least squares estimator is consistent. In Section 2.5 the generalized least squares model is defined and the optimality of the generalized least squares estimator is established by Aitken’s theorem. In the next section we examine the properties of the ordinary least squares estimator when the appropriate model is the generalized least squares model. Finally, in Section 2.7 we summarize our discussion and briefly outline additional results and readings that are available.