• George A. F. Seber
Part of the Springer Series in Statistics book series (SSS)


Suppose we have the model \(\mathbf{y} =\boldsymbol{\theta } +\boldsymbol{\varepsilon }\), where \(\mathrm{E}[\boldsymbol{\varepsilon }] = \mathbf{0}\), \(\mathrm{Var}[\boldsymbol{\varepsilon }] =\sigma ^{2}\mathbf{I}_{n}\), and \(\boldsymbol{\theta }\in \varOmega\), a p-dimensional vector space. One reasonable estimate of \(\boldsymbol{\theta }\) would be the value \(\hat{\boldsymbol{\theta }}\), called the least squares estimate, that minimizes the total “error” sum of squares
$$\displaystyle{SS =\sum _{ i=1}^{n}\varepsilon _{ i}^{2} =\parallel \mathbf{y}-\boldsymbol{\theta }\parallel ^{2}}$$
subject to \(\boldsymbol{\theta }\in \varOmega\). A clue as to how we might calculate \(\hat{\boldsymbol{\theta }}\) is by considering the simple case in which y is a point P in three dimensions and Ω is a plane through the origin O. We have to find the point Q (\(=\hat{\boldsymbol{\theta }}\)) in the plane so that PQ2 is a minimum; this is obviously the case when OQ is the orthogonal projection of OP onto the plane. This idea can now be generalized in the following theorem.


Normal Equation Multiple Correlation Coefficient Sample Autocorrelation Linear Unbiased Estimate Vector Lagrange Multiplier 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Atiqullah, M. (1962). The estimation of residual variance in quadratically balanced least squares problems and the robustness of the F test. Biometrika, 49, 83–91.zbMATHMathSciNetCrossRefGoogle Scholar
  2. Cook, R. D., & Weisberg, S. (1982). Residuals and influence in regression. New York: Chapman & Hall.zbMATHGoogle Scholar
  3. Rao, C. R. (1952). Some theorems on minimum variance estimation. Sankhyā, 12, 27–42.zbMATHGoogle Scholar
  4. Seber, G. A. F. (2008). A matrix handbook for statisticians. New York: Wiley.Google Scholar
  5. Seber, G. A. F., & Lee, A. J. (2003). Linear regression analysis (2nd ed.). New York: Wiley.zbMATHCrossRefGoogle Scholar
  6. Seber, G. A. F., & Wild, C. J. (1989). Nonlinear regression. New York: Wiley. Also reproduced in paperback by Wiley in (2004).Google Scholar
  7. Swindel, B. F. (1968). On the bias of some least-squares estimators of variance in a general linear model. Biometrika, 55, 313–316.zbMATHMathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • George A. F. Seber
    • 1
  1. 1.Department of StatisticsThe University of AucklandAucklandNew Zealand

Personalised recommendations