Advertisement

# Estimation

• George A. F. Seber
Part of the Springer Series in Statistics book series (SSS)

## Abstract

Suppose we have the model $$\mathbf{y} =\boldsymbol{\theta } +\boldsymbol{\varepsilon }$$, where $$\mathrm{E}[\boldsymbol{\varepsilon }] = \mathbf{0}$$, $$\mathrm{Var}[\boldsymbol{\varepsilon }] =\sigma ^{2}\mathbf{I}_{n}$$, and $$\boldsymbol{\theta }\in \varOmega$$, a p-dimensional vector space. One reasonable estimate of $$\boldsymbol{\theta }$$ would be the value $$\hat{\boldsymbol{\theta }}$$, called the least squares estimate, that minimizes the total “error” sum of squares
$$\displaystyle{SS =\sum _{ i=1}^{n}\varepsilon _{ i}^{2} =\parallel \mathbf{y}-\boldsymbol{\theta }\parallel ^{2}}$$
subject to $$\boldsymbol{\theta }\in \varOmega$$. A clue as to how we might calculate $$\hat{\boldsymbol{\theta }}$$ is by considering the simple case in which y is a point P in three dimensions and Ω is a plane through the origin O. We have to find the point Q ($$=\hat{\boldsymbol{\theta }}$$) in the plane so that PQ2 is a minimum; this is obviously the case when OQ is the orthogonal projection of OP onto the plane. This idea can now be generalized in the following theorem.

## Keywords

Normal Equation Multiple Correlation Coefficient Sample Autocorrelation Linear Unbiased Estimate Vector Lagrange Multiplier
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

## References

1. Atiqullah, M. (1962). The estimation of residual variance in quadratically balanced least squares problems and the robustness of the F test. Biometrika, 49, 83–91.
2. Cook, R. D., & Weisberg, S. (1982). Residuals and influence in regression. New York: Chapman & Hall.
3. Rao, C. R. (1952). Some theorems on minimum variance estimation. Sankhyā, 12, 27–42.
4. Seber, G. A. F. (2008). A matrix handbook for statisticians. New York: Wiley.Google Scholar
5. Seber, G. A. F., & Lee, A. J. (2003). Linear regression analysis (2nd ed.). New York: Wiley.
6. Seber, G. A. F., & Wild, C. J. (1989). Nonlinear regression. New York: Wiley. Also reproduced in paperback by Wiley in (2004).Google Scholar
7. Swindel, B. F. (1968). On the bias of some least-squares estimators of variance in a general linear model. Biometrika, 55, 313–316.

## Copyright information

© Springer International Publishing Switzerland 2015

## Authors and Affiliations

• George A. F. Seber
• 1
1. 1.Department of StatisticsThe University of AucklandAucklandNew Zealand