The Linear Model and Hypothesis pp 27-45 | Cite as

# Estimation

Chapter

## Abstract

Suppose we have the model \(\mathbf{y} =\boldsymbol{\theta } +\boldsymbol{\varepsilon }\), where \(\mathrm{E}[\boldsymbol{\varepsilon }] = \mathbf{0}\), \(\mathrm{Var}[\boldsymbol{\varepsilon }] =\sigma ^{2}\mathbf{I}_{n}\), and \(\boldsymbol{\theta }\in \varOmega\), a subject to \(\boldsymbol{\theta }\in \varOmega\). A clue as to how we might calculate \(\hat{\boldsymbol{\theta }}\) is by considering the simple case in which

*p*-dimensional vector space. One reasonable estimate of \(\boldsymbol{\theta }\) would be the value \(\hat{\boldsymbol{\theta }}\), called the*least squares estimate*, that minimizes the total “error” sum of squares$$\displaystyle{SS =\sum _{ i=1}^{n}\varepsilon _{ i}^{2} =\parallel \mathbf{y}-\boldsymbol{\theta }\parallel ^{2}}$$

**y**is a point*P*in three dimensions and*Ω*is a plane through the origin*O*. We have to find the point*Q*(\(=\hat{\boldsymbol{\theta }}\)) in the plane so that*PQ*^{2}is a minimum; this is obviously the case when*OQ*is the orthogonal projection of*OP*onto the plane. This idea can now be generalized in the following theorem.## Keywords

Normal Equation Multiple Correlation Coefficient Sample Autocorrelation Linear Unbiased Estimate Vector Lagrange Multiplier
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

## References

- Atiqullah, M. (1962). The estimation of residual variance in quadratically balanced least squares problems and the robustness of the
*F*test.*Biometrika, 49*, 83–91.zbMATHMathSciNetCrossRefGoogle Scholar - Cook, R. D., & Weisberg, S. (1982).
*Residuals and influence in regression*. New York: Chapman & Hall.zbMATHGoogle Scholar - Rao, C. R. (1952). Some theorems on minimum variance estimation.
*Sankhyā, 12*, 27–42.zbMATHGoogle Scholar - Seber, G. A. F. (2008).
*A matrix handbook for statisticians*. New York: Wiley.Google Scholar - Seber, G. A. F., & Lee, A. J. (2003).
*Linear regression analysis*(2nd ed.). New York: Wiley.zbMATHCrossRefGoogle Scholar - Seber, G. A. F., & Wild, C. J. (1989).
*Nonlinear regression*. New York: Wiley. Also reproduced in paperback by Wiley in (2004).Google Scholar - Swindel, B. F. (1968). On the bias of some least-squares estimators of variance in a general linear model.
*Biometrika, 55*, 313–316.zbMATHMathSciNetCrossRefGoogle Scholar

## Copyright information

© Springer International Publishing Switzerland 2015