Abstract
Suppose we have the model \(\mathbf{y} =\boldsymbol{\theta } +\boldsymbol{\varepsilon }\), where \(\mathrm{E}[\boldsymbol{\varepsilon }] = \mathbf{0}\), \(\mathrm{Var}[\boldsymbol{\varepsilon }] =\sigma ^{2}\mathbf{I}_{n}\), and \(\boldsymbol{\theta }\in \varOmega\), a p-dimensional vector space. One reasonable estimate of \(\boldsymbol{\theta }\) would be the value \(\hat{\boldsymbol{\theta }}\), called the least squares estimate, that minimizes the total “error” sum of squares
subject to \(\boldsymbol{\theta }\in \varOmega\). A clue as to how we might calculate \(\hat{\boldsymbol{\theta }}\) is by considering the simple case in which y is a point P in three dimensions and Ω is a plane through the origin O. We have to find the point Q (\(=\hat{\boldsymbol{\theta }}\)) in the plane so that PQ 2 is a minimum; this is obviously the case when OQ is the orthogonal projection of OP onto the plane. This idea can now be generalized in the following theorem.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Atiqullah, M. (1962). The estimation of residual variance in quadratically balanced least squares problems and the robustness of the F test. Biometrika, 49, 83–91.
Cook, R. D., & Weisberg, S. (1982). Residuals and influence in regression. New York: Chapman & Hall.
Rao, C. R. (1952). Some theorems on minimum variance estimation. Sankhyā, 12, 27–42.
Seber, G. A. F. (2008). A matrix handbook for statisticians. New York: Wiley.
Seber, G. A. F., & Lee, A. J. (2003). Linear regression analysis (2nd ed.). New York: Wiley.
Seber, G. A. F., & Wild, C. J. (1989). Nonlinear regression. New York: Wiley. Also reproduced in paperback by Wiley in (2004).
Swindel, B. F. (1968). On the bias of some least-squares estimators of variance in a general linear model. Biometrika, 55, 313–316.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Seber, G.A.F. (2015). Estimation. In: The Linear Model and Hypothesis. Springer Series in Statistics. Springer, Cham. https://doi.org/10.1007/978-3-319-21930-1_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-21930-1_3
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-21929-5
Online ISBN: 978-3-319-21930-1
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)