Least-squares regression and covariance


Most of regression analysis is based on least-squares estimates of the parameters of the linear regression equation. Although we have discussed some of the properties of the least-squares regression coefficient already, we have not presented any equations for computing this coefficient. It turns out that the least-squares regression coefficient is based on a quantity known as the covariance. The covariance between two variables is not especially interpretable, but we must become familiar with this quantity because it is central to much of statistical analysis. The generic equation for the covariance between two variables, x and y, is given as follows:
$$ \begin{gathered} Cov\left( {x,y} \right) = c_{xy} = \frac{1} {N}\left( {x - \bar x} \right)\prime \left( {y - \bar y} \right) \hfill \\ = \frac{1} {N}\Sigma \left( {x_i - \bar x} \right)\left( {y_i - \bar y} \right) \hfill \\ \end{gathered} $$
The covariance between two variables is simply the average product of the values of two variables that have been expressed as deviations from their respective means. It can be seen that this equation bears a striking resemblance to the equation for the variance of a variable. After all, a variance is simply the average squared value of a variable that has been expressed as a deviation from its mean.


Regression Coefficient Scalar Product Cross Product Average Product Linear Regression Equation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Plenum Press, New York 1997

Personalised recommendations