# Least-squares regression and covariance

## Abstract

Most of regression analysis is based on least-squares estimates of the parameters of the linear regression equation. Although we have discussed some of the properties of the least-squares regression coefficient already, we have not presented any equations for computing this coefficient. It turns out that the least-squares regression coefficient is based on a quantity known as the covariance. The covariance between two variables is not especially interpretable, but we must become familiar with this quantity because it is central to much of statistical analysis. The generic equation for the covariance between two variables, x and y, is given as follows:
$$\begin{gathered} Cov\left( {x,y} \right) = c_{xy} = \frac{1} {N}\left( {x - \bar x} \right)\prime \left( {y - \bar y} \right) \hfill \\ = \frac{1} {N}\Sigma \left( {x_i - \bar x} \right)\left( {y_i - \bar y} \right) \hfill \\ \end{gathered}$$
The covariance between two variables is simply the average product of the values of two variables that have been expressed as deviations from their respective means. It can be seen that this equation bears a striking resemblance to the equation for the variance of a variable. After all, a variance is simply the average squared value of a variable that has been expressed as a deviation from its mean.

## Keywords

Regression Coefficient Scalar Product Cross Product Average Product Linear Regression Equation

## Preview

Unable to display preview. Download preview PDF.