Covariance and linear independence
Although the covariance between two variables is not immediately interpretable, it is not as esoteric as it might appear at first glance. Indeed, we can understand the concept more clearly if we consider it in the context of the relationship between two binary variables. A binary variable is one that assumes a value of either zero or one. Binary variables are often used to measure simple dichotomies such a gender. For example, we might assign zeros to all the men and ones to all the women. It turns out that the equation for the covariance of two binary variables has a rather simple statistical interpretation. This interpretation obtains for two reasons. First, the mean of a binary variable is equal to its observed probability. Therefore, under the assumption of statistical independence, the product of the means of two binary variables is equal to their expected joint probability. Second, the average cross product of two binary variables is equal to their observed joint probability. Consequently, it can be shown that a covariance of two binary variables measures the extent to which the observed joint distribution of these variables differs from their expected joint distribution under the assumption that they are statistically independent.
KeywordsRegression Coefficient Inverse Relationship Binary Variable Column Vector Related Variable
Unable to display preview. Download preview PDF.