Why is zero covariance imply zero correlation but not vice versa? If we go by the formula covariance(p,M) = correlation(p,M) * sigma(p) / sigma(M), wouldn't zero correlation mean zero covariance as well?
Hi @kevolution Per your formula, σ(12) = σ(1)* σ(2)*ρ(1,) which is classically important, zero correlation implies zero covariance, and indeed just as logical converse is true: zero covariance implies zero correlation. The key semantic distinction (at least for our purposes) is dependence versus independence. Independence is a key statistical assumption, proven when joint probability(X,Y) = P(X)*P(Y), but it's commonly violated in practice. If variables are not independent, they are (to some degree) dependent. But dependence encompasses an entire set of different relations, only one of which is linear dependence (i.e., correlation or covariance). My diagram below is maybe not the best Venn diagram ever but something like this:
So my Venn tries to convey:
Non-zero correlation or covariance = linear dependence
If independence --> correlation and covariance are zero (outside the circle is zero correlation including all of the yellow square)
If zero correlation, then we can't infer dependence or independence (could be either). I hope that's helpful!
... so independent variables do imply zero correlation (and zero covariance), but it is not true that zero correlation (or zero covariance) implies independence because there can be non-linear dependence. I hope that helps!
The 2nd formula is valid in all generality, it is the mathematical definition of the covariance. Note that it makes use of the expectation operator E[.], which might take different forms depending on the type of variable or properties of the (mathematical) space under consideration. The first formula gives the form of the expectation operator as it applies to discrete random variables. You'd see an integral sign rather than a sum for continuous variables. It is basically a question of broad mathematical definition versus how to apply the operator in practice given a particular type of random variable under consideration. They're both correct, one implies more context than the other. Hope this helps!