# Covariance

Tags |
---|

# Covariance of two RV’s

Expectation of two random variables are defined as follows:

Covariance of two random variables are defined as follows:

## Important identities to know

- expectation is still linear

- $Var(X + Y) = Var(X) + Var(Y) + Cov(X, Y)$

- Independent $X, Y$ means that $Cov(X, Y) = 0$ ( but it's not a bidirectional relationship)

# Covariance matrix

A `covariance matrix`

is a matrix that contains pairs of random variable, and it is symmetric. It is defined as the following:

Furthermore, it is also positive semidefinite (partially because variances can't be negative)

## Covariance in expectation

Now, we can make a covariance matrix $\Sigma$ such that $\Sigma_{i, j} = Cov(X_i, X_j)$. This is how you deal with variances in vectors.

Using element analysis, we get two equivalent forms:

So you can see that vector covariance has the same form as scalar variance!

## 🚀Covariance Matrix is PSD

The covariance matrix is positive semi-definite, which can be helpful for derivations of convesity

## Proof (PSD definition that yields a norm inside expectation)