Covariance

The covariance of $X$ and $Y$ is the Product of Moments about the mean

\begin{aligned} σ_{X Y} = c o v (X, Y) & = E ((X - μ_{X}) (Y - μ_{Y})) \\ = E (X Y) - E (X) μ_{Y} - μ_{X} E (Y) + μ_{X} μ_{Y} \\ = E (X Y) - μ_{X} μ_{Y} \\ = E (X Y) - E (X) E (Y) \end{aligned}

Which is useful for calculating the variance of independent variables

\begin{aligned} V a r (X + Y) & = E [(X + Y)^{2}] - [E (X + Y)]^{2} \\ = E (X^{2}) + 2 E (X Y) + E (Y^{2}) - ([E (X)]^{2} + 2 E (X Y) + [E (Y)]^{2}) \\ = V a r (X) + V a r (Y) + 2 E (X Y) - 2 E (X) E (Y) \\ V a r (X + Y) & = V a r (X) + V a r (Y) + 2 c o v (X, Y) \end{aligned}

Covariance measures the relation between the variables

Expectations are inversely related when covariance is negative, and directly related when it is positive
negative $\to$ positive change in $X \to$ negative change in $Y$