3100 Cheat Sheet

Discrete

F (x, y) = P (X \leq x, Y \leq y) = \sum_{s \leq x} \sum_{t \leq y} f (s, t) for x, y \in R

Marginals

$f_{X} (x) = P (x = x) = \sum_{y} f (x, y), f_{Y} (y) = P (Y = y) = \sum_{x} f (x, y)$
For $n$ discrete random variables $X_{1}, X_{2}, \dots, X_{n}$ with a joint probability distribution $f (x_{1}, x_{2}, \dots, x_{n})$ and marginals distributions $f_{X_{i}} (x_{i})$ :

f_{X} (x_{1}, x_{2}, \dots x_{n}) = f_{X_{1}} (x_{1}) \cdot f_{X_{2}} (x_{2}) \dots f_{X_{n}} (x_{n}) \forall (x_{1}, x_{2}, \dots, x_{n})

if and only if the $n$ random variables are independent
uniform

f (x) = \frac{1}{k}, μ = \frac{n + 1}{2}, σ^{2} = \frac{n^{2} - 1}{12}

Bernoulli

f (x) = p^{x} (1 - p)^{1 - x}, μ = p, σ^{2} = p (1 - p)

binomial

f (x) = (\binom{n}{x}) p^{x} (1 - p)^{n - x}, μ = n p, σ^{2} = n p (1 - p), M_{X} (t) = [1 + p (e^{t} - 1)]^{n}

negative binomial (geometric when k =1)

f (x; k, p) = (\binom{x - 1}{k - 1}) p^{k} (1 - p)^{x - k}, μ = \frac{k}{p}, σ^{2} = \frac{k}{p} (\frac{1}{p} - 1)

hypergeometric (sampling without replacement)

f (x; n, N, M) = \frac{(\begin{matrix} M \\ x \end{matrix}) (\begin{matrix} N - M \\ n - x \end{matrix})}{(\begin{matrix} N \\ n \end{matrix})}, μ = \frac{n M}{N}, σ^{2} = \frac{n M (N - M) (N - n)}{N^{2} (N - 1)}

poisson

f(x;\lambda)=\frac{\lambda {#xe} ^{-\lambda}}{x!}, \mu =\sigma^2=\lambda, \quad M_{X}(t)=e^{\lambda(e^t-1)}

Continuous
Joint Cumulative Density

F (X, Y) = P (X \leq x, Y \leq y) = \int_{- \infty}^{y} \int_{- \infty}^{x} f (s, t) d s d t x, y \in (- \infty, \infty)

f (x, y) = \frac{\partial^{2}}{\partial x \partial y} F (x, y)

marginals f_{X} (x) = \int_{- \infty}^{\infty} f (x, y) d y a n d f_{Y} (y) = \int_{- \infty}^{\infty} f (x, y) d x

For $n$ continuous random variables $X_{1}, X_{2}, \dots, X_{n}$ with a joint probability densities $f (x_{1}, x_{2}, \dots, x_{n})$ and marginals densities $f_{X_{i}} (x_{i})$ :

f_{X} (x_{1}, x_{2}, \dots x_{n}) = f_{X_{1}} (x_{1}) \cdot f_{X_{2}} (x_{2}) \dots f_{X_{n}} (x_{n}) \forall (x_{1}, x_{2}, \dots, x_{n})

if and only if the $n$ random variables are independent
uniform

f (x; α, β) = \frac{1}{β - α}, μ = \frac{α + β}{2}, σ^{2} = \frac{1}{12} (α - β)^{2}

gamma function (continuous factorial)

Γ (α) = \int_{0}^{\infty} y^{α - 1} e^{- y} d y, Γ (\frac{1}{2}) = \sqrt{π}

Gamma Distribution

f (x; α, β) = \frac{1}{β^{α} Γ (α)} x^{α - 1} e^{- x / β}, μ = β α, σ^{2} = β^{2} α

Exponential Distribution is a gamma distribution when $α = 1$
The Chi-Squared Distribution is a gamma distribution with $α = \frac{v}{2}$ and $β = 2$ , $v$ = df
beta distribution

f (x; α, β) = \frac{Γ (α + β)}{Γ (α) Γ (β)} x^{α - 1} (1 - x)^{β - 1}, μ = \frac{α}{α + β}, σ^{2} = \frac{α β}{(α + β)^{2} (α + β + 1)}

normal distribution

f (x; μ, σ^{2}) = \frac{1}{\sqrt{2 π σ^{2}}} e^{- (x - μ)^{2} / 2 σ^{2}}, M_{X} (t) = e^{μ t + \frac{1}{2} σ^{2} t^{2}}

If $X \sim B i n (n, p)$ then the moment-generating function of

Z = \frac{X - n p}{\sqrt{n p (1 - p)}} = \frac{X - μ}{σ}

P (A | B) = \frac{P (B | A) P (A)}{P (B)} = \frac{P (A \cap B)}{P (B)}

Conditional

\begin{aligned} f_{X | Y} (x | y) = P (A | B) & = \frac{P (A \cap B)}{P (B)} = \frac{f (x, y)}{f_{Y} (y)} \\ E [g (X) | Y = y] & = \sum_{x} g (x) f_{X | Y} (x | y) \\ E [g (X) | Y = y] & = \int_{- \infty}^{\infty} g (x) f_{X | Y} (x | y) \\ V a r (X | Y = y) & = E (X^{2} | Y = y) - [E (X | Y = y)]^{2} \end{aligned}

Expectation

\begin{aligned} E [g (X, Y)] & = \int_{- \infty}^{\infty} \int_{- \infty}^{\infty} g (x, y) \cdot f_{X, Y} (x, y) d x \\ E [g (X, Y)] & = \sum_{x} \sum_{y} g (x, y) \cdot f_{X, Y} (x, y) \end{aligned}

properties

\begin{aligned} E [\sum_{i = 1}^{n} c_{i} g_{i} (X)] & = \sum_{i = 1}^{n} c_{i} E [g_{i} (x)] \\ E [(a X + b)^{n}] & = \sum_{i = 0}^{n} (\binom{n}{i}) a^{n - i} b^{i} E (X^{n - 1}) \\ E [a X + b] & = a E [X] + b \\ var (a X + b) & = a^{2} [E (X^{2}) - [E (X)]^{2}] = a^{2} σ^{2} = a^{2} var (X) \end{aligned}

Moments

\begin{aligned} μ_{r}^{'} = E (X^{r}) = \sum_{x} x^{r} f (x) & (d i s c r e t e) \\ μ_{r}^{'} = E (X^{r}) = \int_{- \infty}^{\infty} x^{r} f (x) d x & (c o n t i n u o u s) \end{aligned}

Central moments

\begin{aligned} μ_{r} & = E [(X - μ)^{r}] = \sum_{x} (x - μ)^{r} \cdot f (x) & (d i s c r e t e) \\ μ_{r} & = E [(X - μ)^{r}] = \int_{- \infty}^{\infty} (x - μ)^{r} \cdot f (x) d x & (c o n t i n u o u s) \\ var (X) & = σ^{2} = μ_{2} = E [(X - μ)^{2}] = E (X^{2}) - [E (X)]^{2} \end{aligned}

Moment generating functions are a bijection between functions

\begin{aligned} M_{X} (t) = E (e^{t X}) & = \sum_{x} e^{t x} f (x) & discrete \\ M_{X} (t) = E (e^{t X}) & = \int_{- \infty}^{\infty} e^{t x} f (x) d x & continuous \\ μ_{r}^{'} = E (X^{r}) & = {\frac{d^{r} M_{X} (t)}{d t^{r}} |}_{t = 0} \end{aligned}

\begin{aligned} 1. & M_{X + a} (t) = E [e^{(X + a) t}] = e^{a t} \cdot M_{X} (t) \\ 2. & M_{b X} (t) = E [e^{b X t}] = M_{X} (b t) \\ 3. & M_{\frac{X + a}{b}} (t) = E [e^{(\frac{x + a}{b})} t] = E [e^{\frac{a}{b} t}] M_{X} (\frac{t}{b}) \end{aligned}

Product of Moments

\begin{aligned} μ_{r, s}^{'} & = \sum_{x} \sum_{y} x^{r} y^{s} \cdot f_{X, Y} (x, y) = E [X^{r} Y^{s}] \\ μ_{r, s}^{'} & = \int_{- \infty}^{\infty} \int_{- \infty}^{\infty} x^{r} y^{s} \cdot f_{X, Y} (x, y) d x d y \\ P (| X - μ | < k σ) & \geq 1 - \frac{1}{k^{2}}, σ \neq 0 \end{aligned}

The covariance of $X$ and $Y$ is the Product of Moments about the mean, expectations are inversely related when covariance is negative, and directly related when it is positive

μ_{1, 1} = σ_{X Y} = c o v (X, Y) = E ((X - μ_{X})^{1} (Y - μ_{Y})^{1}) = E (X Y) - E (X) E (Y)

If $X$ and $Y$ are independent:

$E (X Y) = E (X) E (Y), c o v (X, Y) = σ_{X Y} = 0$

If $A_{1}, A_{2}, A_{3}, \dots A_{n}$ are in a sample space such that $P (A_{1}) \neq 0, P (A_{1} A_{2}) \neq 0, \dots P (A_{1} A_{2} \dots A_{n - 1}) \neq 0$ then $P (A_{1} A_{2} \dots A_{n}) = P (A_{1}) P (A_{2} | A_{1}) P (A_{3} | A_{1} A_{2}) \dots P (A_{n} | A_{1} A_{2} \dots A_{n - 1})$
If $A$ and $B$ are independent $\to (A$ and $\overset{―}{B})$ and $(\overset{―}{A}$ and $B)$ are also independent
If the sample space S can be partitioned into events $B_{1}, B_{2}, \dots B_{k}$ and $P (B_{k}) \neq 0$ $\forall i = 1, 2 \dots k$
Then for any event A in S $P (A) = \sum_{i = 1}^{k} P (A | B_{i}) P (B_{i})$
Independence
If $X$ and $Y$ are independent:

\begin{aligned} f (x, y) & = P (X = x, Y = y) \\ = P (X = x) P (Y = y) \\ f (x, y) & = f_{X} (x) f_{Y} (y) \forall (x,y) \end{aligned}

\begin{aligned} F (x, y) & = P (X \leq x, Y \leq y) \\ = P (X \leq x) P (Y \leq y) \\ = F_{X} (x) F_{Y} (y) \forall (x,y) \end{aligned}

The cdfs can be used instead to determine the dependency of the variables

\begin{aligned} f (x, y) & = f_{X} (x) f_{Y} (y) \\ F (x, y) & = F_{X} (x) F_{Y} (y) \end{aligned}