Moment Generating Functions

\begin{aligned} M_{X} (t) = E (e^{t X}) = \sum_{x} e^{t x} f (x) & discrete \\ M_{X} (t) = E (e^{t X}) = \int_{- \infty}^{\infty} e^{t x} f (x) d x & continuous \end{aligned}

The function can be used to find an specific moment:

μ_{r}^{'} = E (X^{r}) = {\frac{d^{r} M_{X} (t)}{d t^{r}} |}_{t = 0}

as we want to evaluate at $t = 0$ , we handle the function considering t to be a very small value, $t \in (- δ, δ)$

Using the series expansion for $e^{t x}$ we arrive at a form for the function that resembles a Taylor Series

m (t) = 1 + t μ_{1}^{'} + \frac{t^{2}}{2!} μ_{2}^{'} + \frac{t^{3}}{3!} μ_{3}^{'} + \dots

Taking the derivative removes the unwanted terms from the left, evaluating at $t = 0$ eliminates everything from the right

If $a$ and $b$ are constants:

\begin{aligned} 1. & M_{X + a} (t) = E [e^{(X + a) t}] = e^{a t} \cdot M_{X} (t) \\ 2. & M_{b X} (t) = E [e^{b X t}] = M_{X} (b t) \\ 3. & M_{\frac{X + a}{b}} (t) = E [e^{(\frac{x + a}{b})} t] = E [e^{\frac{a}{b} t}] M_{X} (\frac{t}{b}) \end{aligned}

The problem with MGFs is that they don't always exist

sum/integral may not converge for the expectation
In that case, the characteristic function

φ_{X} (t) = E (e^{i t X}), i = \sqrt{- 1}

always exists for any distribution

Properties

If $X$ and $Y$ have the same MGF $\to$ they must have the same pdf

Suppose $X \sim f (x)$ , if $M_{X} (t)$ exists then $M_{X} (t)$ and $f (x)$ have a one-to-one correspondence
Suppose $X$ and $Y$ are two random variables such that

\begin{aligned} M_{X} (t) & ⟶ M_{Y} (t) \\ then \\ f_{X} (x) & ⟶ f_{Y} (y) \forall x, \forall y \end{aligned}

If $Y = a X + b$ then

M_{Y} (t) = e^{b t} M_{X} (a t)

If $X$ and $Y$ are independent then

M_{X + Y} (t) = M_{X} (t) M_{Y} (t)

$X \sim b i n o m (n, p)$

\begin{aligned} M_{X} (t) = E (e^{x t}) & = \sum e^{x t} f (x) \\ = \sum_{x = 0}^{n} e^{x t} (\binom{n}{x}) p^{x} (1 - p)^{n - x} \\ = \sum_{x = 0}^{n} (\binom{n}{x}) (e^{t} p)^{x} (1 - p)^{n - x} (binomial expansion) \\ = [e^{t} p + (1 - p)]^{n} = [1 + p (e^{t} - 1)]^{n} \end{aligned}

F M_{X} (t) = E (t^{X}) = \sum_{x} t^{x} f (x)

rth factorial moment

μ_{(r)}^{'} = E [X (X - 1) (X - 2) \dots (X - r + 1)] = {\frac{d^{r}}{d t^{r}} F M_{X} (t) |}_{t = 1}

\begin{aligned} M_{X} (t) = E (e^{x t}) & = \sum_{x = 0}^{\infty} e^{x t} \frac{e^{- λ} λ^{x}}{x!} \\ e^{e^{t} λ} is a constant & = e^{- λ} e^{e^{t} λ} \sum_{x = 0}^{\infty} \frac{e^{- e^{t} λ} (e^{t} λ)^{x}}{x!} \\ = e^{- λ} e^{e^{t} λ} \cdot 1 (pmf of poisson) \\ = e^{λ (e^{t} - 1)} \end{aligned}

\begin{aligned} M_{X} (t) = E (e^{t X}) & = \int_{- \infty}^{\infty} e^{t x} \frac{1}{β^{α} Γ (α)} x^{α - 1} e^{- x / β} d x \\ = \frac{1}{β^{α} Γ (α)} \int_{0}^{\infty} e^{t x - x / β} x^{α - 1} d x |_{t = 0} \\ = \frac{1}{β^{α} Γ (α)} \int_{0}^{\infty} e^{- x (1 / β - t)} x^{α - 1} d x (u = x (\frac{1}{β} - t)) \\ = \frac{1}{β^{α} Γ (α)} \int_{0}^{\infty} e^{- u} {(\frac{u}{\frac{1}{β} - t})}^{α - 1} \frac{d u}{\frac{1}{β} - t} (x = \frac{u}{\frac{1}{β} - t}, d x = \frac{d u}{(\frac{1}{β} - t)}) \\ = \frac{1}{β^{α} Γ (α)} {(\frac{1}{\frac{1}{β} - t})}^{α} \int_{0}^{\infty} e^{- u} u^{α - 1} d u \\ = {(\frac{1}{β} - t)}^{- α} \frac{1}{β^{α} Γ (α)} Γ (α) \\ = {((\frac{1}{β} - t) β)}^{- α} \\ M_{X} (t) & = (1 - t β)^{- α} \end{aligned}

\begin{aligned} M_{X} (t) = E (e^{t X}) & = \int_{0}^{\infty} e^{t x} \frac{1}{λ} e^{- x / λ} d x \\ = \frac{1}{λ} \int_{0}^{\infty} e^{- (1 / λ - t) x} d x \\ = \frac{1}{λ} \frac{1}{\frac{1}{λ} - t} \int_{0}^{\infty} (\frac{1}{λ} - t) e^{- (1 / λ - t) x} d x \\ = \frac{1}{1 - λ t} \cdot 1 (pdf of \exp (0)) \end{aligned}