Consistency

$\hat{θ}$ is a consistent estimator of the parameter $θ$ if and only if $\exists ϵ > 0$

lim_{n \to \infty} P (| \hat{θ} - θ | < ϵ) = 1

the estimate $\hat{θ}$ converges to the true value of $θ$ as $n \to \infty$

The sample mean $\bar{X}$ is a consistent estimate of the population mean, as $V a r (\bar{X}) = \frac{σ^{2}}{n}$ . Applying Chebyshev's Theorem:

P (| \bar{X} - μ | < ϵ) \geq 1 - \frac{σ^{2}}{n ϵ^{2}} ⟶ 1 as n \to \infty

If $\hat{θ}$ is an unbiased estimator of the parameter $θ$ and the $V a r (\hat{θ}) \to 0$ as $n \to \infty$ , then $\hat{θ}$ is a consistent estimator of $θ$

like how $\frac{σ^{2}}{n} \to 0$ as $n \to \infty$

Consistent tests will have their power converge to 1 as $n \to \infty$

Examples

The sample variance $S^{2} = \frac{1}{n - 1} \sum_{i = 1}^{n} (X_{i} - \bar{X})^{2}$ is a consistent estimator of the population variance $σ^{2}$

\begin{align} \frac{(n-1)S^2}{\sigma^2} & \sim \huge\chi_{n-1}^2 \\ E\left( \frac{(n-1)S^2}{\sigma^2} \right) & =n-1 \\ E(S^2) & =\sigma^2 \\ Var\left( \frac{(n-1)S^2}{\sigma^2} \right)& = 2(n-1) \\ Var(S {#2} ) & =\frac{2\sigma^4}{n-1}\to 0 \text{ as } n\to \infty \end{align}

Looking at the minimum statistic $X_{(1)}$ for a double exponential distribution

f (x, δ) = e^{- (x - λ)}, x \geq δ

\begin{aligned} F_{X} (x) & = \int_{- \infty}^{\infty} f (x, δ) = \int_{δ}^{x} e^{- (t - δ)} d t \\ = 1 - \int_{x}^{\infty} e^{- (t - δ)} \\ = 1 - e^{- x - δ}, x > δ \\ 1 - F_{X} (x) & = e^{- x - δ} \\ f_{X_{(1)}} & = n [e^{- (x - δ)}]^{n - 1} e^{- (x - δ)} \\ = n e^{- n (x - δ)} \\ E (X_{(1)}) & = \int_{δ}^{\infty} x n e^{- n (x - δ)} d x \\ = \int_{0}^{\infty} (y + δ) n e^{- n y} \\ = \int_{0}^{\infty} y n e^{- n y} d y + δ \int_{0}^{\infty} n e^{- n y} \\ = E (Y) + δ \\ = \frac{1}{n} + δ \neq δ \end{aligned}

$E (X_{(1)}) = \frac{1}{n} + δ$ is asymptotically unbiased
To prove that it is a consistent estimator for $δ$

\begin{aligned} lim_{n \to \infty} P (| X_{(1)} - δ | < ϵ) & = P (- ϵ \leq X_{(1)} - δ \leq ϵ) \\ = P (δ - ϵ \leq X_{(1)} \leq δ + ϵ) \\ = P (δ \leq X_{(1)} \leq δ + ϵ) \\ = \int_{δ}^{δ + ϵ} n e^{- n (x - δ)} d x \\ = 1 - e^{- n ϵ} ⟶ 1 as n \to \infty \end{aligned}