Efficiency

Suppose ${\hat{θ}}_{1}$ and ${\hat{θ}}_{2}$ are two unbiased estimator of a given populations' parameter $θ$

We say that ${\hat{θ}}_{1}$ is relatively more efficient than ${\hat{θ}}_{2}$ if

v a r ({\hat{θ}}_{1}) \leq v a r ({\hat{θ}}_{2})

The relative efficiency is used to measure the efficiency of ${\hat{θ}}_{2}$ relative to ${\hat{θ}}_{1}$

R E = \frac{v a r ({\hat{θ}}_{1})}{v a r ({\hat{θ}}_{2})}

Suppose $X_{1}, \dots, X_{n} \overset{i i d}{\sim} P o i s s o n (λ)$ , ${\hat{λ}}_{1} = \bar{X}$ , ${\hat{λ}}_{2} = S^{2} = \frac{1}{n - 1} V a r (\bar{X})$
Both are unbiased:

E (\bar{X}) = λ, E (S^{2}) = v a r (X) = λ

And the variances:

\begin{aligned} V a r (\bar{X}) & = \frac{λ}{n} & V a r (S^{2}) & = V a r (\frac{1}{n - 1} \sum (x_{i} - \bar{x})^{2}) \\ = \frac{n}{(n - 1)^{2}} V a r ((x_{i} - \bar{x})^{2}) \\ (no closed form) & = \frac{n}{(n - 1)^{2}} E [[(x_{i} - \bar{x}) - E (x_{i} - \bar{x}]^{2}] \end{aligned}

For some estimators, their variances are difficult to calculate, so we may focus on their lower bound to determine to find the most optimal one.

The efficiency of an unbiased estimator of $θ$ is the ratio of the CRLB of the variance of the estimator

just as how in Unbiasedness calculations we compare it to the mean

efficiency (\hat{θ}) = \frac{C R L B}{V a r (\hat{θ})} \leq 1

The best ratio is 1, where $V a r (\hat{θ}) = C R L B$

If $\hat{θ}$ is unbiased for $θ$ , and $lim_{n \to \infty} \frac{V a r (\hat{θ})}{C R L B} = 1$ , then $\hat{θ}$ is asymptotically efficient, as $lim_{n \to \infty} C R L B = 0$

If $\hat{θ}$ is an unbiased estimator for $θ$ ( $E (\hat{θ}) = θ$ ) and the variance of $\hat{θ}$ attains (is equal to) the value of the Cramér-Rao inequality, then $\hat{θ}$ is the uniformly minimum variance unbiased estimator (UMVUE) for $θ$ , and $\hat{θ}$ is optimally efficient

Comparing ${\hat{λ}}_{1}$ from the earlier example:

\begin{aligned} V a r ({\hat{λ}}_{1}) & = \frac{λ}{n} \\ C R L B & = \frac{1}{n E [{(\frac{\partial \ln e^{- λ} λ^{x} / x!}{\partial λ})}^{2}]} \\ = \frac{1}{n E [{(\frac{\partial - λ + x \ln λ - \ln (x!)}{\partial λ})}^{2}]} \\ = \frac{1}{n E [{(- 1 + \frac{x}{λ})}^{2}]} \\ = \frac{1}{n E [{(\frac{x - λ}{λ})}^{2}]} \\ = \frac{1}{n \frac{1}{λ^{2}} E [(x - λ)^{2}]} \\ = \frac{1}{n \frac{λ}{λ^{2}}} = \frac{λ}{n} \end{aligned}

So ${\hat{λ}}_{1}$ is the UMVUE for this estimator (best in this category)