If as estimator $\hat{\theta }(X_1,\dots ,X_n)$ provides all information that the sample contains for estimating $\theta$, $\hat{\theta }$ is said to be sufficient for $\theta$

• each observation provides information about the value of $\theta$

We determine the sufficiency by determining if the conditional joint distribution of $X_1,\dots ,X_n$ given the estimator $\hat{\theta }$ depends on the parameter $\theta$ or not

If $f(X_1=x_1,\dots ,X_n=x_n|\hat{\theta })$ depends on $\theta$, the sample values of $x_1,x_2,\dots ,x_n$ would provide additional information of $\theta$ (given $\hat{\theta }$), so $\hat{\theta }$ is not sufficient enough

If it is independent on $\theta$, then (given $\hat{\theta }$) $\theta$ does not provide any more information not already contained in $\hat{\theta }$

• not a function of $\theta$

Factorization Theorem

Factorize the joint into two parts to show the sufficiency of an estimator without (a lot of work)

$\hat{\theta }$ is a sufficient estimator of the parameter $\theta$ if and only if the joint distribution/density of the random sample can be factorized as:

where $g(\hat{\theta },\theta )$ depends only on $\hat{\theta }$ and $\theta$, and $h(x_1,\dots ,x_n)$ does not depend on $\theta$

We can use the dependence of the sample space on the parameter $\theta$ to show the sufficiency of a estimator

examples

$X_1,X_2,X_3\sim Be\mathrm{rnoulli}(\mathrm{p})$
$\hat{p}=\frac{1}{6}(X_1+2X_2+3X_3)$

Let $x_1=1,x_2=1,x_3=0$ be a realization of the sample, then $\hat{p}=\frac{1}{2}$

$\hat{p}$ is not sufficient for p

factorization theorem

$\sim N(\mu ,{\sigma }^2)$
$\overline{X}=\frac{1}{n}\sum X_i$