Interval estimation for mean

Let $(θ_{l}, θ_{u})$ be a random interval. With an appropriate probability $1 - α$ (confidence coefficient), we want to find values of $θ_{l}$ and $θ_{u}$ such that

P (θ_{l} \leq θ \leq θ_{u}) = 1 - α

We refer to $(θ_{l}, θ_{u})$ as a $100 (1 - α) %$ confidence interval (CI) for $θ$

Interpretation: on repeating sampling, there will be $k$ CI's for $θ$ , and $θ$ will be covered by these CI's about $100 (1 - α) %$ of the time

from all of those $k$ CI's, 95% of them will get it right

To determine the confidence limits $θ_{l}, θ_{u}$ , we use a statistic with a known distribution/parameters (Pivot Statistic)

If $\bar{x}$ is the sample mean of a random sample from $N (μ, σ^{2})$ and $σ^{2}$ is known, then

C I = (\bar{x} - Z_{1 - α / 2}, \bar{x} + Z_{1 - α / 2}) w r o n g

If $θ$ is a location parameter, then the interval estimate usually involves a difference

$μ$ indicates the location for $N (μ, σ^{2})$
If $θ$ is a scale parameter, then the interval estimate usually involves a ratio
$σ^{2}$ indicates the spread of a $N (μ, σ^{2})$

The MLE or a sufficient statistic is often a good place to start for finding $θ_{l}$ and $θ_{u}$

Examples

Knowing $σ^{2}$ from a normal distribution, but wanting to estimate the $μ$ :

\begin{aligned} \hat{μ} & = \bar{x} = \sum \frac{x_{i}}{n} \\ Z & = \frac{x - μ}{σ / \sqrt{n}} \sim N (0, 1), (a pivot with known params) \\ P (μ_{l} \leq μ \leq μ_{u}) & = 1 - α \\ P (\frac{\bar{x} - μ_{u}}{σ / \sqrt{n}} \leq \frac{\bar{x} - μ}{σ / \sqrt{n}} \leq \frac{\bar{x} - μ_{l}}{σ / \sqrt{n}}) & = 1 - α \\ P (Z_{1 - \frac{α}{2}}) \\ {\hat{μ}}_{l} & = \bar{x} - 1.96 \frac{σ}{\sqrt{n}} \\ {\hat{μ}}_{u} & = \bar{x} + 1.96 \frac{σ}{\sqrt{n}} \end{aligned}

$σ^{2}$ unknown

\begin{aligned} \hat{μ} = \bar{x}, σ^{2} = s^{2} = \frac{1}{n - 1} \sum (x_{i} - \bar{x})^{2} \\ t = \frac{\bar{x} - μ}{s / \sqrt{n}} \sim t_{n - 1} t-statistic is a pivot statistic \\ P (μ_{l} \leq μ \leq μ_{u}) = 1 - α \\ P (\frac{\bar{x} - μ}{s / \sqrt{n}} \leq \frac{\bar{x} - μ}{s / \sqrt{n}} \leq \frac{\bar{x} - μ}{s / \sqrt{n}}) = 1 - α \\ C I = (\bar{x} - t_{1 - α / 2} \frac{s}{\sqrt{n}}, \bar{x} + t_{1 - α / 2} \frac{s}{\sqrt{n}}) \end{aligned}

By CLT:

\begin{aligned} Z = \frac{\bar{x} - μ}{\sqrt{V a r (\bar{x})}} = \frac{\bar{x} - μ}{σ / \sqrt{n}}, Z \sim N (0, 1) approx for big n \geq 25 \\ Example \\ X_{1}, \dots, X_{n} \sim B i n (N, p), C I for μ and p : \\ μ = E (X) = n p, V a r (X) = σ^{2} = n p (1 - p) \\ \hat{μ} = \bar{x} = \sum \frac{x_{i}}{n}, \hat{p} = \frac{\hat{μ}}{N} = \frac{\bar{x}}{N}, E (\hat{μ}) = E (\bar{x}) = μ \\ V a r (\hat{μ}) = V a r (\bar{X}) = \frac{1}{n^{2}} n V a r (X) = \frac{σ^{2}}{n} = \frac{N p (1 - p)}{n} \\ E (\hat{p}) = E (\frac{\hat{μ}}{N}) = \frac{1}{N} N p = p \\ V a r (\hat{p}) = V a r (\frac{\bar{X}}{N}) = \frac{1}{N^{2}} \frac{n V a r (X)}{n^{2}} = \frac{1}{n N^{2}} N p (1 - p) = \frac{p (1 - p)}{n N} \\ P (μ_{l} \leq μ \leq μ_{u}) = 1 - α \\ P (\frac{\bar{x} - μ_{u}}{\sqrt{\frac{N p (1 - p)}{n}}} \leq \frac{\bar{x} - μ}{\sqrt{\frac{N p (1 - p)}{n}}} \leq \frac{\bar{x} - μ_{l}}{\sqrt{\frac{N p (1 - p)}{n}}}) = 1 - α \\ P (- Z_{1 - \frac{α}{2}} \leq N (0, 1) \leq Z_{1 - \frac{α}{2}}) = 1 - α \\ C I for μ = E (X) is (\bar{X} - Z_{1 - \frac{α}{2}} \sqrt{\frac{N \hat{p} (1 - \hat{p})}{n}}, \bar{X} + Z_{1 - \frac{α}{2}} \sqrt{\frac{N \hat{p} (1 - \hat{p})}{n}}) \end{aligned}