Neyman-Pearson lemma

Used to derive the most powerful test, by finding the optimal critical region

Concept of fixing $α$ and then maximizing the power, instead of having $α$ and $β$ balancing each other out

Suppose $X_{1}, \dots, X_{n} \sim i i d f (x; θ)$ . To test $H_{0} : θ = θ_{0} v s H_{a} : θ = θ_{1}$ , let $C$ be the critical region such that:

P (\underset{\sim}{x} \in C | H_{0} is true) = α

If $\forall$ $C^{*}$ such that $P (\underset{\sim}{x} \in C^{*} | H_{0} is true) = α$ we have that the power of $C >$ power of $C^{*}$

P (\underset{\sim}{x} \in C | H_{0} is false) \geq P (\underset{\sim}{x} \in C^{*} | H_{0} is false)

Then we say that $C$ is the best Critical Region / most powerful test

100-101?

NP provides a method for testing two simple hypothesis, for a more general method of composite hypothesis we use the Likelihood ratio test

Examples

$X_{1}, \dots, X_{n} \sim N (μ, 1)$ To test $H_{0} : μ = μ_{0} v s H_{a} : μ = μ_{1}, μ_{1} > μ_{0}$

\begin{aligned} L_{0} & = f (x_{1}, \dots, x_{n}; μ_{0}, 1) \\ = \prod \frac{1}{\sqrt{2 π}} e^{- (x_{i} - μ_{0})^{2} / 2} = (2 π)^{- n / 2} e^{- \sum (x_{i} - μ_{0})^{2} / 2} \\ L_{1} & = f (x_{1}, \dots, x_{n}; μ_{1}, 1) \\ = (2 π)^{- n / 2} e^{- \sum (x_{i} - μ_{1})^{2} / 2} \\ \frac{L_{0}}{L_{1}} & = \frac{(2 π)^{- n / 2} e^{- \sum (x_{i} - μ_{0})^{2} / 2}}{(2 π)^{- n / 2} e^{- \sum (x_{i} - μ_{1})^{2} / 2}} \\ = \exp {- \frac{1}{2} (\sum x_{i}^{2} - 2 μ_{0} \sum x_{i} + n μ_{0}^{2} - \sum x_{i}^{2} + 2 μ_{1} \sum x_{i} - n μ_{1}^{2})} \\ = \exp (n \bar{x} (μ_{0} - μ_{1}) - \frac{n}{2} (μ_{0}^{2} - μ_{1}^{2})) \\ now to find a region and K : \\ \frac{L_{0}}{L_{1}} & = e^{n \bar{x} (μ_{0} - μ_{1}) - \frac{n}{2} (μ_{0}^{2} - μ_{1}^{2})} \leq k s t (x_{1}, \dots, x_{n}) \in C \\ \to n \bar{x} (μ_{0} - μ_{1}) - \frac{n}{2} (μ_{0}^{2} - μ_{1}^{2}) \leq \ln k \\ \bar{x} \geq \frac{1}{n (μ_{0} - m_{1})} [\ln k + \frac{n}{2} (μ_{0}^{2} - μ_{1}^{2})] = k^{*} \\ \to & {\begin{cases} \bar{x} > k^{*}, (x_{1}, \dots, x_{n}) \in C \\ \bar{x} \leq k^{*}, (x_{1}, \dots, x_{n}) \notin C \end{cases} \\ α & = P (reject H_{0} | H_{0} is true), \bar{x} \sim N (μ_{0}, \frac{1}{n}) \\ α & = P (\bar{x} \geq k^{*} | μ = μ_{0}) \\ α & = P (\frac{\bar{x} - μ_{0}}{\sqrt{1 / n}} \geq \frac{k^{*} - μ_{0}}{\sqrt{1 / n}}) \\ k^{*} & = μ_{0} + Z_{1 - α} \sqrt{\frac{1}{n}} \\ The most powerful critical region for the test is : \\ C & = {(x_{1}, \dots, x_{n}); \bar{x} \geq μ_{0} + Z_{1 - α} \sqrt{\frac{1}{n}})} \end{aligned}

When $\bar{x}$ increases, it favors $H_{a}$
When $\bar{x}$ decreases, it favors $H_{0}$

The most powerful test at $α$ -level of significance is

ϕ (x_{1}, \dots, x_{n}) = {\begin{cases} 1, if Z \geq Z_{1 - α} \\ 0, o.w. \end{cases}