Broadly used algorithm in situations where the observed data is considered incomplete when finding Method of maximum likelihood

• standard tool for mixture models

The complete data $y$

consists of observed data $x$ (which is incomplete) and unobserved data $z$

Formulation

Suppose we have the pdfs or pmfs for the complete data $f_c(y;\theta )$ and the incomplete data $f(x;\theta )$

The complete-data log-likelihood could be formed for $\theta$ if $y$ are fully observed and available

In the first iteration, the E-step finds the expectation of the complete-data log likelihood given ${\theta }^{(0)}$, the initial value for $\theta$

The M-step is to find the value of $\theta$ that maximizes the $Q(\theta ;{\theta }^{(0)})$, ${\theta }^{(1)}$ such that

for all possible $\theta$

On the $(i+1{)}^{th}$ iteration, we define:
E-Step: Calculate

M-Step: Find ${\underset{\sim }{\theta }}^{(i+1)}$ that maximizes $Q(\theta ;{\theta }^{(i)})$, that is

for all $\theta$

Stopping Criteria

The pdf of the incomplete data is given by integrating out z and getting the marginal of x

Repeat both the E and M steps alternatively until convergence as the difference of the incomplete-data likelihood $L({\theta }^{(1)})-L({\theta }^{(i)})$ is very small and less than a pre-specified tolerance amount.