The Expectation-Maximization Algorithm

Although the models in Table 17.5 are fitted using direct observed data likelihood maximization in PROC MIXED, Little and Rubin (1987) obtained these same results using the Expectation-Maximization algorithm. Special forms of the algorithm, designed for specific applications, had been proposed for about half a century (e.g., Yates 1933), but the first unifying and formal account was given by Dempster, Laird, and Rubin (1977). McLach-lan and Krishnan (1997) devoted a whole volume to the EM algorithm and its extensions.

Even though the SAS procedure MIXED uses direct likelihood maximization, the EM algorithm is generally useful to maximize certain complicated likelihood functions. For example, it has been used to maximize mixture likelihoods in Section 12.3. Liu and Rubin (1995) used it to estimate the t-distribution, based on EM, its extension ECM (expectation conditional maximization), and ECME (expectation conditional maximization, either), which are described in Meng and Rubin (1993), Liu and Rubin (1994), and van Dyk, Meng, and Rubin (1995). EM methods specifically for mixed-effects models are discussed in Meng and van Dyk (1998). A nice review is given in Meng (1997), where the focus is on EM applications in medical studies.