Computational Statistics

, Volume 19, Issue 3, pp 385–406 | Cite as

Optimization of Mixture Models: Comparison of Different Strategies

  • André Berchtold


Mixture model parameters are usually computed with maximum likelihood using an Expectation-Maximization (EM) algorithm. However, it is wellknown that this method can sometimes converge toward a critical point of the solution space which is not the global maximum. To minimize this problem, different strategies using different combinations of algorithms can be used. In this paper, we compare by the mean of numerical simulations strategies using EM, Classification EM, Stochastic EM, and Genetic algorithms for the optimization of mixture models. Our results indicate that two-stage procedures combining both an exploration phase and an optimization phase provide the best results, especially when these methods axe applied on several sets of initial conditions rather than on one single starting point.


Mixture models Mixture Transition Distribution model (MTD) Expectation-Maximization (EM) algorithm Classification EM (CEM) Stochastic EM (SEM) Genetic Algorithm (GA) 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Berchtold A. (2001), Mixture Modeling of Non-Gaussian Probability Distributions, Proceedings of the 10th International Symposium on Applied Stochastic Models and Data Analysis, 188–193, Compiègne, France.Google Scholar
  2. Berchtold A. (2003), Mixture Transition Distribution (MTD) Modeling of Heteroscedastic Time Series. Computational Statistics & Data Analysis, 41(3–4), 399–411.MathSciNetCrossRefMATHGoogle Scholar
  3. Berchtold, A., Raftery, A.E. (2002), The Mixture Transition Distribution (MTD) Model for High-Order Markov Chains and Non-Gaussian Time Series. Statistical Science, 17(3), 328–356.MathSciNetCrossRefMATHGoogle Scholar
  4. Biernacki C., Celeux G., Govaert, G. (2003), Choosing starting values for the EM algorithm for getting the highest likelihood in multivariate Gaussian mixture models. Computational Statistics & Data Analysis, 41(3–4), 561–575.MathSciNetCrossRefMATHGoogle Scholar
  5. Böhning, D. (2001), The Potential of Recent Developments in Nonparametric Mixture Distributions, Proceedings of the 10th International Symposium on Applied Stochastic Models and Data Analysis, 6–13, Compiègne, France.Google Scholar
  6. Celeux, G., Diebolt, J. (1985), The SEM algorithm: a probabilistic teacher algorithm derived from the EM algorithm for the mixture problem, Computational Statistics Quarterly, 2, 73–82.Google Scholar
  7. Celeux, G., Govaert, G. (1992), A Classification EM Algorithm for Clustering and Two Stochastic Versions, Computational Statistics & Data Analysis, 14, 315–332.MathSciNetCrossRefMATHGoogle Scholar
  8. Davis, L. et al. (1991), Handbook of Genetic Algorithms, Davis Editor, Van Nostrand Reinhold, New York.Google Scholar
  9. Holland, J.H. (1975), Adaptation in Natural and Artificial Systems, University of Michigan Press, Ann Arbor.Google Scholar
  10. Le, N.D., Martin, R.D., Raftery, A.E. (1996), Modeling Flat Stretches, Bursts, and Outliers in Time Series Using Mixture Transition Distribution Models, Journal of the American Statistical Association, 91, 1504–1515.MathSciNetMATHGoogle Scholar
  11. McLachlan, G.J., Krishnan, T. (1996), The EM Algorithm and Extensions, John Wiley & Sons, New York.MATHGoogle Scholar
  12. Raftery, A.E. (1985), A model for high-order Markov chains, Journal of the Royal Statistical Society B, 47(3), 528–539.MathSciNetMATHGoogle Scholar
  13. Vose, M.D. (1999), The Simple Genetic Algorithm, Foundations and Theory, MIT Press, Cambridge.MATHGoogle Scholar
  14. Wong, C.S., Li, W.K. (2000), On a mixture autoregression model, Journal of the Royal Statistical Society B, 62, 95–115.MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Physica-Verlag 2004

Authors and Affiliations

  1. 1.Institute of Applied Mathematics, SSP, BFSH2University of LausanneLausanneSwitzerland

Personalised recommendations