Online k-MLE for Mixture Modeling with Exponential Families

  • Christophe Saint-JeanEmail author
  • Frank Nielsen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9389)


This paper address the problem of online learning finite statistical mixtures of exponential families. A short review of the Expectation-Maximization (EM) algorithm and its online extensions is done. From these extensions and the description of the k-Maximum Likelihood Estimator (k-MLE), three online extensions are proposed for this latter. To illustrate them, we consider the case of mixtures of Wishart distributions by giving details and providing some experiments.


Mixture modeling Online learning k-MLE Wishart distribution 


  1. 1.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B (Methodol.) 39(1), 1–38 (1977)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Titterington, D.M.: Recursive parameter estimation using incomplete data. J. R. Stat. Soc. Ser. B (Methodol.) 46(2), 257–267 (1984)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Cappé, O., Moulines, E.: On-line expectation-maximization algorithm for latent data models. J. R. Stat. Soc. Ser. B (Methodol.) 71(3), 593–613 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Neal, R.M., Hinton, G.E.: A view of the EM algorithm that justifies incremental, sparse, and other variants. In: Jordan, M.I. (ed.) Learning in Graphical Models, pp. 355–368. MIT Press, Cambridge (1999)Google Scholar
  5. 5.
    Nielsen, F.: On learning statistical mixtures maximizing the complete likelihood. In: Bayesian Inference and Maximum Entropy Methods in Science and Engineering (MaxEnt 2014), AIP Conference Proceedings Publishing, 1641, pp. 238–245 (2014)Google Scholar
  6. 6.
    Celeux, G., Govaert, G.: A classification EM algorithm for clustering and two stochastic versions. Comput. Stat. Data Anal. 14(3), 315–332 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Samé, A., Ambroise, C., Govaert, G.: An online classification EM algorithm based on the mixture model. Stat. Comput. 17(3), 209–218 (2007)MathSciNetCrossRefGoogle Scholar
  8. 8.
    MacQueen, J.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, 1(14) (1967)Google Scholar
  9. 9.
    Saint-Jean, C., Nielsen, F.: Hartigan’s method for k-MLE : mixture modeling with Wishart distributions and its application to motion retrieval. In: Nielsen, F. (ed.) Geometric Theory of Information. Signals and Communication Technology, pp. 301–330. Springer, Switzerland (2014)CrossRefGoogle Scholar
  10. 10.
    Nielsen, F., Garcia, V.: Statistical exponential families: a digest with flash cards, November 2009.
  11. 11.
    Wishart, J.: The generalised product moment distribution in samples from a normal multivariate population. Biometrika 20(1/2), 32–52 (1928)CrossRefzbMATHGoogle Scholar
  12. 12.
    Liu, Q., Ihler, A.T.: Distributed estimation, information loss and exponential families. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 27, pp. 1098–1106. MIT Press, Cambridge (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Mathématiques, Image, Applications (MIA)Université de La RochelleLa RochelleFrance
  2. 2.LIXÉcole PolytechniquePalaiseauFrance

Personalised recommendations