A New Method for Random Initialization of the EM Algorithm for Multivariate Gaussian Mixture Learning

  • Wojciech Kwedlo
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 226)


In the paper a new method for random initialization of the EM algorithm for multivariate Gaussian mixture models is proposed. In the method booth mean vector and covariance matrix of a mixture component are initialized randomly. The mean vector of the component is initialized by the feature vector, selected from a randomly chosen set of candidate feature vectors, located farthest from already initialized mixture components as measured by the Mahalanobis distance. In the experiments the EM algorithm was applied to the clustering problem. Our approach was compared to three well known EM initialization methods. The results of the experiments, performed on synthetic datasets, generated from the Gaussian mixtures with the varying degree of overlap between clusters, indicate that our method outperforms three others.


Feature Vector Gaussian Mixture Model Mixture Component Random Initialization Initialization Method 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)zbMATHGoogle Scholar
  2. 2.
    Caglar, A., Aksoy, S., Arikan, O.: Maximum likelihood estimation of Gaussian mixture models using stochastic search. Pattern Recognit. 45(7), 2804–2816 (2012)zbMATHCrossRefGoogle Scholar
  3. 3.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. Royal Stat. Soc. Ser. B 39(1), 1–38 (1977)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Everitt, B.S., Landau, S., Leese, M.: Cluster Analysis. Arnold, London (2001)zbMATHGoogle Scholar
  5. 5.
    Golub, G.H., van Loan, C.F.: Matrix Computations. Johns Hopkins, Baltimore (1996)zbMATHGoogle Scholar
  6. 6.
    Hastie, T., Tibshirani, R.: Discriminant analysis by Gaussian mixtures. J. Royal Stat. Soc. Ser. B 58(1), 155–176 (1996)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Hubert, L., Arabie, P.: Comparing partitions. J. Classif. 2(1), 193–218 (1985)CrossRefGoogle Scholar
  8. 8.
    Kwedlo, W.: A clustering method combining differential evolution with the K-means algorithm. Pattern Recognit. Lett. 32(12), 1613–1621 (2011)CrossRefGoogle Scholar
  9. 9.
    Maitra, R., Melnykov, V.: Simulating data to study performance of finite mixture modeling and clustering algorithms. J. Comput. Graph. Stat. 19(2), 354–376 (2010)MathSciNetCrossRefGoogle Scholar
  10. 10.
    McLachlan, G., Peel, D.: Finite Mixture Models. Wiley, New York (2000)zbMATHCrossRefGoogle Scholar
  11. 11.
    McQueen, J.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, pp. 281–297 (1967)Google Scholar
  12. 12.
    Pernkopf, F., Bouchaffra, D.: Genetic-based EM algorithm for learning Gaussian mixture models. IEEE Trans. Pattern Analysis Mach. Intell. 27(8), 1344–1348 (2005)CrossRefGoogle Scholar
  13. 13.
    Redner, R.A., Walker, H.F.: Mixture densities, maximum likelihood and the EM algorithm. SIAM Rev. 26(2), 195–239 (1984)MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Verbeek, J.J., Vlassis, N., Kröse, B.: Efficient greedy learning of Gaussian mixture models. Neural Comput. 15(2), 469–485 (2003)zbMATHCrossRefGoogle Scholar
  15. 15.
    Xu, R., Wunsch, D.: Survey of clustering algorithms. IEEE Trans. Neural Netw. 16(3), 645–678 (2005)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2013

Authors and Affiliations

  1. 1.Faculty of Computer ScienceBialystok University of TechnologyBiałystokPoland

Personalised recommendations