Bayesian Learning of Finite Asymmetric Gaussian Mixtures

  • Shuai FuEmail author
  • Nizar BouguilaEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10868)


Asymmetric Gaussian mixture (AGM) model has been proven to be more flexible than the classic Gaussian mixture model from many aspects. In contrast with previous efforts that have focused on maximum likelihood estimation, this paper introduces a fully Bayesian learning approach using Metropolis-Hastings (MH) within Gibbs sampling method to learn AGM model. We show the merits of the proposed model using synthetic data and a challenging intrusion detection application.


Asymmetric Gaussian mixture Metropolis-Hastings Gibbs sampling MCMC Intrusion detection 



The completion of this research work was made possible thanks to Concordia University via a Concordia University Research Chair Tier II.


  1. 1.
    Bouguila, N., Ziou, D.: MML-based approach for finite dirichlet mixture estimation and selection. In: Perner, P., Imiya, A. (eds.) MLDM 2005. LNCS (LNAI), vol. 3587, pp. 42–51. Springer, Heidelberg (2005). Scholar
  2. 2.
    Bouguila, N., Ziou, D., Hammoud, R.I.: A Bayesian non-Gaussian mixture analysis: application to eye modeling. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2007), 18–23 June 2007, Minneapolis, Minnesota, USA (2007)Google Scholar
  3. 3.
    Sefidpour, A., Bouguila, N.: Spatial color image segmentation based on finite non-Gaussian mixture models. Expert Syst. Appl. 39(10), 8993–9001 (2012)CrossRefGoogle Scholar
  4. 4.
    Bouguila, N., Amayri, O.: A discrete mixture-based kernel for SVMs: application to spam and image categorization. Inf. Process. Manag. 45(6), 631–642 (2009)CrossRefGoogle Scholar
  5. 5.
    McLachlan, G.J., Peel, D.: Finite Mixture Models. Wiley, Hoboken (2000)CrossRefGoogle Scholar
  6. 6.
    Bouguila, N., Ziou, D., Monga, E.: Practical Bayesian estimation of a finite beta mixture through Gibbs sampling and its applications. Stat. Comput. 16(2), 215–225 (2006)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Elguebaly, T., Bouguila, N.: Bayesian learning of generalized Gaussian mixture models on biomedical images. In: Schwenker, F., El Gayar, N. (eds.) ANNPR 2010. LNCS (LNAI), vol. 5998, pp. 207–218. Springer, Heidelberg (2010). Scholar
  8. 8.
    Elguebaly, T., Bouguila, N.: Background subtraction using finite mixtures of asymmetric Gaussian distributions and shadow detection. Mach. Vis. Appl. 25(5), 1145–1162 (2014)CrossRefGoogle Scholar
  9. 9.
    Bouguila, N., Almakadmeh, K., Boutemedjet, S.: A finite mixture model for simultaneous high-dimensional clustering, localized feature selection and outlier rejection. Expert Syst. Appl. 39(7), 6641–6656 (2012)CrossRefGoogle Scholar
  10. 10.
    Boutemedjet, S., Ziou, D., Bouguila, N.: Model-based subspace clustering of non-Gaussian data. Neurocomputing 73(10–12), 1730–1739 (2010)CrossRefGoogle Scholar
  11. 11.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B (Methodol.) 39(1), 1–38 (1977)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Bouguila, N., Ziou, D.: On fitting finite Dirichlet mixture using ECM and MML. In: Singh, S., Singh, M., Apte, C., Perner, P. (eds.) ICAPR 2005. LNCS, vol. 3686, pp. 172–182. Springer, Heidelberg (2005). Scholar
  13. 13.
    Bouguila, N., Ziou, D., Hammoud, R.I.: On Bayesian analysis of a finite generalized Dirichlet mixture via a metropolis-within-Gibbs sampling. Pattern Anal. Appl. 12(2), 151–166 (2009)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Bouguila, N., Elguebaly, T.: A fully Bayesian model based on reversible jump MCMC and finite beta mixtures for clustering. Expert Syst. Appl. 39(5), 5946–5959 (2012)CrossRefGoogle Scholar
  15. 15.
    Hastings, W.K.: Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57(1), 97–109 (1970)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Bouguila, N.: Bayesian hybrid generative discriminative learning based on finite Liouville mixture models. Pattern Recogn. 44(6), 1183–1200 (2011)CrossRefGoogle Scholar
  17. 17.
    Bourouis, S., Mashrgy, M.A., Bouguila, N.: Bayesian learning of finite generalized inverted Dirichlet mixtures: application to object classification and forgery detection. Expert Syst. Appl. 41(5), 2329–2336 (2014)CrossRefGoogle Scholar
  18. 18.
    Elguebaly, T., Bouguila, N.: Bayesian learning of finite generalized Gaussian mixture models on images. Sig. Process. 91(4), 801–820 (2011)CrossRefGoogle Scholar
  19. 19.
    Bouguila, N., Ziou, D.: A powreful finite mixture model based on the generalized Dirichlet distribution: unsupervised learning and applications. In: 17th International Conference on Pattern Recognition, ICPR 2004, Cambridge, UK, 23–26 August 2004, pp. 280–283. IEEE Computer Society (2004)Google Scholar
  20. 20.
    Bouguila, N., Ziou, D.: Dirichlet-based probability model applied to human skin detection [image skin detection]. In: 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2004, 17–21 May 2004, Montreal, Quebec, Canada, pp. 521–524. IEEE (2004)Google Scholar
  21. 21.
    Luengo, D., Martino, L.: Fully adaptive Gaussian mixture metropolis-hastings algorithm. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2013, 26–31 May 2013, Vancouver, BC, Canada, pp. 6148–6152. IEEE (2013)Google Scholar
  22. 22.
    Tavallaee, M., Bagheri, E., Lu, W., Ghorbani, A.A.: A detailed analysis of the KDD CUP 99 data set. In: IEEE Symposium on Computational Intelligence for Security and Defense Applications, CISDA 2009, 8–10 July 2009, Ottawa, Canada, pp. 1–6. IEEE (2009)Google Scholar
  23. 23.
    Stephens, M.: Bayesian analysis of mixture models with an unknown number of components - an alternative to reversible jump methods. Ann. Stat. 40–74 (2000)Google Scholar
  24. 24.
    Puttini, R.S., Marrakchi, Z., Mé, L.: A Bayesian classification model for real-time intrusion detection. In: AIP Conference Proceedings, vol. 659, pp. 150–162 (2003)Google Scholar
  25. 25.
    Zhong, S., Khoshgoftaar, T.M., Seliya, N.: Clustering-based network intrusion detection. Int. J. Reliab. Qual. Saf. Eng. 14(02), 169–187 (2007)CrossRefGoogle Scholar
  26. 26.
    Fan, W., Bouguila, N., Ziou, D.: Unsupervised anomaly intrusion detection via localized Bayesian feature selection. In: Cook, D.J., Pei, J., Wang, W., Zaïane, O.R., Wu, X. eds.: 11th IEEE International Conference on Data Mining, ICDM 2011, 11–14 December, Vancouver, BC, Canada, pp. 1032–1037. IEEE Computer Society (2011)Google Scholar
  27. 27.
    Hartigan, J.A., Wong, M.A.: Algorithm as 136: a k-means clustering algorithm. J. Roy. Stat. Soc. Ser. C (Appl. Stat.) 28(1), 100–108 (1979)zbMATHGoogle Scholar
  28. 28.
    Bouguila, N.: Spatial color image databases summarization. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2007, 15–20 April 2007, Honolulu, Hawaii, USA, pp. 953–956. IEEE (2007)Google Scholar
  29. 29.
    Bouguila, N., Ziou, D.: Improving content based image retrieval systems using finite multinomial Dirichlet mixture. In: Proceedings of the 2004 14th IEEE Signal Processing Society Workshop Machine Learning for Signal Processing, pp. 23–32. IEEE (2004)Google Scholar
  30. 30.
    Boutemedjet, S., Ziou, D., Bouguila, N.: Unsupervised feature selection for accurate recommendation of high-dimensional image data. In: Platt, J.C., Koller, D., Singer, Y., Roweis, S.T. (eds.) Advances in Neural Information Processing Systems 20, Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, 3–6 December 2007, Vancouver, British Columbia, Canada, pp. 177–184. Curran Associates, Inc. (2007)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Concordia UniversityMontrealCanada

Personalised recommendations