Advertisement

Multivariate Bounded Asymmetric Gaussian Mixture Model

  • Muhammad AzamEmail author
  • Basim Alghabashi
  • Nizar Bouguila
Chapter
Part of the Unsupervised and Semi-Supervised Learning book series (UNSESUL)

Abstract

In this chapter, bounded asymmetric Gaussian mixture model (BAGMM) is proposed. In the described model, parameter estimation is performed by maximization of log-likelihood via expectation–maximization (EM) and Newton–Raphson algorithm. This model is applied to several applications for data clustering. As a first step, to validate our model, we have chosen spambase dataset for clustering spam and non-spam emails. Another application selected for validation of our algorithm is object data clustering and we have used two popular datasets (Caltech 101 and Corel) in this task. Finally we have performed clustering on texture data and VisTex dataset is employed for this task. In order to evaluate the clustering, in all abovementioned applications, several performance metrics are employed and experimental results are further compared in similar settings with asymmetric Gaussian mixture model (AGMM). From the experiments and results in all applications, it is examined that BAGMM has outperformed AGMM in the clustering task.

Keywords

Bounded asymmetric Gaussian mixture model (BAGMM) Maximum likelihood (ML) Expectation–maximization (EM) Newton–Raphson Data clustering Object categorization 

Notes

Acknowledgement

The completion of this research was made possible thanks to the Natural Sciences and Engineering Research Council of Canada (NSERC).

References

  1. 1.
    Ahmad, T., Jameel, A., Ahmad, B.: Pattern recognition using statistical and neural techniques. In: International Conference on Computer Networks and Information Technology, pp. 87–91 (2011).  https://doi.org/10.1109/ICCNIT.2011.6020913
  2. 2.
    Bdiri, T., Bouguila, N., Ziou, D.: Object clustering and recognition using multi-finite mixtures for semantic classes and hierarchy modeling. Expert Syst. Appl. 41(4), 1218–1235 (2014)CrossRefGoogle Scholar
  3. 3.
    Berg, A.C., Berg, T.L., Malik, J.: Shape matching and object recognition using low distortion correspondences. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR, pp. 26–33. IEEE, Piscataway (2005)Google Scholar
  4. 4.
    Bouguila, N., ElGuebaly, W.: On discrete data clustering. In: Proceedings of the 12th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, PAKDD’08, pp. 503–510. Springer, Berlin (2008). http://dl.acm.org/citation.cfm?id=1786574.1786622
  5. 5.
    Bouguila, N., Ziou, D.: A nonparametric Bayesian learning model: Application to text and image categorization. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 463–474. Springer, Berlin (2009)CrossRefGoogle Scholar
  6. 6.
    Braunl, T., Stefan Feyrer, D.I., Wolfgang Rapf, D.I., Michael Reinhardt, D.I.: Texture recognition. In: Parallel Image Processing, pp. 121–130. Springer, Berlin (2001)zbMATHGoogle Scholar
  7. 7.
    Chadha, A., Mallik, S., Johar, R.: Comparative study and optimization of feature-extraction techniques for content based image retrieval. arXiv preprint arXiv:1208.6335 (2012)Google Scholar
  8. 8.
    Clausi, D.A., Jernigan, M.E.: A fast method to determine co-occurrence texture features. IEEE Trans. Geosci. Remote Sens. 36(1), 298–300 (1998). https://doi.org/10.1109/36.655338 CrossRefGoogle Scholar
  9. 9.
    De Siqueira, F.R., Schwartz, W.R., Pedrini, H.: Multi-scale gray level co-occurrence matrices for texture description. Neurocomputing 120, 336–345 (2013)CrossRefGoogle Scholar
  10. 10.
    Dua, D., Graff, C.: UCI machine learning repository (2017)Google Scholar
  11. 11.
    Elguebaly, T., Bouguila, N.: Background subtraction using finite mixtures of asymmetric Gaussian distributions and shadow detection. Mach. Vis. Appl. 25(5), 1145–1162 (2014)CrossRefGoogle Scholar
  12. 12.
    Elguebaly, T., Bouguila, N.: Simultaneous high-dimensional clustering and feature selection using asymmetric Gaussian mixture models. Image Vis. Comput. 34, 27–41 (2015)CrossRefGoogle Scholar
  13. 13.
    Espindola, R., Ebecken, N.: On extending f-measure and G-mean metrics to multi-class problems. WIT Trans. Inf. Commun. Technol. 35, 10 (2005)Google Scholar
  14. 14.
    Fan, W., Bouguila, N., Ziou, D.: Variational learning for finite Dirichlet mixture models and applications. IEEE Trans. Neural Netw. Learn. Syst. 23(5), 762–774 (2012)CrossRefGoogle Scholar
  15. 15.
    Farag, A., El-Baz, A., Gimel’farb, G.: Precise segmentation of multimodal images. IEEE Trans. Image Process. 15(4), 952–968 (2006).  https://doi.org/10.1109/TIP.2005.863949 CrossRefGoogle Scholar
  16. 16.
    Fei-Fei, L., Fergus, R., Perona, P.: Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categories. In: 2004 Conference on Computer Vision and Pattern Recognition Workshop, pp. 178–178 (2004)Google Scholar
  17. 17.
    Figueiredo, M.A., Jain, A.K.: Unsupervised learning of finite mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 24(3), 381–396 (2002)CrossRefGoogle Scholar
  18. 18.
    Fu, S., Bouguila, N.: Asymmetric Gaussian mixtures with reversible jump MCMC. In: 2018 IEEE Canadian Conference on Electrical Computer Engineering (CCECE), pp. 1–4. IEEE, Piscataway (2018).  https://doi.org/10.1109/CCECE.2018.8447816
  19. 19.
    Gorodkin, J.: Comparing two K-category assignments by a K-category correlation coefficient. Comput. Biol. Chem. 28(5–6), 367–374 (2004)CrossRefGoogle Scholar
  20. 20.
    Grauman, K., Darrell, T.: The pyramid match kernel: Discriminative classification with sets of image features. In: Tenth IEEE International Conference on Computer Vision, ICCV, pp. 1458–1465. IEEE Computer Society, Silver Spring (2005)Google Scholar
  21. 21.
    Haralick, R.M., Shanmugam, K., Dinstein, I.: Textural features for image classification. IEEE Trans. Syst. Man Cybern. SMC-3(6), 610–621 (1973)CrossRefGoogle Scholar
  22. 22.
    Hedelin, P., Skoglund, J.: Vector quantization based on Gaussian mixture models. IEEE Trans. Speech Audio Process. 8(4), 385–401 (2000). https://doi.org/10.1109/89.848220 CrossRefGoogle Scholar
  23. 23.
    Holub, A., Welling, M., Perona, P.: Exploiting unlabelled data for hybrid object classification. In: Proceedings of the Neural Information Processing Systems. Workshop Inter-Class Transfer, vol. 7, p. 2 (2005)Google Scholar
  24. 24.
    Holub, A.D., Welling, M., Perona, P.: Combining generative models and fisher kernels for object recognition. In: Tenth IEEE International Conference on Computer Vision (ICCV’05), vol. 1, pp. 136–143. IEEE, Piscataway (2005)Google Scholar
  25. 25.
    Hong, J.: The state of phishing attacks. Commun. ACM 55(1), 74–81 (2012)CrossRefGoogle Scholar
  26. 26.
    Ihou, K.E., Bouguila, N.: Variational-based latent generalized Dirichlet allocation model in the collapsed space and applications. Neurocomputing 332, 372–395 (2019)CrossRefGoogle Scholar
  27. 27.
    Ji, Z., Huang, Y., Sun, Q., Cao, G.: A spatially constrained generative asymmetric Gaussian mixture model for image segmentation. J. Vis. Commun. Image Represent. 40, 611–626 (2016)CrossRefGoogle Scholar
  28. 28.
    Jian, M., Liu, L., Guo, F.: Texture image classification using perceptual texture features and Gabor wavelet features. In: 2009 Asia-Pacific Conference on Information Processing, vol. 2, pp. 55–58 (2009).  https://doi.org/10.1109/APCIP.2009.150
  29. 29.
    Jurman, G., Furlanello, C.: A unifying view for performance measures in multi-class prediction. arXiv preprint arXiv:1008.2908 (2010)Google Scholar
  30. 30.
    Jurman, G., Riccadonna, S., Furlanello, C.: A comparison of MCC and CEN error measures in multi-class prediction. PLoS One 7(8), e41882 (2012)CrossRefGoogle Scholar
  31. 31.
    Kato, T., Omachi, S., Aso, H.: Asymmetric Gaussian and its application to pattern recognition. In: Joint IAPR International Workshops on Statistical Techniques in Pattern Recognition (SPR) and Structural and Syntactic Pattern Recognition (SSPR), pp. 405–413. Springer, Berlin (2002)Google Scholar
  32. 32.
    Khodaskar, A.A., Ladhake, S.A.: Pattern recognition: Advanced development, techniques and application for image retrieval. In: 2014 International Conference on Communication and Network Technologies, pp. 74–78 (2014).  https://doi.org/10.1109/CNT.2014.7062728
  33. 33.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol. 2, pp. 2169–2178. IEEE, Piscataway (2006)Google Scholar
  34. 34.
    Lindblom, J., Samuelsson, J.: Bounded support Gaussian mixture modeling of speech spectra. IEEE Trans. Speech Audio Process. 11(1), 88–99 (2003).  https://doi.org/10.1109/TSA.2002.805639 CrossRefGoogle Scholar
  35. 35.
    Liu, G.H., Li, Z.Y., Zhang, L., Xu, Y.: Image retrieval based on micro-structure descriptor. Pattern Recogn. 44(9), 2123–2133 (2011)CrossRefGoogle Scholar
  36. 36.
    Liu, G.H., Yang, J.Y., Li, Z.: Content-based image retrieval using computational visual attention model. Pattern Recogn. 48(8), 2554–2566 (2015)CrossRefGoogle Scholar
  37. 37.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  38. 38.
    Ma, H., Chan, J.C., Saha, T.K., Ekanayake, C.: Pattern recognition techniques and their applications for automatic classification of artificial partial discharge sources. IEEE Trans. Dielectr. Electr. Insul. 20(2), 468–478 (2013).  https://doi.org/10.1109/TDEI.2013.6508749 CrossRefGoogle Scholar
  39. 39.
    Ma, J., Jiang, J., Liu, C., Li, Y.: Feature guided Gaussian mixture model with semi-supervised EM and local geometric constraint for retinal image registration. Inf. Sci. 417, 128–142 (2017)MathSciNetCrossRefGoogle Scholar
  40. 40.
    Malsiner-Walli, G., Frühwirth-Schnatter, S., Grün, B.: Model-based clustering based on sparse finite Gaussian mixtures. Stat. Comput. 26(1–2), 303–324 (2016)MathSciNetCrossRefGoogle Scholar
  41. 41.
    Marin-Jimenez, M.J., De La Blanca, N.P.: Empirical study of multi-scale filter banks for object categorization. In: International Conference on Pattern Recognition, ICPR, pp. 578–581. IEEE, Piscataway (2006)Google Scholar
  42. 42.
    Mclachlan, G., Basford, K.: Mixture models: Inference and applications to clustering. J. R. Stat. Soc. Ser. C 38(2), 384–385 (1989). https://doi.org/10.2307/2348072 Google Scholar
  43. 43.
    McLachlan, G., Krishnan, T.: The EM Algorithm and Extensions, vol. 382. Wiley, London (2007)zbMATHGoogle Scholar
  44. 44.
    McLachlan, G., Peel, D.: Finite Mixture Models. Wiley, London (2004)zbMATHGoogle Scholar
  45. 45.
    MIT Media Lab.: Vistex texture database (1995). https://vismod.media.mit.edu/vismod/imagery/VisionTexture/vistex.html
  46. 46.
    Mutch, J., Lowe, D.G.: Multiclass object recognition with sparse, localized features. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol. 1, pp. 11–18. IEEE, Piscataway (2006)Google Scholar
  47. 47.
    Neal, R.M., Hinton, G.E.: A view of the EM algorithm that justifies incremental, sparse, and other variants. In: Learning in graphical models, pp. 355–368. Springer, Berlin (1998)CrossRefGoogle Scholar
  48. 48.
    Nguyen, T.M., Wu, Q.J., Zhang, H.: Bounded generalized Gaussian mixture model. Pattern Recogn. 47(9) (2014)CrossRefGoogle Scholar
  49. 49.
    Park, I., Sharman, R., Rao, H.R., Upadhyaya, S.: The effect of spam and privacy concerns on e-mail users’ behavior. J. Trans. Inf. Syst. Secur. 3(1), 39–62 (2016)Google Scholar
  50. 50.
    Serre, T., Wolf, L., Poggio, T.: Object recognition with features inspired by visual cortex. Tech. rep., Massachusetts Institute of Technology Cambridge Department of Brain and Cognitive Sciences (2006)Google Scholar
  51. 51.
    Tang, X.: Texture information in run-length matrices. IEEE Trans. Image Process. 7(11), 1602–1609 (1998)CrossRefGoogle Scholar
  52. 52.
    Titterington, D., Smith, A., Makov, U.: Statistical Analysis of Finite Mixture Distributions. Wiley, New York (1985)zbMATHGoogle Scholar
  53. 53.
    Viitaniemi, V., Laaksonen, J.: Techniques for still image scene classification and object detection. In: Kollias, S., Stafylopatis, A., Duch, W., Oja, E. (eds.) Artificial Neural Networks – ICANN 2006, pp. 35–44. Springer, Berlin (2006)CrossRefGoogle Scholar
  54. 54.
    Wang, G., Zhang, Y., Fei-Fei, L.: Using dependent regions for object categorization in a generative framework. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol. 2, pp. 1597–1604. IEEE, Piscataway (2006)Google Scholar
  55. 55.
    Wang, M., Zhang, W., Zhang, Y., Ji, X.: Detecting image spam based on cross entropy. In: 2011 Eighth Web Information Systems and Applications Conference, pp. 19–22 (2011).  https://doi.org/10.1109/WISA.2011.11
  56. 56.
    Wang, G., Wang, Z., Chen, Y., Zhao, W.: A robust non-rigid point set registration method based on asymmetric Gaussian representation. Comput. Vis. Image Underst. 141, 67–80 (2015)CrossRefGoogle Scholar
  57. 57.
    Xu, L., Jordan, M.I.: On convergence properties of the EM algorithm for Gaussian mixtures. Neural Comput. 8(1), 129–151 (1996)CrossRefGoogle Scholar
  58. 58.
    Xu, H., Yu, B.: Automatic thesaurus construction for spam filtering using revised back propagation neural network. Expert Syst. Appl. 37(1), 18 – 23 (2010)CrossRefGoogle Scholar
  59. 59.
    Yang, M.H., Ahuja, N.: Gaussian mixture model for human skin color and its applications in image and video databases. In: Storage and retrieval for image and video databases VII, vol. 3656, pp. 458–467. International Society for Optics and Photonics (1998)Google Scholar
  60. 60.
    Yang, J., Liao, X., Yuan, X., Llull, P., Brady, D.J., Sapiro, G., Carin, L.: Compressive sensing by learning a Gaussian mixture model from measurements. IEEE Trans. Image Process. 24(1), 106–119 (2015)MathSciNetCrossRefGoogle Scholar
  61. 61.
    Yin, D., Pan, J., Chen, P., Zhang, R.: Medical image categorization based on Gaussian mixture model. In: 2008 International Conference on BioMedical Engineering and Informatics, vol. 2, pp. 128–131 (2008)Google Scholar
  62. 62.
    Zhang, H., Berg, A.C., Maire, M., Malik, J.: SVM-KNN: Discriminative nearest neighbor classification for visual category recognition. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol. 2, pp. 2126–2136. IEEE, Piscataway (2006)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Muhammad Azam
    • 1
    Email author
  • Basim Alghabashi
    • 2
  • Nizar Bouguila
    • 2
  1. 1.Department of Electrical and Computer Engineering (ECE)Concordia UniversityMontrealCanada
  2. 2.Concordia Institute for Information Systems EngineeringConcordia UniversityMontrealCanada

Personalised recommendations