Advertisement

A Frequentist Inference Method Based on Finite Bivariate and Multivariate Beta Mixture Models

  • Narges ManouchehriEmail author
  • Nizar Bouguila
Chapter
Part of the Unsupervised and Semi-Supervised Learning book series (UNSESUL)

Abstract

Modern technological improvement, revolutionized computers, progress in scientific methods, and other related factors led to generate a massive volume of structured and unstructured data. Such valuable data has potential to be mined for information retrieval and analyzed computationally to reveal patterns, trends, and associations that lead to better decisions and strategies. Thus, machine learning and specifically, unsupervised learning methods have become the topic of interest of much recent researches in data engineering. Finite mixture models as unsupervised learning methods, namely clustering, are considered as capable techniques for discovery, extraction, and analysis of knowledge from data. Traditionally Gaussian mixture model (GMM) has drawn lots of attention in previous literature and has been studied extensively. However, other distributions demonstrate more flexibility and convenience in modeling and describing data.

The novel aspect of this work is to develop a framework to learn mixture models based on bivariate and multivariate Beta distributions. Moreover, we tackle simultaneously the problems of parameters estimation, cluster validation, or model selection which are principal challenges in deployment of mixture models. The effectiveness, utility, and advantages of the proposed method are illustrated through extensive empirical results using real datasets and challenging applications involving image segmentation, sentiment analysis, credit approval, and medical inference.

Keywords

Unsupervised learning Mixture models Maximum likelihood Model selection Bivariate Beta Minimum message length 

References

  1. 1.
    Diaz-Rozo, J., Bielza, C., Larrañaga, P.: Clustering of data streams with dynamic gaussian mixture models: an IoT application in industrial processes. IEEE Internet Things J. 5(5), 3533 (2018)CrossRefGoogle Scholar
  2. 2.
    Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)zbMATHGoogle Scholar
  3. 3.
    Olkin, I., Liu, R.: A bivariate beta distribution. Statist. Probab. Lett. 62(4), 407–412 (2003)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Bouguila, N.: Clustering of count data using generalized Dirichlet multinomial distributions. IEEE Trans. Knowl. Data Eng. 20(4), 462–474 (2008)CrossRefGoogle Scholar
  5. 5.
    Bouguila, N., ElGuebaly, W.: Integrating spatial and color information in images using a statistical framework. Expert Syst. Appl. 37(2), 1542–1549 (2010)CrossRefGoogle Scholar
  6. 6.
    Bouguila, N., Ziou, D.: A Dirichlet process mixture of generalized Dirichlet distributions for proportional data modeling. IEEE Trans. Neural Netw. 21(1), 107–122 (2010)CrossRefGoogle Scholar
  7. 7.
    Bouguila, N.: Spatial color image databases summarization. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 953–956 (2007)Google Scholar
  8. 8.
    Bouguila, N., Amayri, O.: A discrete mixture-based kernel for SVMs: application to spam and image categorization. Inf. Process. Manag. 45, 631–642 (2009)CrossRefGoogle Scholar
  9. 9.
    Bouguila, N., Ziou, D.: Dirichlet-based probability model applied to human skin detection [image skin detection]. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 521–524 (2004)Google Scholar
  10. 10.
    Bouguila, N., Ziou, D.: Improving content based image retrieval systems using finite multinomial Dirichlet mixture. In: 14th IEEE Signal Processing Society Workshop Machine Learning for Signal Processing, pp. 23–32 (2004)Google Scholar
  11. 11.
    Bouguila, N., Ziou, D.: A powerful finite mixture model based on the generalized Dirichlet distribution: unsupervised learning and applications. In: 17th International Conference on Pattern Recognition, pp. 280–283 (2004)Google Scholar
  12. 12.
    Bouguila, N., Ziou, D.: Using unsupervised learning of a finite Dirichlet mixture model to improve pattern recognition applications. Pattern Recogn. Lett. 26(12), 1916–1925 (2005)CrossRefGoogle Scholar
  13. 13.
    Bouguila, N., Ziou, D.: A Dirichlet process mixture of Dirichlet distributions for classification and prediction. In: IEEE Workshop on Machine Learning for Signal Processing, pp. 297–302 (2008)Google Scholar
  14. 14.
    Bouguila, N., Ziou, D.: A countably infinite mixture model for clustering and feature selection. Knowl. Inf. Syst. 33(2), 351–370 (2012)CrossRefGoogle Scholar
  15. 15.
    Bouguila, N., Ziou, D., Hammoud, R.I.: On Bayesian analysis of a finite generalized Dirichlet mixture via a Metropolis-within-Gibbs sampling. Pattern. Anal. Applic. 12(2) (2009)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Olkin, I., Trikalinos, T.A.: Constructions for a bivariate beta distribution. Statist. Probab. Lett. 96, 54–60 (2015)MathSciNetCrossRefGoogle Scholar
  17. 17.
    McLachlan, G.J.: Mixture models in statistics. In: International Encyclopedia of the Social and Behavioral Sciences, pp. 624–628 (2015)CrossRefGoogle Scholar
  18. 18.
    Ganesaligman, S.: Classification and mixture approaches to clustering via maximum likelihood. Appl. Stat. 38(3), 455–466 (1989)MathSciNetCrossRefGoogle Scholar
  19. 19.
    McLachlan, G.J., Krishnan, T.: The EM Algorithm and Extensions. Wiley-Interscience, Hoboken (2008)CrossRefGoogle Scholar
  20. 20.
    Bouguila, N., Ziou, D.: Unsupervised selection of a finite Dirichlet mixture model: an MML-based approach. IEEE Trans. Knowl. Data Eng. 18(8), 993 (2006)CrossRefGoogle Scholar
  21. 21.
    Bouguila, N., Ziou, D.: On fitting finite Dirichlet mixture using ECM and MML. In: Third International Conference on Advances in Pattern Recognition, vol. 3686, pp. 172–182 (2005)Google Scholar
  22. 22.
    Bouguila, N., Ziou, D.: MML-based approach for finite Dirichlet mixture estimation and selection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), pp. 42–51 (2005)Google Scholar
  23. 23.
    Bouguila, N., Ziou, D.: Online clustering via finite mixtures of Dirichlet and minimum message length. Eng. Appl. Artif. Intell. 19, 371–379 (2006)CrossRefGoogle Scholar
  24. 24.
    Bouguila, N., Ziou, D.: Unsupervised selection of a finite Dirichlet mixture model: an MML-based approach. IEEE Trans. Knowl. Data Eng. 18(8), 993–1009 (2006)CrossRefGoogle Scholar
  25. 25.
    Bouguila, N., Ziou, D.: High-dimensional unsupervised selection and estimation of a finite generalized Dirichlet mixture model based on minimum message length. IEEE Trans. Pattern Anal. Mach. Intell. 29(10), 1716 (2007)CrossRefGoogle Scholar
  26. 26.
    Wallace, C.S., Dowe, D.L.: MML clustering of multistate, Poisson, von Mises circular and gaussian distributions. Stat. Comput. 10(1), 73–83 (2000)CrossRefGoogle Scholar
  27. 27.
    Baxter, R.A.: Minimum Message Length Inference: Theory and Applications. Monash University, Clayton (1996)Google Scholar
  28. 28.
    Baxter, R.A., Oliver, J.J.: Finding overlapping components with MML. Stat. Comput. 3(1), 5–16 (2000)CrossRefGoogle Scholar
  29. 29.
    Sefidpour, A., Bouguila, N.: Spatial color image segmentation based on finite non-Gaussian mixture models. Expert Syst. Appl. 39(10), 8993–9001 (2012)CrossRefGoogle Scholar
  30. 30.
    Figueiredo, M.A.T., Jain, A.K.: Unsupervised learning of finite mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 24(3), 381–396 (2002)CrossRefGoogle Scholar
  31. 31.
    Bouguila, N., Ziou, D.: High-dimensional unsupervised selection and estimation of a finite generalized Dirichlet mixture model based on minimum message length. IEEE Trans. Pattern Anal. Mach. Intell. 29(10), 1716 (2007)CrossRefGoogle Scholar
  32. 32.
    Agusta, Y., Dowe, D.L.: Unsupervised learning of gamma mixture models using minimum message length. In: Hamza, M.H. (ed.) Proceeding Third ASTED Conference Artificial Intelligence and Applications, pp. 457–462 (2003)Google Scholar
  33. 33.
    Jefferys, W.H., Berger, J.O.: Ockham’s razor and Bayesian analysis. Am. Sci. 80(1), 64–72 (1992)Google Scholar
  34. 34.
    UCI Repository Data Set (1999). https://archive.ics.uci.edu/ml/machine-learningdatabases. Accessed 2 August 1999
  35. 35.
    The Berkeley Segmentation Dataset and Benchmark Dataset [Online]. https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/
  36. 36.
    Hubert, L., Arabie, P.: Comparing partitions. J. Classif. 2, 193–218 (1985)CrossRefGoogle Scholar
  37. 37.
    Strehl, A., Joydeep, G.: Cluster ensembles—a knowledge reuse framework for combining multiple partitions. J. Mach. Learn. Res. 3, 583–617 (2002)MathSciNetzbMATHGoogle Scholar
  38. 38.
    Vinh, N.X., Epps, J., Bailey, J.: Information theoretic measures for clusterings comparison. In: Proceedings of the 26th Annual International Conference on Machine Learning-ICML (2009)Google Scholar
  39. 39.
    Vinh, N.X., Epps, J., Bailey, J.: Information theoretic measures for clusterings comparison: variants, properties, normalization and correction for chance. J. Mach. Learn. Res. 11, 2837–2854 (2010)MathSciNetzbMATHGoogle Scholar
  40. 40.
    Yang, Z., Algesheimer, R., Tessone, C.J.: A comparative analysis of community detection algorithms on artificial networks. Sci. Rep. 6 (2016)Google Scholar
  41. 41.
    Rosenberg, A., Hirschberg, J.: V-Measure a conditional entropy-based external cluster evaluation measure. In: Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 410–420 (2007)Google Scholar
  42. 42.
    Becker, H.: Identification and Characterization of Events in Social Media. PhD Thesis (2011)Google Scholar
  43. 43.
    Caliński, T., Harabasz, J.: A dendrite method for cluster analysis. Commun. Stat. Theory Methods 3, 1–27 (1974)MathSciNetCrossRefGoogle Scholar
  44. 44.
    Tan, P.N., Steinbach, M., Kumar, V.: Introduction to Data Mining. Addison-Wesley Longman Publishing Co., Boston (2005)Google Scholar
  45. 45.
    Jaccard, P.: Étude comparative de la distribution florale dans une portion des Alpes et des Jura. Bull. Soc. Vaud. Sci. Nat. 37, 547–579 (1901)Google Scholar
  46. 46.
    Jaccard, P.: The Distribution of the flora in the alpine zone. New Phytol. 11, 37–50 (1912)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Department of Electrical and Computer Engineering (ECE)Concordia UniversityMontrealCanada
  2. 2.Concordia Institute for Information Systems EngineeringConcordia UniversityMontrealCanada

Personalised recommendations