A Frequentist Inference Method Based on Finite Bivariate and Multivariate Beta Mixture Models

Abstract

Modern technological improvement, revolutionized computers, progress in scientific methods, and other related factors led to generate a massive volume of structured and unstructured data. Such valuable data has potential to be mined for information retrieval and analyzed computationally to reveal patterns, trends, and associations that lead to better decisions and strategies. Thus, machine learning and specifically, unsupervised learning methods have become the topic of interest of much recent researches in data engineering. Finite mixture models as unsupervised learning methods, namely clustering, are considered as capable techniques for discovery, extraction, and analysis of knowledge from data. Traditionally Gaussian mixture model (GMM) has drawn lots of attention in previous literature and has been studied extensively. However, other distributions demonstrate more flexibility and convenience in modeling and describing data.

The novel aspect of this work is to develop a framework to learn mixture models based on bivariate and multivariate Beta distributions. Moreover, we tackle simultaneously the problems of parameters estimation, cluster validation, or model selection which are principal challenges in deployment of mixture models. The effectiveness, utility, and advantages of the proposed method are illustrated through extensive empirical results using real datasets and challenging applications involving image segmentation, sentiment analysis, credit approval, and medical inference.

Appendix

Proof of Eq.(9.12):

$$\displaystyle \begin{aligned} \mathcal{L} (\Theta,Z,\mathcal{X})&= \sum_{j=1}^{M} \sum_{n=1}^{N} \hat{Z}_{nj}\Big(\log p_j + \log p({\mathbf{X}}_n|{\mathbf{a}}_j)\Big)\\ &= \vspace{0.7cm} \sum_{j=1}^{M}\sum_{n=1}^{N}\ \hat{Z}_{nj}\Bigg(\log p_j + \log \Big(\frac{\prod_{i=1}^{d} X_{ni}^{(a_{ji}-1)}}{\prod_{i=1}^{d}(1-X_{ni})^{(a_{ji}+1)}} \\ &\quad \qquad \qquad \qquad \times \Big[ 1+\sum_{i=1}^{d}\frac{ X_{ni}}{(1-X_{ni})}\Big]^{-a_j} \times \frac{\Gamma(\sum_{i=1}^{d}a_{ji})}{\prod_{i=1}^{d} \Gamma(a_{ji})} \Big)\Bigg)\\&= \vspace{0.5cm} \sum_{j=1}^{M} \sum_{n=1}^{N} \hat{Z}_{nj} \Bigg( \log {p_j} + \log \Big( \prod_{i=1}^{d} X_{ni}^{(a_{ji}-1)} \Big) \\&\quad \qquad \qquad \qquad - \log \big( \prod_{i=1}^{d}(1-X_{ni})^{(a_{ji}+1)} \big) + \log \big( \Gamma(a_j) \big)\\&\quad \qquad \qquad \qquad \vspace{0.45cm} -\log\prod_{i=1}^{d} \Gamma(a_{ji})+\log\Big[ 1+\sum_{i=1}^{d}\frac{ X_{ni}}{(1-X_{ni})}\Big]^{-a_j} \Bigg)\\&= \vspace{0.45cm} \sum_{j=1}^{M}\sum_{n=1}^{N}\ \hat{Z}_{nj} \Bigg(\log p_j +\sum_{i=1}^{d} (a_{ji}-1)\big(\log(X_{ni})\big)\\&\quad \qquad \qquad \qquad - \vspace{0.15cm} \sum_{i=1}^{d}(a_{ji}+1)\big(\log(1-X_{ni})\big) \\ \vspace{0.45cm} &\quad \qquad \qquad \qquad +\log\big(\Gamma (a_{j})\big)- \sum_{i=1}^{d}\log\big( \Gamma(a_{ji})\big)\\&\quad \qquad \qquad \qquad -a_{j}\log\Big( \Big[ 1+\sum_{i=1}^{d}\frac{ X_{ni}}{(1-X_{ni})}\Big]\Big)\Bigg) \end{aligned} $$