Relabelling in Bayesian mixture models by pivotal units

Abstract

Label switching is a well-known and fundamental problem in Bayesian estimation of finite mixture models. It arises when exploring complex posterior distributions by Markov Chain Monte Carlo (MCMC) algorithms, because the likelihood of the model is invariant to the relabelling of mixture components. If the MCMC sampler randomly switches labels, then it is unsuitable for exploring the posterior distributions for component-related parameters. In this paper, a new procedure based on the post-MCMC relabelling of the chains is proposed. The main idea of the method is to perform a clustering technique on the similarity matrix, obtained through the MCMC sample, whose elements are the probabilities that any two units in the observed sample are drawn from the same component. Although it cannot be generalized to any situation, it may be handy in many applications because of its simplicity and very low computational burden.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  1. Celeux, G., Hurn, M., Robert, C.P.: Computational and inferential difficulties with mixture posterior distributions. J. Am. Stat. Assoc. 95(451), 957–970 (2000)

    MathSciNet  Article  MATH  Google Scholar 

  2. Chung, H., Loken, E., Schafer, J.L.: Difficulties in drawing inferences with finite-mixture models. Am. Stat. 58(2), 152–158 (2004)

    Article  Google Scholar 

  3. Egidi, L., Pappadà, R., Pauli, F., Torelli, N.: Maxima units search (MUS) algorithm: methodology and applications (2016). ArXiv e-prints arXiv:1611.01069

  4. Grün, B.: Bayesmix: bayesian mixture models with JAGS. R package version 0.7-2. http://CRAN.R-project.org/package=bayesmix (2011)

  5. Jasra, A.: Bayesian inference for mixture models via Monte Carlo computation. Ph.D. thesis, Imperial College London (University of London) (2006)

  6. Marin, J.M., Robert, C.P.: Bayesian Core: A Practical Approach to Computational Bayesian Statistics. Springer, New York (2007)

    Google Scholar 

  7. Marin, J.M., Mengersen, K., Robert, C.P.: Bayesian modelling and inference on mixtures of distributions. Handb. Stat. 25, 459–507 (2005)

    MathSciNet  Article  Google Scholar 

  8. McLachlan, J., Peel, D.: Finite Mixture Models. Wiley, New York (2000)

    Google Scholar 

  9. Papastamoulis, P.: Label.switching: an R package for dealing with the label switching problem in MCMC outputs. J. Stat. Soft. 69(1), 1–24 (2016). doi:10.18637/jss.v069.c01

    Google Scholar 

  10. Papastamoulis, P., Iliopoulos, G.: An artificial allocations based solution to the label switching problem in Bayesian analysis of mixtures of distributions. J. Comput. Graph. Stat. 19(2), 313–331 (2010)

  11. Puolamäki, K., Kaski, S.: Bayesian solutions to the label switching problem. In: Adams, N.M., Robardet, C., Siebes, A., Boulicaut, J.-F. (Eds.) Advances in Intelligent Data Analysis VIII 8th International Symposium on Intelligent Data Analysis, IDA 2009, Lyon, France, pp. 381–392. Springer, Berlin (2009). http://www.springer.com/gp/book/9783642039140

  12. Rodríguez, C.E., Walker, S.G.: Label switching in Bayesian mixture models: deterministic relabeling strategies. J. Comput. Graph. Stat. 23(1), 25–45 (2014)

    MathSciNet  Article  Google Scholar 

  13. Sperrin, M., Jaki, T., Wit, E.: Probabilistic relabelling strategies for the label switching problem in Bayesian mixture models. Stat. Comput. 20(3), 357–366 (2010)

    MathSciNet  Article  Google Scholar 

  14. Stephens, M.: Dealing with label switching in mixture models. J. R. Stat. S.: Ser. B (Stat. Methodol.) 62(4), 795–809 (2000)

    MathSciNet  Article  MATH  Google Scholar 

  15. Titterington, D.M., Smith, A.F., Makov, U.E.: Statistical Analysis of Finite Mixture Distributions. Wiley, New York (1985)

    Google Scholar 

  16. Yao, W., Li, L.: An online Bayesian mixture labelling method by minimizing deviance of classification probabilities to reference labels. J. Stat. Comput. Simul. 84(2), 310–323 (2014)

    MathSciNet  Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Leonardo Egidi.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 565 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Egidi, L., Pappadà, R., Pauli, F. et al. Relabelling in Bayesian mixture models by pivotal units. Stat Comput 28, 957–969 (2018). https://doi.org/10.1007/s11222-017-9774-2

Download citation

Keywords

  • Label switching
  • Complex posterior distributions
  • MCMC
  • Finite mixture model