Relabelling in Bayesian mixture models by pivotal units

  • Leonardo Egidi
  • Roberta Pappadà
  • Francesco Pauli
  • Nicola Torelli
Article

Abstract

Label switching is a well-known and fundamental problem in Bayesian estimation of finite mixture models. It arises when exploring complex posterior distributions by Markov Chain Monte Carlo (MCMC) algorithms, because the likelihood of the model is invariant to the relabelling of mixture components. If the MCMC sampler randomly switches labels, then it is unsuitable for exploring the posterior distributions for component-related parameters. In this paper, a new procedure based on the post-MCMC relabelling of the chains is proposed. The main idea of the method is to perform a clustering technique on the similarity matrix, obtained through the MCMC sample, whose elements are the probabilities that any two units in the observed sample are drawn from the same component. Although it cannot be generalized to any situation, it may be handy in many applications because of its simplicity and very low computational burden.

Keywords

Label switching Complex posterior distributions MCMC Finite mixture model 

Supplementary material

11222_2017_9774_MOESM1_ESM.pdf (566 kb)
Supplementary material 1 (pdf 565 KB)

References

  1. Celeux, G., Hurn, M., Robert, C.P.: Computational and inferential difficulties with mixture posterior distributions. J. Am. Stat. Assoc. 95(451), 957–970 (2000)MathSciNetCrossRefMATHGoogle Scholar
  2. Chung, H., Loken, E., Schafer, J.L.: Difficulties in drawing inferences with finite-mixture models. Am. Stat. 58(2), 152–158 (2004)CrossRefGoogle Scholar
  3. Egidi, L., Pappadà, R., Pauli, F., Torelli, N.: Maxima units search (MUS) algorithm: methodology and applications (2016). ArXiv e-prints arXiv:1611.01069
  4. Grün, B.: Bayesmix: bayesian mixture models with JAGS. R package version 0.7-2. http://CRAN.R-project.org/package=bayesmix (2011)
  5. Jasra, A.: Bayesian inference for mixture models via Monte Carlo computation. Ph.D. thesis, Imperial College London (University of London) (2006)Google Scholar
  6. Marin, J.M., Robert, C.P.: Bayesian Core: A Practical Approach to Computational Bayesian Statistics. Springer, New York (2007)MATHGoogle Scholar
  7. Marin, J.M., Mengersen, K., Robert, C.P.: Bayesian modelling and inference on mixtures of distributions. Handb. Stat. 25, 459–507 (2005)MathSciNetCrossRefGoogle Scholar
  8. McLachlan, J., Peel, D.: Finite Mixture Models. Wiley, New York (2000)CrossRefMATHGoogle Scholar
  9. Papastamoulis, P.: Label.switching: an R package for dealing with the label switching problem in MCMC outputs. J. Stat. Soft. 69(1), 1–24 (2016). doi:10.18637/jss.v069.c01 Google Scholar
  10. Papastamoulis, P., Iliopoulos, G.: An artificial allocations based solution to the label switching problem in Bayesian analysis of mixtures of distributions. J. Comput. Graph. Stat. 19(2), 313–331 (2010)Google Scholar
  11. Puolamäki, K., Kaski, S.: Bayesian solutions to the label switching problem. In: Adams, N.M., Robardet, C., Siebes, A., Boulicaut, J.-F. (Eds.) Advances in Intelligent Data Analysis VIII 8th International Symposium on Intelligent Data Analysis, IDA 2009, Lyon, France, pp. 381–392. Springer, Berlin (2009). http://www.springer.com/gp/book/9783642039140
  12. Rodríguez, C.E., Walker, S.G.: Label switching in Bayesian mixture models: deterministic relabeling strategies. J. Comput. Graph. Stat. 23(1), 25–45 (2014)MathSciNetCrossRefGoogle Scholar
  13. Sperrin, M., Jaki, T., Wit, E.: Probabilistic relabelling strategies for the label switching problem in Bayesian mixture models. Stat. Comput. 20(3), 357–366 (2010)MathSciNetCrossRefGoogle Scholar
  14. Stephens, M.: Dealing with label switching in mixture models. J. R. Stat. S.: Ser. B (Stat. Methodol.) 62(4), 795–809 (2000)MathSciNetCrossRefMATHGoogle Scholar
  15. Titterington, D.M., Smith, A.F., Makov, U.E.: Statistical Analysis of Finite Mixture Distributions. Wiley, New York (1985)MATHGoogle Scholar
  16. Yao, W., Li, L.: An online Bayesian mixture labelling method by minimizing deviance of classification probabilities to reference labels. J. Stat. Comput. Simul. 84(2), 310–323 (2014)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  • Leonardo Egidi
    • 1
  • Roberta Pappadà
    • 2
  • Francesco Pauli
    • 2
  • Nicola Torelli
    • 2
  1. 1.Dipartimento di Scienze StatisticheUniversità degli Studi di PadovaPaduaItaly
  2. 2.Dipartimento di Scienze Economiche, Aziendali, Matematiche e Statistiche ‘Bruno de Finetti’Università degli Studi di TriesteTriesteItaly

Personalised recommendations