Skip to main content
SpringerLink
Log in

Variational Bayesian Approximation Method for Classification and Clustering with a Mixture of Student-t Model

  • Conference paper
  • First Online:
Geometric Science of Information (GSI 2015)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 9389))

Included in the following conference series:

Abstract

Clustering, classification and Pattern Recognition in a set of data are between the most important tasks in statistical researches and in many applications. In this paper, we propose to use a mixture of Student-t distribution model for the data via a hierarchical graphical model and the Bayesian framework to do these tasks. The main advantages of this model is that the model accounts for the uncertainties of variances and covariances and we can use the Variational Bayesian Approximation (VBA) methods to obtain fast algorithms to be able to handle large data sets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Forgy, E.W.: Cluster analysis of multivariate data: efficiency vs interpretability of classifications. Biometrics 21, 768–769 (1965)

    Google Scholar 

  2. MacKay, D.J.C.: A practical Bayesian framework for backpropagation networks. Neural Comput. 4, 448–472 (1992)

    Article  Google Scholar 

  3. Redner, R., Walker, H.: Mixture densities, maximum likelihood and the em algorithm. SIAM Rev. 26(2), 195–239 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  4. Husmeier, D., Penny, W., Roberts, S.: Empirical evaluation of Bayesian sampling for neural classifiers. In: Niklason, M.L., Ziemke, T. (eds.): ICANN 98: Proceedings of the 8th International Conference on Artificial Neural Networks (1998)

    Google Scholar 

  5. Lee, T., Lewicki, M., Sejnowski, T.: Independent component analysis using an extended infomax algorithm for mixed sub-gaussian and super-gaussian sources. Neural Comput. 11, 409–433 (1999)

    Article  Google Scholar 

  6. Hyvärinen, A., Oja, E.: Independent component analysis: algorithms and applications. Neural Netw. 13, 411–430 (2000)

    Article  Google Scholar 

  7. Ma, J., Xu, L., Jordan, M.I.: Asymptotic convergence rate of the EM algorithm for Gaussian mixtures. Neural Comput. 12, 2881–2907 (2001)

    Article  Google Scholar 

  8. Nielsen, F., Nock, R.: Clustering multivariate normal distributions. In: Nielsen, F. (ed.) ETVC 2008. LNCS, vol. 5416, pp. 164–174. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  9. Quandt, R., Ramsey, J.: Estimating mixtures of normal distributions and switching regressions. J. Am. Stat. Assoc. 73, 2 (1978)

    MathSciNet  MATH  Google Scholar 

  10. Hathaway, R.: A constrained formulation of maximum-likelihood estimation for normal mixture distributions. Ann. Stat. 13(2), 795–800 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  11. Hastie, T., Tibshirani, R.: Discriminant analysis by gaussian mixture. J. Roy. Stat. Soc. 58, 155–176 (1996)

    MathSciNet  MATH  Google Scholar 

  12. Dempster, A., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Stat. Soc. Ser. B 39(1), 1–38 (1977)

    MathSciNet  MATH  Google Scholar 

  13. Neal, R., Hinton, G.: A view of the EM algorithm that justifies incremental, sparse, and other variants. Learn. Graph. Models 89, 355–368 (1998)

    Article  MATH  Google Scholar 

  14. Tipping, M.E., Bishop, C.M.: Mixtures of probabilistic principal components analysis. Neural Comput. 11, 443–482 (1999)

    Article  MathSciNet  Google Scholar 

  15. Jordan, M., Ghahramani, Z., Jaakkola, T., Saul, L.: An introduction to variational methods for graphical models. Mach. Learn. 37, 183–233 (2006)

    Article  MATH  Google Scholar 

  16. Jaakkola, T.S., Jordan, M.I.: Bayesian parameter estimation via variational methods. Stat. Comput. 10, 25–37 (2000)

    Article  Google Scholar 

  17. Friston, K., Penny, W.: Bayesian inference and posterior probability maps. In: Proceedings of the 9th International Conference on Neural Information Processing (ICONIP 2002), pp. 413–417 (2002)

    Google Scholar 

  18. Winn, J., Bishop, C.M., Jaakkola, T.: Variational message passing. J. Mach. Learn. Res. 6, 661–694 (2005)

    MathSciNet  MATH  Google Scholar 

  19. David, M.B., Michael, I.J.: Variational inference for the dirichlet process mixtures. Bayesian Anal. 1(1), 121–144 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  20. Beal, M.: Variational Algorithms for Approximate Bayesian Inference. Ph.D. thesis, Gatsby Computational Neuroscience Unit, University College London (2003)

    Google Scholar 

  21. Beal, M., Ghahramani, Z.: Variational Bayesian learning of directed graphical models with hidden variables. Bayesian Stat. 1(4), 793–832 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  22. Kim, H., Ghahramani, Z.: Bayesian gaussian process classification with the em-ep algorithm. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1948–1959 (2006)

    Article  Google Scholar 

  23. Nasios, N., Bors, A.: Variational learning for gaussian mixture models. IEEE Trans. Syst. Man Cybern. Part B 36, 849–862 (2006)

    Article  Google Scholar 

  24. Ghahramani, Z., Griffiths, T., Sollich, P.: Bayesian nonparametric latent feature models. Bayesian Stat. 8, 1–25 (2007)

    MathSciNet  MATH  Google Scholar 

  25. McGrory, C., Titterington, D.: Variational approximations in Bayesian model selection for finite mixture distributions. Comput. Stat. Data Anal. 51, 5352–5367 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  26. Qiao, Z., Zhou, L., Huang, J.Z.: Sparse linear discriminant analysis with applications to high dimensional low sample size data. Int. J. Appl. Math. 39, 48–60 (2008)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgment

This work has been supported partially by the C5SYS (https://www.erasysbio.net/index.php?index=272) project of ERASYSBIO.

Author information

Authors and Affiliations

  1. Laboratoire des signaux et systèmes (L2S), CNRS-CentraleSupélec-UNIV PARIS SUD, Gif-sur-yvette, France

    Ali Mohammad-Djafari

Authors
  1. Ali Mohammad-Djafari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ali Mohammad-Djafari .

Editor information

Editors and Affiliations

  1. Bâtiment Alan Turing, CS35003, École Polytechnique, Palaiseau, France

    Frank Nielsen

  2. Thales Land\& Air Systems, Limours, France

    Frédéric Barbaresco

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Mohammad-Djafari, A. (2015). Variational Bayesian Approximation Method for Classification and Clustering with a Mixture of Student-t Model. In: Nielsen, F., Barbaresco, F. (eds) Geometric Science of Information. GSI 2015. Lecture Notes in Computer Science(), vol 9389. Springer, Cham. https://doi.org/10.1007/978-3-319-25040-3_77

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-25040-3_77

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-25039-7

  • Online ISBN: 978-3-319-25040-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation