Bayesian Model Selection in Factor Analytic Models

  • Joyee Ghosh
  • David B. Dunson
Part of the Lecture Notes in Statistics book series (LNS, volume 192)


Factor analytic models are widely used in social science applications to study latent traits, such as intelligence, creativity, stress, and depression, that cannot be accurately measured with a single variable. In recent years, there has been a rise in the popularity of factor models due to their flexibility in characterizing multivari-ate data. For example, latent factor regression models have been used as a dimensionality reduction tool for modeling of sparse covariance structures in genomic applications (West, 2003; Carvalho et al. 2008). In addition, structural equation models and other generalizations of factor analysis are widely useful in epidemi-ologic studies involving complex health outcomes and exposures (Sanchez et al., 2005). Improvements in Bayesian computation permit the routine implementation of latent factor models via Markov chain Monte Carlo (MCMC) algorithms, and a very broad class of models can be fitted easily using the freely available software package WinBUGS. The literature on methods for fitting and inferences in latent factor models is vast (for recent books, see Loehlin, 2004; Thompson, 2004).


Markov Chain Monte Carlo Gibbs Sampler Marginal Likelihood Reversible Jump Markov Chain Monte Carlo Posterior Model Probability 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This research was supported by the Intramural Research Program of the NIH, National Institute of Environmental Health Sciences.


  1. Akaike, H. (1987). Factor analysis and AIC. Psychometrika 52, 317–332MathSciNetMATHCrossRefGoogle Scholar
  2. Arminger, G. (1998). A Bayesian approach to nonlinear latent variable models using the Gibbs sampler and the Metropolis-Hastings algorithm. Psychometrika 63, 271–300CrossRefGoogle Scholar
  3. Berger, J. and Pericchi, L. (1996). The intrinsic Bayes factor for model selection and prediction. Journal of the American Statistical Association 91, 109–122MathSciNetMATHCrossRefGoogle Scholar
  4. Berger, J. and Pericchi, L. (2001). Objective Bayesian methods for model selection: introduction and comparison [with discussion]. In: Model Selection, P. Lahiri (ed.). Institute of Mathematical Statistics Lecture Notes, Monograph Series Volume 38, Beachwood Ohio, 135–207CrossRefGoogle Scholar
  5. Berger, J.O., Ghosh, J.K. and Mukhopadhyay, N. (2003). Approximation and consistency of Bayes factors as model dimension grows. Journal of Statistical Planning and Inference 112, 241–258MathSciNetMATHCrossRefGoogle Scholar
  6. Carvalho, C., Lucas, J., Wang, Q., Nevins, J. and West, M. (2008). High-dimensional sparse factor modelling: applications in gene expression genomics. Journal of the American Statistical Association, to appearGoogle Scholar
  7. Chib, S. (1995). Marginal likelihoods from the Gibbs output. Journal of the American Statistical Association 90, 1313–1321MathSciNetMATHCrossRefGoogle Scholar
  8. DiCiccio, T.J., Kass, R., Raftery, A. and Wasserman, L. (1997). Computing Bayes factors by combining simulations and asymptotic approximations. Journal of the American Statistical Association 92, 903–915MathSciNetMATHCrossRefGoogle Scholar
  9. Gelfand, A.E. and Dey, D.K. (1994). Bayesian model choice: asymptotics and exact calculations. Journal of the Royal Statistical Society B, 501–514MathSciNetGoogle Scholar
  10. Gelfand, A.E., Sahu, S.K. and Carlin, B.P. (1995). Efficient parameterisations for normal linear mixed models. Biometrika 82, 479–488MathSciNetMATHCrossRefGoogle Scholar
  11. Gelman, A. (2006). Prior distributions for variance parameters in hierarchical models. Bayesian Analysis 3, 515–534MathSciNetGoogle Scholar
  12. Gelman, A. and Meng, X.L. (1998). Simulating normalizing constants: from importance sampling to bridge sampling to path sampling. Statistical Science 13, 163–185MathSciNetMATHCrossRefGoogle Scholar
  13. Gelman, A., van Dyk, D., Huang, Z. and Boscardin, W.J. (2007). Using redundant parameters to fit hierarchical models. Journal of Computational and Graphical Statistics, to appearGoogle Scholar
  14. Ghosh, J. and Dunson, D.B. (2007). Default priors and efficient posterior computation in Bayesian factor analysis. Journal of Computational and Graphical Statistics, revision requestedGoogle Scholar
  15. Green, P.J. (1995). Reversible jump Markov chain Monte Carlo and Bayesian model determination. Biometrika 82, 711–732MathSciNetMATHCrossRefGoogle Scholar
  16. Lee, S.Y. and Song, X.Y. (2002). Bayesian selection on the number of factors in a factor analysis model. Behaviormetrika 29, 23–40MathSciNetMATHCrossRefGoogle Scholar
  17. Liu, J. and Wu, Y.N. (1999). Parameter expansion for data augmentation. Journal of the American Statistical Association 94, 1264–1274MathSciNetMATHCrossRefGoogle Scholar
  18. Loehlin, J.C. (2004). Latent Variable Models: An Introduction to Factor, Path and Structural Equation Analysis. Lawrence Erlbaum Associates,Google Scholar
  19. Lopes, H.F. and West, M. (2004). Bayesian model assessment in factor analysis. Statistica Sinica 14, 41–67MathSciNetMATHGoogle Scholar
  20. Meng, X.L. and Wong, W.H. (1996). Simulating ratios of normalising constants via a simple identity. Statistica Sinica 11, 552–586MathSciNetGoogle Scholar
  21. Polasek, W. (1997). Factor analysis and outliers: a Bayesian approach. Discussion Paper, University of BaselGoogle Scholar
  22. Press, S.J. and Shigemasu, K. (1999). A note on choosing the number of factors. Communications in Statistics — Theory and Methods 28, 1653–1670MathSciNetMATHCrossRefGoogle Scholar
  23. Rowe, D.B. (1998). Correlated Bayesian factor analysis. Ph.D. Thesis, Department of Statistics, University of California, Riverside, CAGoogle Scholar
  24. Sanchez, B.N., Budtz-Jorgensen, E., Ryan, L.M. and Hu, H. (2005). Structural equation models: a review with applications to environmental epidemiology. Journal of the American Statistical Association 100, 1442–1455MathSciNetCrossRefGoogle Scholar
  25. Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics 6, 461–464MathSciNetMATHCrossRefGoogle Scholar
  26. Song, X.Y. and Lee, S.Y. (2001). Bayesian estimation and test for factor analysis model with continuous and polytomous data in several populations. British Journal of Mathematical & Statistical Psychology 54, 237–263CrossRefGoogle Scholar
  27. Thompson, B. (2004). Exploratory and Confirmatory Factor Analysis: Understanding Concepts and Applications. APA BooksGoogle Scholar
  28. West, M. (2003). Bayesian factor regression models in the “large p, small n” paradigm. In: Bayesian Statistics, Volume 7, J.M. Bernardo, M.J. Bayarri, J.O. Berger, A.P. Dawid, D. Heckerman, A.F.M. Smith and M. West (eds). Oxford University Press, OxfordGoogle Scholar
  29. Zhang, N.L. and Kocka, T. (2004). Effective dimensions of hierarchical latent class models. Journal of Artificial Intelligence Research 21, 1–17MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.Department of Statistical ScienceDuke UniversityDurham
  2. 2.Biostatistics BranchNational Institute of Environmental Health Sciences, RTPUSA

Personalised recommendations