Bayesian Model Uncertainty in Mixed Effects Models

Part of the Lecture Notes in Statistics book series (LNS, volume 192)


Random effects models are widely used in analyzing dependent data, which are collected routinely in a broad variety of application areas. For example, longitudinal studies collect repeated observations for each study subject, while multi-center studies collect data for patients nested within study centers. In such settings, it is natural to suppose that dependence arises due to the impact of important unmeasured predictors that may interact with measured predictors. This viewpoint naturally leads to random effects models in which the regression coefficients vary across the different subjects. In this chapter, we use the term “subject” broadly to refer to the independent experimental units. For example, in longitudinal studies, the subjects are the individuals under study, while in multi-center studies the subjects correspond to the study centers.


Markov Chain Monte Carlo Marginal Likelihood Markov Chain Monte Carlo Algorithm Markov Chain Monte Carlo Chain Posterior Model Probability 



This work was supported in part by the Intramural Research Program of the NIH, National Institute of Environmental Health Sciences, and by the National Science Foundation under Agreement No. DMS-0112069 with the Statistical and Mathematical Sciences Institute (SAMSI). The authors are grateful to Merlise Clyde and Abel Rodriguez at Duke University and to the participants of the Model Uncertainty working group of the SAMSI Program on Latent Variable Modeling in the Social Sciences.


  1. Agresti, A. (1990). Categorical Data Analysis. New York: WileyMATHGoogle Scholar
  2. Albert, J. H. and Chib, S. (1993). Bayesian analysis of binary and polychotomous response data. Journal of the American Statistical Association 88, 669–679MathSciNetMATHCrossRefGoogle Scholar
  3. Albert, J. H. and Chib, S. (2001). Sequential ordinal modeling with applications to survival data. Biometrics 57, 829–836MathSciNetMATHCrossRefGoogle Scholar
  4. Berger, J. O., Ghosh, J. K. and Mukhopadhyay, N. (2003). Approximations and consistency of Bayes factors as model dimension grows. Journal of Statistical Planning and Inference 112, 241–258MathSciNetMATHCrossRefGoogle Scholar
  5. Berger, J. O. and Pericchi, L. R. (2001). Objective Bayesian methods for model selection: Introduction and comparison. In Lahiri, P., editor, Model Selection, volume 38 of IMS Lecture Notes – Monograph Series, pages 135–193. Institute of Mathematical StatisticsGoogle Scholar
  6. Breslow, N. (2003). Whither PQL? UW Biostatistics Working Paper Series Working Paper 192Google Scholar
  7. Breslow, N. and Clayton, D. (1993). Approximate inference in generalized linear mixed models. Journal of the American Statistical Association 88, 9–25MATHGoogle Scholar
  8. Cai, B. and Dunson, D. B. (2006). Bayesian covariance selection in generalized linear mixed models. Biometrics 62, 446–457MathSciNetMATHCrossRefGoogle Scholar
  9. Chen, Z. and Dunson, D. B. (2003). Random effects selection in linear mixed models. Biometrics 59, 762–769MathSciNetMATHCrossRefGoogle Scholar
  10. Chib, S. (1995). Marginal likelihood from the Gibbs output. Journal of the American Statistical Association 90, 1313–1321MathSciNetMATHCrossRefGoogle Scholar
  11. Chib, S. and Greenberg, E. (1998). Bayesian analysis of multivariate probit models. Biometrika 85, 347–361MATHCrossRefGoogle Scholar
  12. Chung, Y. and Dey, D. (2002). Model determination for the variance component model using reference priors. Journal of Statistical Planning and Inference 100, 49–65MathSciNetMATHCrossRefGoogle Scholar
  13. Clyde, M. and George, E. I. (2004). Model uncertainty. Statistical Science 19, 81–94MathSciNetMATHCrossRefGoogle Scholar
  14. Fernandez, C., Ley, E. and Steel, M. F. (2001). Benchmark priors for Bayesian model averaging. Journal of Econometrics 100, 381–427MathSciNetMATHCrossRefGoogle Scholar
  15. Foster, D. P. and George, E. I. (1994). The risk inflation criterion for multiple regression. Annals of Statistics 22, 1947–1975MathSciNetMATHCrossRefGoogle Scholar
  16. Gelfand, A. E. and Smith, A. (1990). Sampling-based approaches to calculating marginal densities. Journal of the American Statistical Association 85, 398–409MathSciNetMATHCrossRefGoogle Scholar
  17. Gelfand, A. E., Sahu, S. K. and Carlin, B. P. (1996). Efficient parameterizations for generalized linear mixed models. Bayesian Statistics 5,Google Scholar
  18. Gelman, A. (2005). Prior distributions for variance parameters in hierarchical models. Bayesian Analysis 1, 1–19Google Scholar
  19. George, E. I. and McCulloch, R. E. (1993). Variable selection via Gibbs sampling. Journal of the American Statistical Association 88, 881–889CrossRefGoogle Scholar
  20. George, E. I. and McCulloch, R. E. (1997). Approaches for Bayesian variable selection. Statistica Sinica 7, 339–374MATHGoogle Scholar
  21. Gerlach, R., Bird, R. and Hall, A. (2002). Bayesian variable selection in logistic regression: Predicting company earnings direction. Australian and New Zealand Journal of Statistics 42, 155–168MathSciNetGoogle Scholar
  22. Geweke, J. (1996). Variable selection and model comparison in regression. In Bayesian Statistics 5 – Proceedings of the Fifth Valencia International Meeting, pages 609–620Google Scholar
  23. Gilks, W., Wang, C., Yvonnet, B. and Coursaget, P. (1993). Random-effects models for longitudinal data using Gibbs sampling. Biometrics 49, 441–453MATHCrossRefGoogle Scholar
  24. Green, P. J. (1995). Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82, 711–732MathSciNetMATHCrossRefGoogle Scholar
  25. Green, P. J. (1997). Discussion of ”The EM algorithm – an old folk song sung to a fast new tune,” by Meng and van Dyk. Journal of the Royal Statistical Society, Series B 59, 554–555Google Scholar
  26. Hall, D. and Praestgaard, J. (2001). Order-restricted score tests for homogeneity in generalised linear and nonlinear mixed models. Biometrika 88, 739–751MathSciNetMATHCrossRefGoogle Scholar
  27. Hastings, W. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57, 97–109MATHCrossRefGoogle Scholar
  28. Hobert, J. P. and Casella, G. (1996). The effect of improper priors on Gibbs sampling in hierarchical linear mixed models. Journal of the American Statistical Association 91, 1461–1473MathSciNetMATHCrossRefGoogle Scholar
  29. Holmes, C. and Knorr-Held, L. (2003). Efficient simulation of Bayesian logistic regression models. Technical report, Ludwig Maximilians University MunichGoogle Scholar
  30. Jang, W. and Lim, J. (2005). Estimation bias in generalized linear mixed models. Technical report, Institute for Statistics and Decision Sciences, Duke UniversityGoogle Scholar
  31. Jiang, J., Rao, J., Gu, Z. and Nguyen, T. (2008). Fence methods for mixed model selection. Annals of Statistics, to appearGoogle Scholar
  32. Johnson, V. E. and Albert, J. H. (1999). Ordinal Data Modeling. Berlin Heidelberg New York: SpringerMATHGoogle Scholar
  33. Kass, R. E. and Wasserman, L. (1995). A reference Bayesian test for nested hypotheses and its relationship to the schwarz criterion. Journal of the American Statistical Association 90, 928–934MathSciNetMATHCrossRefGoogle Scholar
  34. Kinney, S. K. and Dunson, D. B. (2007). Fixed and random effects selection in linear and logistic models. Biometrics 63, 690–698MathSciNetMATHCrossRefGoogle Scholar
  35. Laird, N. and Ware, J. (1982). Random-effects models for longitudinal data. Biometrics 38, 963–974MATHCrossRefGoogle Scholar
  36. Liang, F., Paulo, R., Molina, G., Clyde, M. A. and Berger, J. O. (2005). Mixtures of g-priors for Bayesian variable selection. Technical Report 05-12, ISDS, Duke UniversityGoogle Scholar
  37. Lin, X. (1997). Variance component testing in generalised linear models with random effects. Biometrika 84, 309–326MathSciNetMATHCrossRefGoogle Scholar
  38. Liu, J. S. and Wu, Y. N. (1999). Parameter expansion for data augmentation. Journal of the American Statistical Association 94, 1264–1274MathSciNetMATHCrossRefGoogle Scholar
  39. Liu, C., Rubin, D. B. and Wu, Y. N. (1998). Parameter expansion to accelerate EM: the PX-EM algorithm. Biometrika 85, 755–770MathSciNetMATHCrossRefGoogle Scholar
  40. Meng, X. L. and Wong, W. H. (1996). Simulating ratios of normalizing constants via a simple identity: a theoretical exploration. Statistica Sinica 6, 831–860MathSciNetMATHGoogle Scholar
  41. Mira, A. and Tierney, L. (2002). Efficiency and convergence properties of slice samplers. Scandinavian Journal of Statistics 29, 1–12MathSciNetMATHCrossRefGoogle Scholar
  42. Mitchell, T. J. and Beauchamp, J. J. (1988). Bayesian variable selection in linear regression (with discussion). Journal of the American Statistical Association 83, 1023–1036MathSciNetMATHCrossRefGoogle Scholar
  43. Neal, R. M. (2000). Slice sampling. Technical report, Department of Statistics, University of TorontoGoogle Scholar
  44. Newton, M. and Raftery, A. E. (1994). Approximate Bayesian inference by the weighted likelihood bootstrap (with discussion). Journal of the Royal Statistical Society, Series B 56, 3–48MathSciNetMATHGoogle Scholar
  45. O'Brien, S. M. and Dunson, D. B. (2004). Bayesian multivariate logistic regression. Biometrics 60, 739–746MathSciNetMATHCrossRefGoogle Scholar
  46. Pauler, D. K., Wakefield, J. C. and Kass, R. E. (1999). Bayes factors and approximations for variance component models. Journal of the Americal Statistical Association 94,1242–1253MathSciNetMATHCrossRefGoogle Scholar
  47. Raftery, A. E. (1995). Bayesian model selection in social resarch. Sociological Methodology 25, 111–163CrossRefGoogle Scholar
  48. Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics 6, 461–464MathSciNetMATHCrossRefGoogle Scholar
  49. Sinharay, S. and Stern, H. S. (2001). Bayes factors for variance component testing in generalized linear mixed models. In Bayesian Methods with Applications to Science, Policy, and Official Statistics, pages 507–516Google Scholar
  50. Smith, M. and Kohn, R. (1996). Nonparametric regression using Bayesian variable selection. Journal of Econometrics 75, 317–343MATHCrossRefGoogle Scholar
  51. Stiratelli, R., Laird, N. M. and Ware, J. H. (1984). Random-effects model for several observations with binary response. Biometrics 40, 961–971CrossRefGoogle Scholar
  52. Verbeke, G. and Molenberghs, G. (2003). The use of score tests for inference on variance components. Biometrics 59, 254–262MathSciNetMATHCrossRefGoogle Scholar
  53. Vines, S., Gilks, W. and Wild, P. (1994). Fitting Bayesian multiple random effects models. Technical report, Biostatistics Unit, Medical Research Council, CambridgeGoogle Scholar
  54. West, M. (1987). On scale mixtures of normal distributions. Biometrika 74, 646–648MathSciNetMATHCrossRefGoogle Scholar
  55. Zellner, A. and Siow, A. (1980). Posterior odds ratios for selected regression hypotheses. In Bayesian Statistics: Proceedings of the First International Meeting held in Valencia (Spain) Google Scholar
  56. Zhao, Y., Staudenmayer, J., Coull, B. and Wand, M. (2006). General design Bayesian generalized linear mixed models. Statistical Science 21, 35–51MathSciNetMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.National Institute of Statistical SciencesUSA
  2. 2.Department of Statistical ScienceDuke UniversityDurhamUSA

Personalised recommendations