An Efficient Model Averaging Procedure for Logistic Regression Models Using a Bayesian Estimator with Laplace Prior

  • Christian Heumann
  • Moritz Grenke


Modern statistics has developed numerous methods for linear and nonlinear regression models, but the correct treatment of model uncertainty is still a difficult task. One approach is model selection, where, usually in a stepwise procedure, an optimal model is searched with respect to some (asymptotic) criterion such as AIC or BIC. A draw back of this approach is, that the reported post model selection estimates, especially for the standard errors of the parameter estimates, are too optimistic. A second approach is model averaging, either frequentist (FMA) or Bayesian (BMA). Here, not an optimal model is searched for, but all possible models are combined by some weighting procedure. Although conceptually easy, the approach has mainly one drawback: the number of potential models can be so large that it is infeasible to calculate the estimates for every possible model. In our paper we extend an idea of Magnus et al. (2009), called WALS, to the case of logistic regression. In principal, the method is not restricted to logistic regression but can be applied to any generalized linear model. In the final stage it uses a Bayesian esimator using a Laplace prior with a special hyperparameter.


Model Average Bayesian Estimator Bayesian Model Average Bayesian Model Average Minimax Regret 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Fahrmeir, L. & Tutz, G. (2001). Multivariate Statistical Modelling Based on Generalized Linear Models, Springer; 2nd edition, New York.MATHGoogle Scholar
  2. Hjort, N. & Claeskens, G. (2003). Frequentist model average estimators, Journal of the American Statistical Association 98: 879–899.MATHCrossRefMathSciNetGoogle Scholar
  3. Hoeting, J. A., D., M., Raftery, A. E. & Volinsky, C. (1999). Bayesian model averaging: a tutorial, Statistical Science 14: 382–401.MATHCrossRefMathSciNetGoogle Scholar
  4. Leeb, H. & Pötscher, B. M. (2003). The finite-sample distribution of post-model-selection estimators, and uniform versus non-uniform approximations, Econometric Theory 19: 100–142.MATHCrossRefMathSciNetGoogle Scholar
  5. Leeb, H. & Pötscher, B. M. (2005a). The distribution of a linear predictor after model selection: Conditional finite-sample distributions and asymptotic approximations, Journal of Statistical Planning and Inference 134: 64–89.MATHCrossRefMathSciNetGoogle Scholar
  6. Leeb, H. & Pötscher, B. M. (2005b). Model selection and inference: Facts and fiction, Econometric Theory 21: 21–59.MATHCrossRefMathSciNetGoogle Scholar
  7. Leeb, H. & Pötscher, B. M. (2006). Can one estimate the conditional distribution of post-model-selection estimators?, Annals of Statistics 34: 2554–2591.MATHCrossRefMathSciNetGoogle Scholar
  8. Leeb, H. & Pötscher, B. M. (2008). Can one estimate the unconditional distribution of post-model-selection estimators?, Econometric Theory 24: 338–376.MATHMathSciNetGoogle Scholar
  9. Magnus, J. R., Powell, O. & Prüfer, P. (2009). A comparison of two model averaging techniques with an application to growth empirics, Journal of Econometrics, to appear .Google Scholar
  10. Raftery, A. E., Madigan, D. & Hoeting, J. A. (1997). Bayesian model averaging for linear regression models, Journal of the American Statistical Association 92: 179–191.MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  1. 1.Institut für StatistikLudwig-Maximilians-Universität MünchenMünchenGermany

Personalised recommendations