# An Efficient Model Averaging Procedure for Logistic Regression Models Using a Bayesian Estimator with Laplace Prior

## Abstract

Modern statistics has developed numerous methods for linear and nonlinear regression models, but the correct treatment of model uncertainty is still a difficult task. One approach is model selection, where, usually in a stepwise procedure, an optimal model is searched with respect to some (asymptotic) criterion such as AIC or BIC. A draw back of this approach is, that the reported post model selection estimates, especially for the standard errors of the parameter estimates, are too optimistic. A second approach is model averaging, either frequentist (FMA) or Bayesian (BMA). Here, not an optimal model is searched for, but all possible models are combined by some weighting procedure. Although conceptually easy, the approach has mainly one drawback: the number of potential models can be so large that it is infeasible to calculate the estimates for every possible model. In our paper we extend an idea of Magnus et al. (2009), called WALS, to the case of logistic regression. In principal, the method is not restricted to logistic regression but can be applied to any generalized linear model. In the final stage it uses a Bayesian esimator using a Laplace prior with a special hyperparameter.

## Keywords

Model Average Bayesian Estimator Bayesian Model Average Bayesian Model Average Minimax Regret## Preview

Unable to display preview. Download preview PDF.

## References

- Fahrmeir, L. & Tutz, G. (2001).
*Multivariate Statistical Modelling Based on Generalized Linear Models*, Springer; 2nd edition, New York.MATHGoogle Scholar - Hjort, N. & Claeskens, G. (2003). Frequentist model average estimators,
*Journal of the American Statistical Association***98**: 879–899.MATHCrossRefMathSciNetGoogle Scholar - Hoeting, J. A., D., M., Raftery, A. E. & Volinsky, C. (1999). Bayesian model averaging: a tutorial,
*Statistical Science***14**: 382–401.MATHCrossRefMathSciNetGoogle Scholar - Leeb, H. & Pötscher, B. M. (2003). The finite-sample distribution of post-model-selection estimators, and uniform versus non-uniform approximations,
*Econometric Theory***19**: 100–142.MATHCrossRefMathSciNetGoogle Scholar - Leeb, H. & Pötscher, B. M. (2005a). The distribution of a linear predictor after model selection: Conditional finite-sample distributions and asymptotic approximations,
*Journal of Statistical Planning and Inference***134**: 64–89.MATHCrossRefMathSciNetGoogle Scholar - Leeb, H. & Pötscher, B. M. (2005b). Model selection and inference: Facts and fiction,
*Econometric Theory***21**: 21–59.MATHCrossRefMathSciNetGoogle Scholar - Leeb, H. & Pötscher, B. M. (2006). Can one estimate the conditional distribution of post-model-selection estimators?,
*Annals of Statistics***34**: 2554–2591.MATHCrossRefMathSciNetGoogle Scholar - Leeb, H. & Pötscher, B. M. (2008). Can one estimate the unconditional distribution of post-model-selection estimators?,
*Econometric Theory***24**: 338–376.MATHMathSciNetGoogle Scholar - Magnus, J. R., Powell, O. & Prüfer, P. (2009). A comparison of two model averaging techniques with an application to growth empirics,
*Journal of Econometrics, to appear*.Google Scholar - Raftery, A. E., Madigan, D. & Hoeting, J. A. (1997). Bayesian model averaging for linear regression models,
*Journal of the American Statistical Association***92**: 179–191.MATHCrossRefMathSciNetGoogle Scholar