An Objective Bayesian Procedure for Variable Selection in Regression
- 2.3k Downloads
The Bayesian analysis of the variable selection problem in linear regression when using objective priors needs some form of encompassing the class of all submodels of the full linear model as they are nonnested models. After we provide a nested setting, objective intrinsic priors suitable for computing model posterior probabilities, on which the selection is based, can be derived.
The way of encompassing the models is not unique and there is no clear indications for the optimal way. Typically, the class of linear models are encompassed into the full model.
In this paper, we explore a new way of encompassing the class of linear models that consequently produces a new method for variable selection. This method seems to have some advantages with respect to the usual one.
Specific intrinsic priors and model posterior probabilities are provided along with some of their main properties. Comparisons are made with R 2 and adjusted R 2, along with other frequentist methods for variable selection as lasso. Some illustrations on simulated and real data are provided.
Keywords and phrases:Calibration curve determination coefficient g-priors intrinsic priors lasso criterion model selection normal linear model reference priors
Unable to display preview. Download preview PDF.
- 2.Berger, J. O., and Pericchi, L. R. (1996b). On the justification of default and intrinsic Bayes factor, In Modelling and Prediction (Eds., J. C. Lee, W. Johnson, and A. Zellner), pp. 276–293, Springer-Verlag, New York.Google Scholar
- 4.Casella, G., and Moreno, E. (2006). Objective Bayesian variable selection, Journal of the American Statistical Association (to appear).Google Scholar
- 8.Girón, F. J., Martínez, M. L., Moreno, E., and Torres, F. (2003). Bayesian analysis of matched pairs in the presence of covariates, In Bayesian Statistics 7 (Eds., J. M. Bernardo, M. J. Bayarri, A. P. Dawid, D. Heckerman, A. F. M. Smith, and M. West), pp. 553–563, Oxford University Press, OxfordGoogle Scholar
- 9.Girón, F. J., Martínez, M. L., Moreno, E., and Torres, F. (2006). Objective testing procedures in linear models: Calibration of the p-values, Scandinavian Journal of Statistics (to appear).Google Scholar
- 10.Girón, F. J., Martínez, M. L., Moreno, E., and Torres, F. (2006). Objective Bayesian procedures for variable selection in regression, (submitted).Google Scholar
- 17.Moreno, E., Girón, F. J., and Torres, F. (2003). Intrinsic priors for hypothesis testing in normal regression models. Rev. R. Acad. Cien. Serie A, Mat., 53–61.Google Scholar
- 20.Stamey, T., Kabakin, J., McNeal, J., Johnstone, I. Freiha, F., Redwine, E. and Yang, N. (1989). Prostate-specific antigen in the diagnosis and treated of adenocarcinoma of the prostate II: radical prostatectomy treated patients. Journal of Urology, 16, 1076–1083.Google Scholar
- 22.Zellner, A. (1986). On assessing prior distributions and Bayesian regression analysis with g-prior distributions, In Bayesian Inference and Decision Techniques: Essays in Honor of Bruno deFinetti (Eds., P. K. Goel and A. Zellner), North-Holland, Amsterdam.Google Scholar