Advertisement

An Objective Bayesian Procedure for Variable Selection in Regression

  • F. Javier Girón
  • Elías Moreno
  • M. Lina Martínez
Chapter
Part of the Statistics for Industry and Technology book series (SIT)

Abstract

The Bayesian analysis of the variable selection problem in linear regression when using objective priors needs some form of encompassing the class of all submodels of the full linear model as they are nonnested models. After we provide a nested setting, objective intrinsic priors suitable for computing model posterior probabilities, on which the selection is based, can be derived.

The way of encompassing the models is not unique and there is no clear indications for the optimal way. Typically, the class of linear models are encompassed into the full model.

In this paper, we explore a new way of encompassing the class of linear models that consequently produces a new method for variable selection. This method seems to have some advantages with respect to the usual one.

Specific intrinsic priors and model posterior probabilities are provided along with some of their main properties. Comparisons are made with R 2 and adjusted R 2, along with other frequentist methods for variable selection as lasso. Some illustrations on simulated and real data are provided.

Keywords and phrases:

Calibration curve determination coefficient g-priors intrinsic priors lasso criterion model selection normal linear model reference priors 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Berger, J. O., and Pericchi, L. R. (1996a). The intrinsic Bayes factor for model selection and prediction, Journal of American Statistical Association, 91, 109–122.zbMATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Berger, J. O., and Pericchi, L. R. (1996b). On the justification of default and intrinsic Bayes factor, In Modelling and Prediction (Eds., J. C. Lee, W. Johnson, and A. Zellner), pp. 276–293, Springer-Verlag, New York.Google Scholar
  3. 3.
    Casella, G., and Moreno, E. (2005). Intrinsic meta analysis of contingency tables, Statistics in Medicine, 24, 583–604.CrossRefMathSciNetGoogle Scholar
  4. 4.
    Casella, G., and Moreno, E. (2006). Objective Bayesian variable selection, Journal of the American Statistical Association (to appear).Google Scholar
  5. 5.
    Denison, D. G. T., Holmes, C. C., Mallick, B. K., and Smith, A. F. M. (2002). Bayesian Methods for Nonlinear Classification and Regression, John Wiley_& Sons, Chichester.zbMATHGoogle Scholar
  6. 6.
    Furnival, G. M., and Wilson, R. W. (1974). Regression by leaps and bounds, Technometrics, 16, 499–511.zbMATHCrossRefGoogle Scholar
  7. 7.
    Giróon, F. J., Martínez, M. L., and Moreno, E. (2003). Bayesian analysis of matched pairs, Journal of Statistical Planning and Inference, 113, 49–66.CrossRefMathSciNetGoogle Scholar
  8. 8.
    Girón, F. J., Martínez, M. L., Moreno, E., and Torres, F. (2003). Bayesian analysis of matched pairs in the presence of covariates, In Bayesian Statistics 7 (Eds., J. M. Bernardo, M. J. Bayarri, A. P. Dawid, D. Heckerman, A. F. M. Smith, and M. West), pp. 553–563, Oxford University Press, OxfordGoogle Scholar
  9. 9.
    Girón, F. J., Martínez, M. L., Moreno, E., and Torres, F. (2006). Objective testing procedures in linear models: Calibration of the p-values, Scandinavian Journal of Statistics (to appear).Google Scholar
  10. 10.
    Girón, F. J., Martínez, M. L., Moreno, E., and Torres, F. (2006). Objective Bayesian procedures for variable selection in regression, (submitted).Google Scholar
  11. 11.
    Hastie, T., Tibshirani, R., and Friedman, J. (2001). The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer-Verlag, New York.zbMATHGoogle Scholar
  12. 12.
    Mallows, C. L. (1973). Some comments on C p, Technometrics, 15, 661–675.zbMATHCrossRefGoogle Scholar
  13. 13.
    Miller, A. (2002). Subset Selection in Regression, 2nd ed., Chapman and Hall/CRC, Boca Raton, FL.zbMATHGoogle Scholar
  14. 14.
    Moreno, E., Bertolino, F., and Racugno W. (1998). An intrinsic limiting procedure for model selection and hypothesis testing. Journal of the American Statistical Association, 93, 1451–1460.zbMATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Moreno, E., Bertolino, F., and Racugno, W. (2003). Bayesian inference under partial prior information, Scandinavian Journal of Statistics, 30, 565–580.zbMATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Moreno, E., Bertolino, F., and Racugno, W. (2000). Default Bayesian analysis of the Behrens-Fisher problems, Journal of Statistical Planning and Inference, 81, 323–333.CrossRefMathSciNetGoogle Scholar
  17. 17.
    Moreno, E., Girón, F. J., and Torres, F. (2003). Intrinsic priors for hypothesis testing in normal regression models. Rev. R. Acad. Cien. Serie A, Mat., 53–61.Google Scholar
  18. 18.
    Moreno, E., and Liseo, B. (2003). A default Bayesian test for the number of components in a mixture, Journal of Statistical Planning and Inference, 111, 129–142.zbMATHCrossRefMathSciNetGoogle Scholar
  19. 19.
    Moreno, E., Torres, F., and Casella, G. (2005). Testing equality of regression coefficients in heteroscedastic normal regression models, Journal of Statistical Planning and Inference, 131, 117–134.zbMATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Stamey, T., Kabakin, J., McNeal, J., Johnstone, I. Freiha, F., Redwine, E. and Yang, N. (1989). Prostate-specific antigen in the diagnosis and treated of adenocarcinoma of the prostate II: radical prostatectomy treated patients. Journal of Urology, 16, 1076–1083.Google Scholar
  21. 21.
    Tibshirani, R. (1996). Regression shrinkage and the selection via lasso, Journal of the Royal Statistical Society, Series B, 58, 267–288.zbMATHMathSciNetGoogle Scholar
  22. 22.
    Zellner, A. (1986). On assessing prior distributions and Bayesian regression analysis with g-prior distributions, In Bayesian Inference and Decision Techniques: Essays in Honor of Bruno deFinetti (Eds., P. K. Goel and A. Zellner), North-Holland, Amsterdam.Google Scholar

Copyright information

© Birkhäuser Boston 2006

Authors and Affiliations

  • F. Javier Girón
    • 1
  • Elías Moreno
    • 2
  • M. Lina Martínez
    • 1
  1. 1.Universidad de MálagaMálagaSpain
  2. 2.Universidad de GranadaGranadaSpain

Personalised recommendations