Orthogonalizations and Prior Distributions for Orthogonalized Model Mixing
Prediction methods based on mixing over a set of plausible models can help alleviate the sensitivity of inference and decisions to modeling assumptions. One important application area is prediction in linear models. Computing techniques for model mixing in linear models include Markov chain Monte Carlo methods as well as importance sampling. Clyde, DeSimone and Parmigiani (1996) developed an importance sampling strategy based on expressing the space of predictors in terms of an orthogonal basis. This leads both to a better identified problem and to simple approximations to the posterior model probabilities. Such approximations can be used to construct efficient importance samplers. For brevity, we call this strategy orthogonalized model mixing.
Two key elements of orthogonalized model mixing are: a) the orthogonalization method and b) the prior probability distributions assigned to the models and the coefficients. In this paper we consider in further detail the specification of these two elements. In particular, after identifying the aspects of these specifications that are essential to the success of the importance sampler, we list and briefly discuss a number of different alternatives for both a) and b). We highlight the features that may make each one of the options attractive in specific situations and we illustrate some important points via a simulated data set.
KeywordsPartial Little Square Prior Distribution Model Space Importance Sampling Principal Component Regression
Unable to display preview. Download preview PDF.
- Chipman, H (1996). Bayesian Variable Selection with Related Predictors. Canadian Journal of Statisticsto appearGoogle Scholar
- Clyde, MA, DeSimone, H, and Parmigiani, G (1996). Prediction via Orthogonalized Model Mixing. Journal of the American Statistical Association, forthcomingGoogle Scholar
- Foster, D, George, E and McCulloch, R (1995) Calibrating Bayesian variable selection proceduresGoogle Scholar
- Garthwaite, PH and Dickey, JM (1992). Elicitation of prior distributions for variable selection problems in regression. Annals of Statistics20, pp. 1697–1719Google Scholar
- Geisser, S, Predictive inference. An introduction, Chapman & Hall, New York, 1993Google Scholar
- George, EI and McCulloch, R (1994). Fast Bayes Variable Selection. TR, Graduate School of Business, University of ChicagoGoogle Scholar
- George, E.I. and Oman, S.D. (1993). Multiple shrinkage principal component regression. TR, University of Texas at AustinGoogle Scholar
- Geweke, JF (1994). Bayesian comparison of econometric models. Working Paper 532, Federal Reserve Bank of MinneapolisGoogle Scholar
- Green, PJ (1995). Reversible Jump MCMC Computation and Bayesian Model determination. TR-94-19, University of Bristol.Google Scholar
- Hoeting, J, Raftery, AE, Madigan, DM (1995), A Method for Simultaneous Variable Selection and Outlier Identification in Linear Regression, Technical Report 95–02, Colorado State UniversityGoogle Scholar
- Laud, P and Ibrahim, JG (1994). Predictive Specification of Prior Model Probability in Variable Selection. TR, Division of Statistics, University of Northern IllinoisGoogle Scholar
- Phillips, D.B. and Smith, A.F.M. (1994). Bayesian Model Comparison via Jump Diffusions. TR-94-20, Department of Mathematics, Imperial College, UKGoogle Scholar
- Raftery, AE, Madigan, DM, and Hoeting, J (1993). Model selection and accounting for model uncertainty in linear regression models. TR 262, Department of Statistics, University of WashingtonGoogle Scholar
- Raftery, A.E., Madigan, D.M., and Volinski C.T. (1995). Accounting for Model Uncertainty in Survival Analysis Improves Predictive Performance (with discussion). In Bayesian Statistics5, ed. J.M. Bernardo, J.O. Berger, A.P. Dawid and Smith, A.F.MGoogle Scholar
- Weisberg, S (1985). Applied Linear Regression. 2nd Edition. Wiley, New York.Google Scholar
- Zellner, A., (1986). On assessing prior distribution in Bayesian regression analysis with g-prior distributions. In “Bayesian Inference and decision Techniques: Essays in Honor of Bruno de Finetti”, 233–243, ( P.K. Goel and A. Zellner eds.). North Holland, AmsterdamGoogle Scholar