Akaike’s Theorem and Bayesian Methodology
Elementary statistics provides us with a simple and generally accepted answer to the question which element of a statistical model for the interconnection between a dependent variable and a number of independent variables should be viewed as the optimal one. Such a statistical model M is, basically, a disjunction of mutually exclusive hypotheses each of which gives a probability distribution for the value of the dependent variable when the values of the independent variables are known, and the standard procedure for choosing from these hypotheses the optimal one is to choose the hypothesis which has the maximal likelihood relative to the existing evidence. The question how the statistical model itself should be chosen is much more controversial, however. Statisticians have proposed a variety of model selection criteria for making choices between such models, but there is no general agreement concerning the question which of these should be used.
Unable to display preview. Download preview PDF.
- Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In Petrov, B. N. and Csdki, F., editors, 2nd International Symposium on Information Theory, pp. 267–281. Akadémiai Kiadó, Budapest.Google Scholar
- Forster, M. (2001). The new science of simplicity. In Zellner, A., Keuzenkamp, H. and McAleer, M., editors, Simplicity, Inference and Econometric Modelling, pp. 83–117. Cambridge University Press, Cambridge.Google Scholar
- Kuha, J. (submitted). Simplicity and model fit: implications of a bayesian approach. Submitted for publication in Philosophy of Science.Google Scholar
- R,aftery, A. E. (1995). Bayesian model selection in social research. In Marsden, P. V., editor, Sociological Methodology, pp. 111–163. Blackwell Publishers, Washington DC.Google Scholar
- Sakamoto, Y., Ishiguro, M. and Kitagawa, G. (1986). Akaike Information Criterion Statistics. KTK Scientific Publishers, Tokyo.Google Scholar
- Smith, A. F. M. and Spiegelhalter, D. J. (1980). Bayes factors and choice criteria for linear models. Journal of the Royal Statistical Society,Series B, 42 :213–220.Google Scholar
- Spiegelhalter, D. J. and Smith, A. F. M. (1982). Bayes factors for linear and log-linear models with vague prior information. Journal of the Royal Statistical Society,Series B, 44 :377–387.Google Scholar
- Teräsvirta, T. and Mellin, I. (1989). Linear model selection, criteria and tests. In Kotz, S., Johnson, N. L. and Read, C. B., editors, Encyclopaedia of Statistical Sciences. Supplement Volume, pp. 83–87. John Wiley and Sons, New York.Google Scholar
- Wetherill, G. B., et al. (1986). Regression Analysis with Applications. Chapman and Hall, London.Google Scholar