Philosophical Dimensions of Logic and Science pp 117-137 | Cite as

# Akaike’s Theorem and Bayesian Methodology

## Abstract

Elementary statistics provides us with a simple and generally accepted answer to the question which element of a *statistical model* for the interconnection between a dependent variable and a number of independent variables should be viewed as the optimal one. Such a statistical model *M* is, basically, a disjunction of mutually exclusive hypotheses each of which gives a probability distribution for the value of the dependent variable when the values of the independent variables are known, and the standard procedure for choosing from these hypotheses the optimal one is to choose the hypothesis which has the maximal likelihood relative to the existing evidence. The question *how the statistical model itself should be chosen is* much more controversial, however. Statisticians have proposed a variety of *model selection criteria* for making choices between such models, but there is no general agreement concerning the question which of these should be used.

## Preview

Unable to display preview. Download preview PDF.

## References

- Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In Petrov, B. N. and Csdki, F., editors,
*2nd International Symposium on Information Theory*, pp. 267–281. Akadémiai Kiadó, Budapest.Google Scholar - Akaike, H. (1974). A new look at the statistical model identification.
*IEEE Transactions on Automatic Control*,**19**: 716–723.CrossRefGoogle Scholar - Atkinson, A. C. (1978). Posterior probabilities for choosing a regression model.
*Biometrika*,**65**: 39–48.CrossRefGoogle Scholar - Forster, M. and Sober, E. (1994). How to tell when simpler, more unified, or less ad hoc theories will provide more accurate predictions.
*The British Journal for the Philosophy of Science*,**45**: 1–35.CrossRefGoogle Scholar - Forster, M. (2001). The new science of simplicity. In Zellner, A., Keuzenkamp, H. and McAleer, M., editors,
*Simplicity*,*Inference and Econometric Modelling*, pp. 83–117. Cambridge University Press, Cambridge.Google Scholar - Kieseppä,, I. A. (1997). Akaike information criterion, curve-fitting, and the philosophical problem of simplicity.
*The British Journal for the Philosophy of Science*,**48**: 21–48.CrossRefGoogle Scholar - Kuha, J. (submitted). Simplicity and model fit: implications of a bayesian approach. Submitted for publication in
*Philosophy of Science*.Google Scholar - R,aftery, A. E. (1995). Bayesian model selection in social research. In Marsden, P. V., editor,
*Sociological Methodology*, pp. 111–163. Blackwell Publishers, Washington DC.Google Scholar - Sakamoto, Y., Ishiguro, M. and Kitagawa, G. (1986).
*Akaike Information Criterion Statistics*. KTK Scientific Publishers, Tokyo.Google Scholar - Schwarz, G. (1978). Estimating the dimension of a model.
*Annals of Statistics*,**6**: 461–464.CrossRefGoogle Scholar - Smith, A. F. M. and Spiegelhalter, D. J.
*(1980)*. Bayes factors and choice criteria for linear models.*Journal of the Royal Statistical Society*,Series B,**42***:213–220*.Google Scholar - Spiegelhalter, D. J. and Smith, A. F. M.
*(1982)*. Bayes factors for linear and log-linear models with vague prior information.*Journal of the Royal Statistical Society*,Series B,**44***:377–387*.Google Scholar - Teräsvirta, T. and Mellin, I.
*(1989)*. Linear model selection, criteria and tests. In Kotz, S., Johnson, N. L. and Read, C. B., editors,*Encyclopaedia of Statistical Sciences*. Supplement Volume, pp.*83–87*. John Wiley and Sons, New York.Google Scholar - Wetherill, G. B., et al.
*(1986). Regression Analysis with Applications*. Chapman and Hall, London.Google Scholar