Bayes/Orthodox Comparisons

Abstract

Our purpose in this chapter is to compare empirically the performance of Bayes and orthodox approaches to regression and related problems. Specifically, I treat the problem of identifying the degree of a polynomial and the order of a Markov chain. When we raise the degree of a polynomial or the order of a Markov chain, we improve the model’s accuracy at the cost of some simplicity. According to the Bayesian analysis of Chapter 5, there is a well-defined rate of exchange that must be exceeded for the average likelihood (or support) of the model to increase. The exact average likelihoods are computable in both cases. It is worth stressing that the Bayesian approach to such problems is unified: one compares average likelihoods. By contrast, orthodox statistics offers us a mixed bag of tricks with no single (or simple) underlying logic.