The Bayesian Evidence Scheme for Model Selection
The approach of the previous chapter is extended to the derivation of the model evidence as a measure for model weighting and selection. The idea is to integrate out the hyperparameters in the likelihood term by Gaussian approximation, which requires the derivation of the Hessian at the mode. The resulting expression for the model evidence is found to be an intuitively plausible generalisation of the results obtained by MacKay for Gaussian homoscedastic noise on the target. The nature of the various Ockham factors included in the evidence is discussed. The chapter concludes with a critical evaluation of the numerical inaccuracies inherent in this scheme.
KeywordsModel Selection Gaussian Mixture Model Weight Space Regularisation Term Weight Group
Unable to display preview. Download preview PDF.
- 1.The given derivation is only intuitive reasoning; a proper mathematical treatment of this subject can be found in , Chapter 1. In fact, the prior of equation 2 (11.29) is not uninformative in the output weights a, the implications of which are discussed on page 176.Google Scholar
- 2.Strictly speaking, this equation only holds locally within intervals of length , In , and In . However, these details have no influence on the following analysis and will therefore not be further considered here.Google Scholar
- 3.Note that W (the number of weights) is denoted by m in MacKay’s work.Google Scholar
- 4.See also , chapter 10.Google Scholar