A simple Bayesian regularisation scheme
A regularisation scheme is derived from a simple Bayesian approach, where the maximum likelihood estimate of the network parameters is replaced by the mode of their posterior distribution. Conjugate priors for the various network parameters are introduced, which give rise to regularisation terms that can be viewed as a generalisation of simple weight decay. It is shown how the posterior mode can be found with a slightly modified version of the EM algorithm.
Unable to display preview. Download preview PDF.