Advertisement

Appendix: Derivation of the Hessian for the Bayesian Evidence Scheme

  • Dirk Husmeier
Part of the Perspectives in Neural Computing book series (PERSPECT.NEURAL)

Abstract

This chapter provides a derivation of the Hessian of the error function E, which is required for the Bayesian evidence scheme of chapters 10 and 11. The derivation is based on an extended version of the EM algorithm, which allows the full Hessian to be decomposed into three additive components. The derivation of the first term, the Hessian of the EM error function U, is straightforward. The second term, the outer product of the gradient of the EM error function, is found to be cancelled out. An approximation is made for the third term, the expectation value for the outer product of the gradient of Ψ, which is approximated by a diagonal block matrix. The justification for this simplification is given in the text.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Reference

  1. 1.
    This approximation is not always good, but recall from chapters 10 and 11 that the off-diagonal terms Hαi,wk and Hβiwk are not needed for the evidence schemeGoogle Scholar
  2. 2.
    For alternatives schemes, that make use of the whole Hessian (e.g. [26]), approximation (18.45) can be applied.Google Scholar

Copyright information

© Springer-Verlag London Limited 1999

Authors and Affiliations

  • Dirk Husmeier
    • 1
  1. 1.Neural Systems Group, Department of Electrical & Electronic EngineeringImperial CollegeLondonUK

Personalised recommendations