Abstract
Hierarchical Bayesian inference in parameterised models offers an approach for controlling complexity. In this paper we utilise a novel prior for the leaning of a model's structure. We call the prior node relevance determination. It is applicable in a range of models including sigmoid belief networks and Boltzmann machines. We demonstrate how the approach may be applied to determine structure in a multi-layer perceptron.
This work was completed while author was at The Computer Laboratory, Cambridge University, Cambridge, U.K.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
N. D. Lawrence. Variational Inference in Probabilistic Models. PhD thesis, Computer Laboratory, University of Cambridge, New Museums Site, Pembroke Street, Cambridge, CB2 3QG, U.K., 2000. Available from http://www.thelawrences.net/neil./
N. D. Lawrence and M. Azzouzi. A variational Bayesian committee of neural networks. Available from http://www.thelawrences.net/neil
D. J. C. MacKay. Developments in probabilistic modelling with neural networks—ensemble learning. In Neural Networks: Artificial Intelligence and Industrial Applications. Proceedings of the 3rd Annual Symposium on Neural Networks, Nijmegen, Netherlands, 14–15 September 1995, pages 191–198, Berlin, 1995. Springer.
D. J. C. MacKay. Probable networks and plausible predictions — a review of practical Bayesian methods for supervised neural networks. Network: Computation in Neural Systems, 6(3):469–505, 1995.
R. M. Neal. Bayesian Learning for Neural Networks. Springer, 1996. Lecture Notes in Statistics 118.
H. H. Thodberg. A review of Bayesian neural networks with an application to near infrared spectroscopy. IEEE Transactions on Neural Networks, 7(l):56–72, 1996.
M. E. Tipping. The relevance vector machine. In S. A. Solla, T. K. Leen, and K.-R. Müller, editors, Advances in Neural Information Processing Systems, volume 12, pages 652–658, Cambridge, MA, 2000. MIT Press.
H. Tong. Non-linear Time Series: a Dynamical System Approach, volume 6 of Oxford Statistical Science Series. Clarendon Press, Oxford, 1995.
A. S. Weigend, B. A. Huberman, and D. E. Rumelhart. Predicting sunspots and exchange rates with connectionist networks. In S. Eubank and M. Casdagli, editors, Proceedings of the 1990 NATO Workshop on Nonlinear Modeling and Forcasting, Santa Fe, New Mexico. Addison-Wesley, 1990.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag London Limited
About this paper
Cite this paper
Lawrence, N.D. (2002). Note Relevance Determination. In: Tagliaferri, R., Marinaro, M. (eds) Neural Nets WIRN Vietri-01. Perspectives in Neural Computing. Springer, London. https://doi.org/10.1007/978-1-4471-0219-9_11
Download citation
DOI: https://doi.org/10.1007/978-1-4471-0219-9_11
Publisher Name: Springer, London
Print ISBN: 978-1-85233-505-2
Online ISBN: 978-1-4471-0219-9
eBook Packages: Springer Book Archive