Abstract
The Extended Kalman Filter (EKF) algorithm for identification of a state space model is shown to be a sensible tool in estimating a Logistic Regression Model sequentially. A Gaussian probability density over the parameters of the Logistic model is propagated on a sample by sample basis. Two other approaches, the Laplace Approximation and the Variational Approximation are compared with the state space formulation. Features of the latter approach, such as the possibility of inferring noise levels by maximising the ‘innovation probability’ are discussed. Experimental illustrations of these ideas on a synthetic and a real world problems are shown.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
De Freitas, G.F.J., Niranjan, M. & Gee, A.H. (1999), Hierarchical Kalman-Bayes models for Regularisation and Sequential Learning, Neural Computation (In Press); Preprint Available in http://www-svr.eng.cam.ac.uk/jfgf
Jaakkola, T.S. & Jordan, M.I. (1996), ‘A variational approach to Bayesian logistic regression models and their extensions’, Available from psyche.mit.edu
Jazwinski, A.H. (1970), Stochastic Processes and Filtering Theory, Vol. 64 in Mathematics in Science and Engineering, Academic Press.
Kadirkamanathan, V. & Niranjan, M. (1993), ‘A Function Estimation Approach to Sequential Learning with Neural Networks’, Neural Computation 5, pp. 954–975.
Lovell, D.R., Scott, M.J.J., Niranjan, M., Prager, R.W., Dalton, K.J. B & Derom, R. (1997), ‘On the use of expected attainable discrimination for feature selection in large scale medical risk prediction problems’, Report CUED/F-INFENG/TR. 299, Available from http://www-svr.eng.cam.ac.uk/projects/qamc.
Niranjan, M. (1997), ‘Sequential Tracking in Pricing Financial Options using Model Based and Neural Network Approaches’, In Mozer, M.C., Jordan, M.I. & Petsche, T., Ed., Advances in Neural Information Processing Systems 9, MIT Press, pp: 960–966.
Niranjan, M., Cox, I.J. & Hingorani, S. (1994), ‘Recursive estimation of formants in speech’, Proceedings of the International Conference on Acoustics Speech and Signal Processing, ICASSP 94, Adelaide.
Puskorius, G.V. & Feldkamp, L.A. (1994), ‘Neurocontrol of Nonlinear Dynamical Systems with Kalman Filter Trained Recurrent Networks’, IEEE Transactions on Neural Networks, 5(2), pp. 279–297.
Quinlan, R. (1987), ‘Simplifying decision trees’, Int J Man-Machine Studies 27, pp. 221–234.
Spiegelhalter, D. & Lauritzen, S. (1990), ‘Sequential updating of conditional probabilities on directed graphical structures’, Networks 20: 579–605.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1999 Springer-Verlag London Limited
About this paper
Cite this paper
Niranjan, M. (1999). On Sequential Bayesian Logistic Regression. In: Marinaro, M., Tagliaferri, R. (eds) Neural Nets WIRN Vietri-99. Perspectives in Neural Computing. Springer, London. https://doi.org/10.1007/978-1-4471-0877-1_1
Download citation
DOI: https://doi.org/10.1007/978-1-4471-0877-1_1
Publisher Name: Springer, London
Print ISBN: 978-1-4471-1226-6
Online ISBN: 978-1-4471-0877-1
eBook Packages: Springer Book Archive