Robust On-Line Statistical Learning
We describe possible ways of endowing neural networks with statistically robust properties. We especially look at learning schemes resistant to outliers by defining error criteria able to handle deviations from convenient probability distribution assumptions. It comes out to be convenient to cast neural nets in state space representations and apply both Kalman Filter and Stochastic Approximation procedures in order to suggest statistically robustified solutions for on-line learning.
KeywordsPrediction Error Kalman Filter Influence Function State Space Representation Kalman Filter Algorithm
Unable to display preview. Download preview PDF.
- E. Capobianco: “A Unifying view of Stochastic Approximation, Kaiman Filter and Backpropagation”. NNSP-IEEE Proceedings, vol. V, pp 87–94, 1995.Google Scholar
- F.R. Hampel, E.M. Ronchetti, P.J. Rousseeuw and W.A. Stahel: Robust Statistics: the approach based on Infiuence Function. New York, US: Wiley, 1986.Google Scholar
- A.C. Harvey: Forecasting, Structural Time Series Models and the Kaiman Filter. Cambridge, UK: Cambridge University Press, 1989.Google Scholar
- D.E. Rumelhart, G.E. Hinton, R.J. Williams: “Learning internal representations by error propagation”, in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D.E. Rumelhart and J.L. McClelland, Eds. Cambridge, US: MIT Press, vol 1, pp 318–362, 1986.Google Scholar