# Point Estimation, Stochastic Approximation, and Robust Kalman Filtering

## Abstract

Significantly non-normal noise, and particularly the presence of outliers, severely degrades the performance of the Kalman Filter, resulting in poor state estimates, non-white residuals, and invalid inference. An approach to robustifying the Kalman Filter based on minimax theory is described. The relationship between the minimax robust estimator of location formulated by Huber, its recursive versions based on the stochastic approximation procedure of Robbins and Monro, and an approximate conditional mean filter derived via asymptotic expansion, is shown. Consistency and asymptotic normality results are given for the stochastic approximation recursion in the case of multivariate time-varying stochastic linear dynamic systems with no process noise. A first- order approximation is given for the conditional prior distribution of the state in the presence of ε-contaminated normal observation noise and normal process noise. This distribution is then used to derive a first- order approximation of the conditional mean estimator for the case where both observation and process noise are present.

## Keywords

Kalman Filter Fisher Information Robust Estimation Process Noise Stochastic Approximation## Preview

Unable to display preview. Download preview PDF.

## References

- [1]Blum, J.R. (1954) “Multidimensional Stochastic Approximation Methods,”
*Ann. Math. Stat.*, 25, 4, 737–744.CrossRefGoogle Scholar - [2]Englund, J.E., U. Holst, and D. Ruppert (1988) “Recursive M-estimators of Location and Scale for Dependent Sequences,”
*Scandinavian J. Statistics*, 15, 2, 147–159.Google Scholar - [3]Fabian, V. (1968) “On Asymptotic Normality in Stochastic Approximation,”
*Ann. Math. Stat.*, 39, 4, 1327–1332.CrossRefGoogle Scholar - [4]Goel, P.K. and M.H. DeGroot (1980) “Only Normal Distributions Have Linear Posterior Expectations in Linear Regression,”
*J.A.S.A.*, 75, 372, 895–900.Google Scholar - [5]Huber, P.J. (1964) “Robust Estimation of a Location Parameter,”
*Ann. Math. Stat.*, 35, 1, 73–101.CrossRefGoogle Scholar - [6]Huber, P.J. (1969)
*Théorie de l’Inférence Statistique Robuste*,Presses de l’Université de Montréal (Montréal).Google Scholar - [7]Huber, P.J. (1972) “The 1972 Wald Lecture. Robust Statistics: a Review,”
*Ann. Math. Stat.*, 43, 4, 1041–1067.CrossRefGoogle Scholar - [8]Huber, P.J. (1977)
*Robust Statistical Procedures*, Society for Industrial and Applied Mathematics ( Philadelphia, Pennsylvania).Google Scholar - [9]Huber, PJ. (1981)
*Robust Statistics*,John Wiley (New York).Google Scholar - [10]Kushner, HJ. and D.S. Clark (1978)
*Stochastic Approximation Methods for Constrained and Unconstrained Systems*,Springer-Verlag (Berlin and New York).Google Scholar - [11]Martin, R.D. (1972) “Robust Estimation of Signal Amplitude,”
*IEEE Trans. Information Theory*, IT-18, 5, 596–606.Google Scholar - [12]Martin, R.D. and CJ. Masreliez (1975) “Robust Estimation via Stochastic Approximation,”
*IEEE Trans. Information Theory*, IT-21, 3, 263–271.Google Scholar - [13]Masreliez, C.J. (1974) “Approximate Non-Gaussian Filtering with Linear State and Observation Relations,”
*Proc. Eighth Annual Princeton Conf. Information Sciences and Systems*, Dept. Electrical Engineering, Princeton University (Princeton, New Jersey), 398 (abstract only).Google Scholar - [14]Masreliez, C.J. (1975) “Approximate Non-Gaussian Filtering with Linear State and Observation Relations,”
*IEEE Trans. Automatic Control*, AC-20, 1, 107–110.Google Scholar - [15]Masreliez, C.J. and R.D. Martin (1974) “Robust Bayesian Estimation for the Linear Model and Robustizing the Kalman Filter,”
*Proc. Eighth Annual Princeton Conf. Information Sciences and Systems*, Dept. Electrical Engineering, Princeton University ( Princeton, New Jersey ), 488–492.Google Scholar - [16]Masreliez, C.J. and R.D. Martin (1977) “Robust Bayesian Estimation for the Linear Model and Robustifying the Kalman Filter,”
*IEEE Trans. Automatic Control*, AC-22, 3, 361–371.Google Scholar - [17]Moore, J.B. and B.D.O. Anderson (1980) “Coping with Singular Transition Matrices in Estimation and Control Stability Theory,”
*Int. J. Control*, 31, 3, 571–586.CrossRefGoogle Scholar - [18]Nevel’son, M.B. (1975) “On the Properties of the Recursive Estimates for a Functional of an Unknown Distribution Function,” in P. Révész (ed.),
*Limit Theorems of Probability Theory*(Colloq. Limit Theorems of Probability and Statistics, Keszthely ), North-Holland (Amsterdam and London), 227–251.Google Scholar - [19]Nevel’son, M.B. and Ri. Has’minskii (1973)
*Stochastic Approximation and Recursive Estimation*, American Mathematical Society ( Providence, Rhode Island).Google Scholar - [20]Price, E.L. and V.D. Vandelinde (1979) “Robust Estimation Using the Robbins-Monro Stochastic Approximation Algorithm,”
*IEEE Trans. Information Theory*, IT-25, 6, 698–704.Google Scholar - [21]Robbins, H. and S. Monro (1951) “A Stochastic Approximation Method,”
*Ann. Math. Stat.*, 22, 400–407.CrossRefGoogle Scholar - [22]Schick, I.C. (1989) “Robust Recursive Estimation of the State of a Discrete-Time Stochastic Linear Dynamic System in the Presence of Heavy-Tailed Observation Noise,” Ph.D. thesis, Department of Mathematics, Massachusetts Institute of Technology (Cambridge, Massachusetts). Reprinted as Report LIDS-TH-1975, Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, May 1990.Google Scholar
- [23]Schick, I.C. and S.K. Mitter (1991) “Robust Recursive Estimation in the Presence of Heavy-Tailed Observation Noise,” submitted to
*Ann. Stat*.Google Scholar - [24]Spall, J.C. and K.D. Wall (1984) “Asymptotic Distribution Theory for the Kalman Filter State Estimator,”
*Commun. Statist. Theor. Meth.*, 13, 16, 1981–2003.CrossRefGoogle Scholar - [25]Verdú, S. and H.V. Poor (1984) “On Minimax Robustness: a General Approach and Applications,”
*IEEE Trans. Automatic Control*, AC-30, 2, 328–340.Google Scholar - [26]Wasan, M.T. (1969)
*Stochastic Approximation*, Cambridge University Press ( Cambridge, U.K.).Google Scholar - [27]West, M. (1981) “Robust Sequential Approximate Bayesian Estimation,”
*J. Royal Statistical Society*, B, 43, 2, 157–166.Google Scholar - [28]Young, P. (1984)
*Recursive Estimation and Time-Series Analysis: an Introduction*,Springer-Verlag (Berlin and New York).Google Scholar