Point Estimation, Stochastic Approximation, and Robust Kalman Filtering

  • Sanjoy K. Mitter
  • Irvin C. Schick
Part of the Progress in Systems and Control Theory book series (PSCT, volume 12)


Significantly non-normal noise, and particularly the presence of outliers, severely degrades the performance of the Kalman Filter, resulting in poor state estimates, non-white residuals, and invalid inference. An approach to robustifying the Kalman Filter based on minimax theory is described. The relationship between the minimax robust estimator of location formulated by Huber, its recursive versions based on the stochastic approximation procedure of Robbins and Monro, and an approximate conditional mean filter derived via asymptotic expansion, is shown. Consistency and asymptotic normality results are given for the stochastic approximation recursion in the case of multivariate time-varying stochastic linear dynamic systems with no process noise. A first- order approximation is given for the conditional prior distribution of the state in the presence of ε-contaminated normal observation noise and normal process noise. This distribution is then used to derive a first- order approximation of the conditional mean estimator for the case where both observation and process noise are present.


Kalman Filter Fisher Information Robust Estimation Process Noise Stochastic Approximation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Blum, J.R. (1954) “Multidimensional Stochastic Approximation Methods,” Ann. Math. Stat., 25, 4, 737–744.CrossRefGoogle Scholar
  2. [2]
    Englund, J.E., U. Holst, and D. Ruppert (1988) “Recursive M-estimators of Location and Scale for Dependent Sequences,” Scandinavian J. Statistics, 15, 2, 147–159.Google Scholar
  3. [3]
    Fabian, V. (1968) “On Asymptotic Normality in Stochastic Approximation,” Ann. Math. Stat., 39, 4, 1327–1332.CrossRefGoogle Scholar
  4. [4]
    Goel, P.K. and M.H. DeGroot (1980) “Only Normal Distributions Have Linear Posterior Expectations in Linear Regression,” J.A.S.A., 75, 372, 895–900.Google Scholar
  5. [5]
    Huber, P.J. (1964) “Robust Estimation of a Location Parameter,” Ann. Math. Stat., 35, 1, 73–101.CrossRefGoogle Scholar
  6. [6]
    Huber, P.J. (1969) Théorie de l’Inférence Statistique Robuste,Presses de l’Université de Montréal (Montréal).Google Scholar
  7. [7]
    Huber, P.J. (1972) “The 1972 Wald Lecture. Robust Statistics: a Review,” Ann. Math. Stat., 43, 4, 1041–1067.CrossRefGoogle Scholar
  8. [8]
    Huber, P.J. (1977) Robust Statistical Procedures, Society for Industrial and Applied Mathematics ( Philadelphia, Pennsylvania).Google Scholar
  9. [9]
    Huber, PJ. (1981) Robust Statistics,John Wiley (New York).Google Scholar
  10. [10]
    Kushner, HJ. and D.S. Clark (1978) Stochastic Approximation Methods for Constrained and Unconstrained Systems,Springer-Verlag (Berlin and New York).Google Scholar
  11. [11]
    Martin, R.D. (1972) “Robust Estimation of Signal Amplitude,” IEEE Trans. Information Theory, IT-18, 5, 596–606.Google Scholar
  12. [12]
    Martin, R.D. and CJ. Masreliez (1975) “Robust Estimation via Stochastic Approximation,” IEEE Trans. Information Theory, IT-21, 3, 263–271.Google Scholar
  13. [13]
    Masreliez, C.J. (1974) “Approximate Non-Gaussian Filtering with Linear State and Observation Relations,” Proc. Eighth Annual Princeton Conf. Information Sciences and Systems, Dept. Electrical Engineering, Princeton University (Princeton, New Jersey), 398 (abstract only).Google Scholar
  14. [14]
    Masreliez, C.J. (1975) “Approximate Non-Gaussian Filtering with Linear State and Observation Relations,” IEEE Trans. Automatic Control, AC-20, 1, 107–110.Google Scholar
  15. [15]
    Masreliez, C.J. and R.D. Martin (1974) “Robust Bayesian Estimation for the Linear Model and Robustizing the Kalman Filter,” Proc. Eighth Annual Princeton Conf. Information Sciences and Systems, Dept. Electrical Engineering, Princeton University ( Princeton, New Jersey ), 488–492.Google Scholar
  16. [16]
    Masreliez, C.J. and R.D. Martin (1977) “Robust Bayesian Estimation for the Linear Model and Robustifying the Kalman Filter,” IEEE Trans. Automatic Control, AC-22, 3, 361–371.Google Scholar
  17. [17]
    Moore, J.B. and B.D.O. Anderson (1980) “Coping with Singular Transition Matrices in Estimation and Control Stability Theory,” Int. J. Control, 31, 3, 571–586.CrossRefGoogle Scholar
  18. [18]
    Nevel’son, M.B. (1975) “On the Properties of the Recursive Estimates for a Functional of an Unknown Distribution Function,” in P. Révész (ed.), Limit Theorems of Probability Theory (Colloq. Limit Theorems of Probability and Statistics, Keszthely ), North-Holland (Amsterdam and London), 227–251.Google Scholar
  19. [19]
    Nevel’son, M.B. and Ri. Has’minskii (1973) Stochastic Approximation and Recursive Estimation, American Mathematical Society ( Providence, Rhode Island).Google Scholar
  20. [20]
    Price, E.L. and V.D. Vandelinde (1979) “Robust Estimation Using the Robbins-Monro Stochastic Approximation Algorithm,” IEEE Trans. Information Theory, IT-25, 6, 698–704.Google Scholar
  21. [21]
    Robbins, H. and S. Monro (1951) “A Stochastic Approximation Method,” Ann. Math. Stat., 22, 400–407.CrossRefGoogle Scholar
  22. [22]
    Schick, I.C. (1989) “Robust Recursive Estimation of the State of a Discrete-Time Stochastic Linear Dynamic System in the Presence of Heavy-Tailed Observation Noise,” Ph.D. thesis, Department of Mathematics, Massachusetts Institute of Technology (Cambridge, Massachusetts). Reprinted as Report LIDS-TH-1975, Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, May 1990.Google Scholar
  23. [23]
    Schick, I.C. and S.K. Mitter (1991) “Robust Recursive Estimation in the Presence of Heavy-Tailed Observation Noise,” submitted to Ann. Stat.Google Scholar
  24. [24]
    Spall, J.C. and K.D. Wall (1984) “Asymptotic Distribution Theory for the Kalman Filter State Estimator,” Commun. Statist. Theor. Meth., 13, 16, 1981–2003.CrossRefGoogle Scholar
  25. [25]
    Verdú, S. and H.V. Poor (1984) “On Minimax Robustness: a General Approach and Applications,” IEEE Trans. Automatic Control, AC-30, 2, 328–340.Google Scholar
  26. [26]
    Wasan, M.T. (1969) Stochastic Approximation, Cambridge University Press ( Cambridge, U.K.).Google Scholar
  27. [27]
    West, M. (1981) “Robust Sequential Approximate Bayesian Estimation,” J. Royal Statistical Society, B, 43, 2, 157–166.Google Scholar
  28. [28]
    Young, P. (1984) Recursive Estimation and Time-Series Analysis: an Introduction,Springer-Verlag (Berlin and New York).Google Scholar

Copyright information

© Springer Science+Business Media New York 1992

Authors and Affiliations

  • Sanjoy K. Mitter
    • 1
  • Irvin C. Schick
    • 2
  1. 1.Department of Electrical Engineering and Computer Science, and Laboratory for Information and Decision SystemsMassachusetts Institute of TechnologyCambridgeUSA
  2. 2.Network Analysis Department, BBN Communications DivisionBolt Beranek and Newman, Inc.CambridgeUSA

Personalised recommendations