Advertisement

Lectures on Wiener and Kalman Filtering

  • Thomas Kailath
Part of the International Centre for Mechanical Sciences book series (CISM, volume 140)

Abstract

Suppose we have two random variables X, Y with a known joint density function fx,y(.,.). Assume that in a particular experiment, the random variable Y can be measured and takes the value y. What can be said about the corresponding value, say x, of the unobservable variable X?

Keywords

Covariance Function Riccati Equation Wiener Filter Optimum Filter Innovation Approach 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    V.S. Pugachev, “The Determination of an Optimal System by Some Arbitrary Criterion”, Automation and Remote Control, vol. 19, pp. 513–532, June 1958. See also R.B. Street, IEEE Trans. on Automatic Control, vol. AC-8, pp. 375–376, October 1963. Also M. Zalcai, IEEE Trans. on Information Theory, vol. IT-10, pp. 95–96, January 1964.Google Scholar
  2. [2]
    T. Kailath, “The Innovations Approach to Detection and Estimation Theory”, Proc. IEEE, vol. 58, pp. 680–695, May 1970.MathSciNetGoogle Scholar
  3. [3]
    T. Kailath, “A View of Three Decades of Linear Filtering Theory”, IEEE Transactions on Information Theory, vol. IT-20, no. 2, pp. 145–181, March 1974.MathSciNetGoogle Scholar
  4. [4]
    T. Kailath, ed., “Linear Least-Squares Estimation”, Benchmark Papers in Electrical Engineering, vol. 17. Academic Press, New York, 1977.Google Scholar
  5. [1]
    N. Wiener, Extrapolation, Interpolation and Smoothing of Stationary Time Series, with Engineering Applications. New York: Technology Press and Wiley, 1949, (Originally issued in February 1942, as a classified Nat. Defense Res. Council Rep).Google Scholar
  6. [2]
    M.C. Yovits and J.L. Jackson, “Linear filter optimization with game theory considerations”, in IRE Nat. Cony. Rec., pt. 4, pp. 193–199, 1955.Google Scholar
  7. [3]
    N. Levinson, “A heuristic exposition of Wiener’s mathematical theory of prediction and filtering”, J. Math. Phys., vol. 15, pp. 110–119, July 1947; reprinted as an Appendix in [i].Google Scholar
  8. [4]
    M.G. Krein, “Integral equations on a half -axis with kernel depending on the difference of the arguments”, Usp. Math. Nauk., vol. 13, pp. 3–120, 1958, Amer. Math. Socy. TransLGoogle Scholar
  9. [5]
    H.W. Bode and C.E. Shannon, “A simplified derivation of linear least squares smoothing and prediction theory”, Proc. IRE, vol. 38, pp. 417–425, April 1950.MathSciNetGoogle Scholar
  10. [6]
    L.A. Zadeh and J.R. Ragazzini, “An extension of Wiener’s theory of prediction”, J. Appl. Phys., vol. 21, pp. 645–655, July 1950.MathSciNetADSGoogle Scholar
  11. [7]
    T. Kailath, Linear Systems, New Jersey: Prentice-Hall, 1980.MATHGoogle Scholar
  12. [8]
    T. Kailath, “A view of three decades of linear filtering theory”, IEEE Trans. on Information Theory, vol. IT-20, No. 2, pp. 145–181, March 1974.Google Scholar
  13. [1]
    F. Smithies, Integral Equations, New York: Cambridge University Press, 1962.Google Scholar
  14. [2]
    J.A. Cochran, Analysis of Linear Integral Equations, New York: M Jraw-Hill, 1972.MATHGoogle Scholar
  15. [3]
    H. Laning and R. Battin, Random Processes in Automatic Control, New York; McGraw-Hill, 1958.Google Scholar
  16. [4]
    H.L. Van Trees, Detection, Estimation, and Modulation Theory, Part I, New York: Wiley, 1968.MATHGoogle Scholar
  17. [5]
    P. Whittle, Prediction and Regulation, New York: Van Nostrand Reinhold, 1963.Google Scholar
  18. [6]
    M. Shinbrot, “A generalization of a method for the solution of the integral equation arising in optimization of time-varying linear systems with nonstationary inputs”, IRE Trans. Inform. Theory, vol. IT-3, pp. 220–225, Dec. 1957.Google Scholar
  19. [7]
    J. Bendat, Principles and Applications of Random Noise Theory, New York: Wiley, 1958.Google Scholar
  20. [8]
    E.L. Peterson, Statistical Analysis and Optimization of Systems, New York: Wiley, 1961.MATHGoogle Scholar
  21. [9]
    P. Swerling, “First-order error propagation in a stagewise smoothing procedure for satellite observations”, J. Astronaut. Sci., vol. 6, pp. 46–52, Autumn 1959; see also “A proposed stagewise differential correction procedure for satellite tracking and prediction”, RAND Corp. Rep. p. 1292, Jan. 1958.Google Scholar
  22. [10]
    R.E. Kalman, “A new approach to linear filtering and prediction problems”, J. Basic Eng., vol. 82, pp. 34–45, March 1960.Google Scholar
  23. [11]
    RE. Kalman and R.S. Bucy, “New results in linear filtering and prediction theory”, Trans. ASME, Ser. D, J. Basic Eng., vol. 83, pp. 95–107, Dec. 1961.MathSciNetGoogle Scholar
  24. [12]
    R.L. Stratonovich, “Application of the theory of Markoff processes in optimal signal detection”, Radio Eng. Electron. Phys., vol. 1, pp. 1–19, 1960 (Translation from Russian).Google Scholar
  25. [13]
    T. Kailath, “A View of Three Decades of Linear Filtering Theory’ IEEE Trans. Inform. Theory, vol. IT-20, pp. 146–181, 1974.Google Scholar
  26. [14]
    J. Rissanen and T. Kailath, “Partial realization of stochastic processes”, Automatica, vol. 8, pp. 380–386, July 1972.Google Scholar
  27. [15]
    M. Gevers and T. Kailath, “An innovations approach to least-squares estimation — Part VI: Discrete-time innovations representations and recursive estimation”, IEEE Trans. Automat Contr., vol. AC-18, pp. 588–600, Dec. 1973.Google Scholar
  28. [16]
    T. Kailath and P. Frost, “Mathematical modeling for stochastic processes” in Stochastic Problems in Control”, (Proc. Symp. AACC), Amer. Soc. Mech. Engrs., 1968.Google Scholar
  29. [1]
    T. Kailath, “An Innovations Approach to Least-Squares Estimation, Part I: Linear Filtering in Additive White Noise”, IEEE Trans. Automatic Control, vol. AC-13, pp. 646–655, December 1968.Google Scholar
  30. [2]
    M. Gevers and T. Kailath, “An Innovations Approach to Least-Squares Estimation, Part VI: Discrete-Time Innovations Representations and Recursive Estimation”, IEEE Trans. Automatic Control, vol. AC-18, pp. 588–600, December 1973.Google Scholar
  31. [3]
    R.E. Kalman, “A New Approach to Linear Filtering and Prediction Problems”, J. Basic Eng., vol.. 82, pp. 34–45, March 1960.Google Scholar
  32. [4]
    L. Ljung, T. Kailath and B. Friedlander, “Scattering Theory and Linear Least-Squares Estimation, Part I: Continuous-Time Problems”, Proc. IEEE, vol. 64, No. 1, pp. 131–139, January 1976.MathSciNetGoogle Scholar
  33. [5]
    B. Friedlander, T. Kailath and L. Ljung, “Scattering Theory and Linear Least Squares Estimation, Part II: Discrete-Time Problems”, J. of The Franklin Institute, vol. 301, Nos. 1 & 2, pp. 71–82, January-February 1976.Google Scholar
  34. [6]
    B. Friedlander, G. Verghese and T. Kailath, “Scattering Theory and Linear Least-Squares Estimation”,Part III: The Estimates, Proceedings 1977 IEEE Decision and Control Conference, New Orleans, La, Dec. 1977. Also IEEE Trans. Automat. Control, vol. AC-25, pp. 794–802, Aug. 1980.Google Scholar
  35. [7]
    M. Morf and T. Kailath, “Square-Root Algorithms for Least-Squares Estimation”, IEEE Trans. Automatic Cont ol, vol. AC-20, No. 4, pp. 487–497, August 1975.MathSciNetGoogle Scholar
  36. [8]
    G. Bierman, Factorization Methods for Discrete Sequential Estimation, New York Academic Press, 1977.Google Scholar
  37. [9]
    N. Levinson, “The ’ Wiener rms (root-mean-square) Error Criterion in Filter Design and Prediction”, J. Math. Phys., vol. 25, pp. 261–278, January 1947.MathSciNetGoogle Scholar
  38. [10]
    E.A. Robinson, Multichannel Time-Series Analysis with Digital Computer Programs, San Francisco, Ca.: Holden-Day 1967.MATHGoogle Scholar
  39. [11]
    B. Friedlander, M. Morf, T. Kailath and L. Ljung, “New Inversion Formulas for Matrices Classified in Terms of Their Distance From Toeplitz Matrices”, J. Lin. Alg. and Appins., vol. 27, pp. 31–60, Oct. 1979.MathSciNetMATHGoogle Scholar
  40. [12]
    B. Friedlander, T. Kailath, M. Morf and L. Ljung, “Levinson — and Chandrasekhar — type Equations for a General Discrete-Time Linear Estimation Problem”, Proceedings 1976 IEEE Decision and Control Conference, Florida, December 1976. Also IEEE Trans. Automat. Control, pp. 653–659, Aug. 1978.Google Scholar
  41. [13]
    B. Dickinson, M. Morf and T. Kailath, “Canonical Matrix Fraction and State Space Description for Deterministic and Stochastic Linear Systems”, IEEE Trans. on Automatic Control, Special Issue on System Identification and Time-Series Analysis, vol. AC-19, No. 6, pp. 656–667, December 1974.MathSciNetGoogle Scholar
  42. [14]
    P. Swerling, Uncertain Dynamic Systems, Englewood Cliffs: Prentice-Hall, 1973.Google Scholar
  43. [15]
    F. Schweppe, “Modern State Estimation Methods from the Viewpoint of the Method of Least Squares”, IEEE Trans. Automatic Control, Vol. AC-16, pp. 707–720, December 1971.Google Scholar
  44. R.E. Kalman, “New Methods of Wiener Filtering Theory”, in Proc. 1st Symp. Engineering Applications of Random Function Theory and Probability, J.L. Bogdanoff and F. Kozin, Eds., New York: Wiley, 1963, pp. 270–388; also, RIAS, Baltimore, Md., Tech. Rep. 61–1, 1961.Google Scholar
  45. [la]
    R.E. Kalman and R.S. Bucy, “New Results in Linear Filtering and Prediction Theory”, Trans. ASME, Ser. D, J. Basic Eng., vol. 83, pp. 95–107, December 1961.MathSciNetGoogle Scholar
  46. [2]
    T. Kailath, “An Innovations Approach to Least-Squares Estimation, Pt. I: Linear Filtering in Additive White Noise”, IEEE Trans. on Automatic Control, vol. AC-13, pp. 646–655, December 1968.Google Scholar
  47. [3]
    J. Lawson and G.E. Uhlenbeck, Threshold Signals, MIT Radiation Laboratory Series, vol. 24, McGraw-Hill Book Co., New York, 1948.Google Scholar
  48. [4]
    K.J. Åstrom, Introduction to Stochastic Control Theory, Academic Press, New York, 1970.Google Scholar
  49. [5]
    T. Kailath, “A General Likelihood-Ratio Formula for Random Signals in Gaussian Noise,” IEEE Trans. on Information Theory, vol. IT-15, pp. 350–361, May 1969.Google Scholar
  50. [6]
    T. Kailath, “ A Note on Least-Squares Estimation by the Innovations Method,” J. SIAM Control, vol. 10 pp. 477–486, August 1972.MathSciNetMATHGoogle Scholar
  51. [7]
    T. Kailath, “Some Extensions of the Innovations Theorem,” Bell Syst. Tech. J., vol. 50, pp. 1487–1494, April 1971.MATHGoogle Scholar
  52. [8]
    P.A. Meyer, “Sur un problème de filtration”, Séminaire de probabilités, Pt. VII, Lecture Notes in Mathematics, vol. 321, Springer, New York, pp. 223–247, 1973.Google Scholar
  53. [9]
    A. Segall, “Stochastic processes in estimation theory”, IEEE Trans. Inform. Theory, vol. IT-22, pp. 275–286, 1976.Google Scholar
  54. [10]
    T. Kailath and P. Frost, “An Innovations Approach to Least-Squares Estimation, Pt. II: Linear Smoothing in Additive White Noise,” IEEE Trans. on Automatic Control, vol. AC-13, pp. 655–660, December 1968.Google Scholar
  55. [11]
    S. Fujishige, “Minimum-Variance Estimation for a Linear Continuous-Discrete System with Noisy State-Integral Observation”, IEEE Trans. on Automatic Control, vol. AC-20, pp. 139–140, February 1975.Google Scholar
  56. [12]
    J. Mishra and V.S. Rajamani, “Least-Squares State Estimation in Time-Delay Systems with Colored Observation Noise: An Innovations Approach”, IEEE Trans. on Automatic Control, vol. AC-20, pp. 140–142, February 1975.Google Scholar
  57. [13]
    R. Redheffer, “On the Relation of Transmission-Line Theory to Scattering and Transfer”, J. Math. Phys., vol. 41, pp. 1–41, 1962.MathSciNetMATHGoogle Scholar
  58. [14]
    R. Redheffer, Difference Equations and Functional Equations in Transmission-Line Theory”, Ch. 12 in Modern Mathematics for the Engineer, 2nd series (E.F. Beckenbach, ed.), McGraw-Hill Book Co., New York, 1961.Google Scholar
  59. [15]
    G. Verghese, B. Friedlander and T. Kailath, “Scattering Theory and-Linear Least-Squares Estimation” Part III: The Estimates, Proceedings 1977 IEEE Decision and Control Conference, New Orleans, Dec. 1977. Also IEEE Trans. Automat. Contr., vol. AC-25, pp. 794–802, Aug. 1980.Google Scholar
  60. [16]
    L. Ljung, T. Kailath and B. Friedlander, “Scattering Theory and Linear Squares Estimation” Part I: Continuous-Time Problems, Proc. IEEE, vol. 64, pp. 131–139, January 1976.MathSciNetGoogle Scholar
  61. [17]
    B. Friedlander, T. Kailath and L. Ljung, “Scattering Theory and Linear Least Squares Estimation” Part II: Discrete Time Problems, Journal of The Franklin Institute, vol. 301, Nos. 1&2, pp. 71–82, January-February 1976.Google Scholar
  62. [18]
    L. Ljung and T. Kailath, “A Unified Approach to Smoothing Formulas”, Automatica, vol. 12, pp. 147–157, March 1976.MathSciNetMATHGoogle Scholar
  63. [19]
    B.C. Lévy, Castanon, G.C. Verghese and A.S. Villsky, A Scattering Framework for Decentralized Estimation Problems, Proc. 1981 IEEE Decision and Control Conferetce, San Diego, Ca, Dec. 1981.Google Scholar
  64. [20]
    M.H.A. Davis, Linear Estimation and Stochastic Control, Halsted Press-Wiley, New York, 1977.MATHGoogle Scholar
  65. [1]
    A.J. Viterbi, Principles of Coherent Communication, McGraw-Hill Book Co., New York.Google Scholar
  66. [2]
    H.L. Van Trees, Detection, Estimation, and Modulation Theory, Pt. I, J. Wiley & Sons, New York, 1968. Detection, Estimation and Modulation Theory, Pt. II: Nonlinear Modulation Theory, J. Wiley & Sons, New York 1971.Google Scholar
  67. [3]
    T. Kailath and L. Ljung, “Asymptotic Behaviour of Constant-Coefficient Riccati Differential Equations”, IEEE Trans. on Automatic Control, vol. AC-21, pp. 385–388, 1976.Google Scholar
  68. [4]
    T. Kailath, Linear Systems, Prentice-Hall, New Jersey, 1980.MATHGoogle Scholar
  69. [5]
    V.M. Popov, “Hyperstability and Optimality of Automatic Systems with Several Control Functions”, Rev. Rolm. Sci. Tech., vol. 9, pp. 629–690, 1964.Google Scholar
  70. [6]
    R.E. Kalman, “When is a Linear Control System Optimal ?”, Trans. ASME, Ser. D., J. Basic Eng., vol. 86, pp. 51–60, June 1964.Google Scholar
  71. [7]
    T. Kailath, “A View of Three Decades of Linear Filtering Theory”, IEEE Transactions on Information Theory, vol. IT-20, pp. 145–181, March 1974.Google Scholar
  72. [8]
    B.D.O. Anderson and S. Vongpanitlerd, Network Analysis and Synthesis — A Modern Systems Theory Approach, Prentice-Hall, Inc., New Jersey, 1973.Google Scholar
  73. [9]
    V.M. Popov, Hyperstability of Control Systems, Springer-Verlag, New York, 1973. ( Original Rumanian edition, Bucharest, 1966 ).Google Scholar
  74. [10]
    P. Van Dooren, “A generalized eigenvalue approach for solving Riccati equations”, SIAM J. Scient. and Statist. Computing, vol. 2, pp. 121–135, June 1981.MATHGoogle Scholar
  75. [1]
    R.E. Kalman, “Linear Stochastic Filtering-Reappraisal and Outlook Proc. Symp. System Theory, (J. Fox, ed.), Polytechnic Institute of Brooklyn Press, Brooklyn, New York, 1965.Google Scholar
  76. [2]
    P. Faurre, “Representation of Stochastic Processes”, Ph.D. Dissertation, Dept. of Electrical Engineering, Stanford University, Stanford, Ca., 1967.Google Scholar
  77. [3]
    P. Faurre, “Identification par minimisation d’une représentation Markovienne de processus aléatoire”, Lecture Notes in Mathematics, vol. 132, pp. 85–106, Springer-Verlag, New York, 1970.Google Scholar
  78. [4]
    T. Kailath, and R. Geesey, “An Innovations Approach to Least-Squares Estimation, Pt. IV: Recursive Estimation Given Lumped Covariance Functions”, IEEE Trans. on Automatic Control, vol. AC-16, pp. 720–727, December 1971.Google Scholar
  79. [5]
    T. Kailath, “A View of Three Decades of Linear Filtering Theory”, IEEE Trans. on Information Theory, vol. IT-20, pp. 145–181, March 1974.Google Scholar
  80. [6]
    T. Kailath, “Fredholm Resolvents, Wiener-Hopf Equations and Riccati Differential Equations”, IEEE Trans. on Information Theory, vol. IT-15, pp. 665–672, November 1969.Google Scholar
  81. [7]
    B.D.O. Anderson and T. Kailath, “The Choice of Signal Process Models in Kalman-Bucy Filtering”, J. Math. Anal. Appl., vol. 35, pp. 659–668, September 1971.MathSciNetMATHGoogle Scholar
  82. [8]
    T. Kailath, R. Geesey and H. Weinert, “Some Relations Between RKHS Norms, and Fredholm Equations, ands Innovations Representations”, IEEE Trans. on Information Theory, vol. IT-18, pp. 341–348, May 1972.Google Scholar
  83. [9]
    A. Schumitzky, “On the Equivalence Between Matrix Riccati Equations and Fredholm Resolvents”, J. Comp. Syst. Sei., vol. 2, pp. 76–87, June 1968.MathSciNetMATHGoogle Scholar
  84. [l0a]
    T. Kailath and R. Geesey, “An Innovations Approach to Least-Squares Estimation, Pt. V: Innovations Representations and Recursive Estimation in Colored Noise”, IEEE Trans. on Automatic Control, vol. AC-18, pp. 435–453, October 1973.Google Scholar
  85. [10b]
    J.B. Moore and B.D.O. Anderson, “Spectral Factorization of Time-Varying Covariance Functions: The Singular Case”, Mathematical System Theory, vol. 4, pp. 10–23, 1970.MathSciNetMATHGoogle Scholar
  86. [11]
    B.D.O. Anderson and J.B. Moore, “The Kalman-Bucy Filter as a True Time-Varying Wiener Filter”, IEEE Trans. Syst., Man, Cybernetics, vol. SMC-1, pp. 119–128, 1971.Google Scholar
  87. [12a]
    M.G. Krein, “On Integral Equations Leading to Differential Equations of Second Order”, Dokl. Akad. Nauk SSSR, 97, pp. 21–24, 1954.MathSciNetGoogle Scholar
  88. [12b]
    T. Kailath, “Application of a Resolvent Identity to a Linear Smoothing Problem”, SIAM J. Contr., vol. 7, pp. 68–74, 1969.MathSciNetMATHGoogle Scholar
  89. [13]
    V.V. Sobolev, A Treatise on Radiative Transfer, D. Van Nostrand, Princeton, N.J., 1963, (Russian edition 1956 ).Google Scholar
  90. [1]
    J.L. Casti, RE. Kalaba and V.K. Murthy, “ A new initial-value method for on-line filtering and estimation,” IEEE Trans. Inform. Theory, vol. IT-18, pp. 515–518, July 1972.Google Scholar
  91. [2]
    S. Chandrasekhar, “On the radiative equilibrium of a stellar atmosphere, Pt XXI,” Astrophys. J., vol. 106, pp. 152–216, 1947; Pt. XXII, ibid, vol. 107, pp. 48–72, 1948.Google Scholar
  92. [3]
    V.A. Ambartsumian, “Diffuse reflection of light by a foggy medium”, Dokl. Akad. Sci. SSSR, vol. 38, pp. 229–322, 1943.Google Scholar
  93. [4]
    V.V. Sobolev, A Treatise on Radiative Transfer, New York: Van Nostrand, 1963 (translated from Russian original, 1958 ).Google Scholar
  94. [5]
    T. Kailath, “Some Chandrasekhar-type algorithms for quadratic regulator problems”, in Proc. IEEE Conf. Decision and Control and 11th Symp. Adaptive Processes, IDec. 1972, pp. 219–223. Also Technical Report, Computing and Control Dept., Imperial College, London, Aug. 1972.Google Scholar
  95. [6]
    T. Kailath, “Some new algorithms for recursive estimation in constant linear systems”, IEEE Trans. Inform. Theory, vol. IT-19, pp. 750–760, Nov. 1973.Google Scholar
  96. [7]
    T. Kailath, M. Morf and G.S. Sidhu, “Some new algorithms for recursive estimation in constant discrete-time linear systems”, in Proc. 7th Princeton Symp. Information and System Science, 1973. See also IEEE Trans. Automat. Contro., vol. AC-19, pp. 315–323, Aug. 1974.Google Scholar
  97. [8]
    B. Dickinson, M. Morf and T. Kailath, “Canonical matrix fraction and state space description for deterministic and stochastic linear systems”, IEEE Trans. on Automatic Control, Special Issue on System Identification and Time-Series Analysis, vol. AC-19, No. 6, pp. 656–667, Dec. 1974.MathSciNetGoogle Scholar
  98. [9]
    M. Morf and T. Kailath, “Square-root algorithms for linear least-squares estimation and control”, IEEE Trans. Auto. Cont., vol. AC-20, pp. 487–497, Aug. 1975.Google Scholar
  99. [10]
    L.Ljung and T. Kailath, “A scattering theory framework for fast least-squares algorithms”, in Multivariate Analysis IV, ed by P.R. Krishnaiah, North-Holland, Amsterdam, 1977. [Presented at Symposium, June 1975 ].Google Scholar
  100. [11]
    L. Ljung, T. Kailath and B. Friedlander, “Scattering Theory and Linear Least-Squares Estimation, Part I — Continuous-time Problems, Proceedings of the IEEE, vol. 64, pp. 131–139, Jan. 1976.MathSciNetGoogle Scholar
  101. [12]
    B. Friedlander, T. Kailath and L. Ljung, “Scattering Theory and Linear Least-squares Estimation”, Part II — Discrete-time Problems, J. Framblin Inst., vol. 301, pp. 71–82, Feb. 1976.MathSciNetMATHGoogle Scholar
  102. [13]
    G. Verghese, B. Friedlander and T. Kailath, “Scattering Theory and Linear Least-squares Estimation”, Part III — The Estimates, IEEE Trans. Automat. Contr., vol. AC-25, pp. 794–802, Aug. 1980.Google Scholar
  103. [14]
    T. Kailath and L. Ljung, “Efficient Change of Initial Conditions, Dual Chandrasekhar Equations, and Some Applications”, IEEE Trans. Autom. Contr., vol. AC-22, pp. 443–447, June 1977.Google Scholar
  104. [15]
    T. Kailath, A. Vieira and M. Morf, “Orthogonal Transformation, Square-root Implementations of Fast Estimation Algorithms”, Int. Symp. on Systems Optim. and Analysis, pp. 81–91, Lecture Notes in Control and Inform. Sciences, vol. 14, New York: Springer-Verlag, 1979.Google Scholar
  105. [16]
    T. Kailath, B. Lévy, L. Ljung and M. Morf, “The factorization and Representation of Operators in the Algebra Generated by Toeplity Operators”, SIAM J. Appl. Math., vol. 37, pp. 467–484, Dec. 1979.MATHGoogle Scholar
  106. [17]
    R.S. Bucy and P.D. Joseph, Filtering for Stochastic Processes with Applications to Guidance. New York: Wiley, 1968.MATHGoogle Scholar
  107. [18]
    R. Bellman, R. Kalaba and J. Wing, “Invariant imbedding and mathematical physics-I: Particle processes”, J. Math. Phys., vol. 1, pp. 280–308, 1960.MathSciNetMATHADSGoogle Scholar
  108. [19]
    G.C. Stokes, Collected Papers, vol. 2, pp. 91–93, Cambridge Univ. Press, 1883.Google Scholar
  109. [20]
    M. Morf, B. Lévy and T. Kailath, “Square-root Algorithms for the Continuous-time Least-squares Estimation Problem”, IEEE Trans. Automat. Contr., vol. AC-23, pp. 907–911, Oct. 1978.Google Scholar
  110. [1]
    T. Kailath, “A View of Three Decades of Linear Filtering Theory”, IEEE Trans. on Inform. Thy., IT-20, no. 2, pp. 145–181, March 1974.MathSciNetGoogle Scholar
  111. [2]
    D. Jacobson, “Totally Singular Quadratic Minimization”, IEEE Trans. on Autom., Contr., AC-16, pp. 651–658, 1971.Google Scholar
  112. [3]
    N. Krasner, “Separable Kernels in Linear Estimation and Control”, Ph.D. Dissertation, Dept. of Elec. Engg., Stanford Univ., Stanford, CA, May 1974.Google Scholar
  113. [4]
    D.J. Clements and B.D.O. Anderson, Singular Optimal Control: The Linear-Quadratic Problem”, Lecture Notes in Control, vol. 5, Springer-Verlag, New York, 1978.Google Scholar
  114. [5]
    L. Silverman, “Discrete Riccati Equations: Alternative Algorithms, Asymptotic Properties and System Theory Interpretations”, in Advances in Control, vol. 12, C.T. Leondes, ed., pp. 313–3386, Academic Press, New York, 1976.Google Scholar
  115. [6]
    B.P. Molinari, “A Strong Controllability and Observability in Linear Multivariable Control”, IEEE Trans. on Autom. Contr., AC-21, pp. 761–764, October 1976.Google Scholar
  116. [7]
    G. Verghese, “Infinite-Frequency Behaviour in Generalized Dynamic Systems”, Ph.D. Dissertation, Dept. of Elec. Engg., Stanford Univ., Stanford, CA, December 1978.Google Scholar
  117. [8]
    A. Emami Naeini, “Application of the Generalized Eigenstructure Problem to Multivariable Systems and the Robust Servomechanism for a Plant which Contains an Implicit Internal Model”, Ph.D. Dissertation, Dept. of Elec., En._., Stanford Univ., Stanford, CA, May 1981.Google Scholar
  118. [9]
    W.L. Chan, “Variational Dualities in the Linear Regulator and Estimation Problems”, J. Inst. Math. Appl., 18, pp. 237–248, October 1976.MathSciNetMATHGoogle Scholar
  119. U.B. Desai and H.L. Weinert, “Generalized Control-Estimation Duality and Inverse Projections”, Proc. 1979 Conf. on Information Sciences and Systems, pp. 115–120, John Hopkins Univ., Maryland, March 1979.Google Scholar
  120. [11]
    A. Lindquist, “Optimal Control of Linear Stochastic Systems with Applications to Time Lag Systems”, Information Sci., 5, pp. 81–126, 1973.Google Scholar
  121. [12]
    M.H.A. Davis Linear Estimation and Stochastic Control, Halsted Press, New York, 1977.Google Scholar
  122. [13]
    T. Kailath, “A General Likelihood Ratio Formula for Random Signals in Noise”, IEEE Trans. on Inform. Thy., IT-15, no. 3, pp. 350–361, May 1969.Google Scholar
  123. [14]
    T. Kailath, Likelihood Ratios for Gaussian Processes, IEEE Trans. on Inform. Thy., IT-16, no. 3, pp. 276–288, May 1970.MathSciNetGoogle Scholar
  124. [15]
    M.H.A. Davis and E. Andreadakis, “Exact and Approximate Filtering in Signal Detection: An Example”, IEEE Trans. on Inform. Thy., IT-23, pp. 768–772, Nov. 1977.Google Scholar
  125. [16]
    T. Kailath, B. Levy, L. Ljung and M. Morf, “Fast Time-Invariant Implementations of Gaussian Signal Detectors”, IEEE Trans. Inform. Thy., IT-24, no. 4, pp. 469–477, July 1978.MathSciNetGoogle Scholar
  126. [17]
    B. Levy, T. Kailath, L. Ljung and M. Morf, “Fast Time-Invariant Implementations for Linear Least-Squares Smoothing Filters”, IEEE Trans. Autom. Contr., 24, no. 5, pp. 770–774, Oct. 1979.MathSciNetMATHGoogle Scholar
  127. [18]
    T. Kailath, A. Vieira and M. Morf, “Inverses of Toeplitz Operators, Innovations, and Orthogonal Polynomials”, SIAM Rev., 20, no. 1, pp. 106–119, Jan. 1978.MathSciNetMATHGoogle Scholar
  128. [19]
    B. Friedlander, M. Morf, T. Kailath and L. Ljung, “New Inversion Formulas for Matrices Classified in Terms of their Distance from Toeplitz Matrices”, Linear Algebra and Its Appls., pp. 31–60, Oct. 1979.Google Scholar
  129. [20]
    M. Morf, “Doubling Algorithms for Toeplitz and Related Equations”, Proc. Int.’I. Conf. on Acoustics, Speech and Signal Proc., pp. 954–959, Denver, Co., 1980.Google Scholar
  130. [21]
    J.M. Delosme, “Fast Algorithms for Finite Shift-Rank Processes”, Ph.D. Dissertation, Stanford Univ., Dept. of Elec. Engg., Stanford, CA, Dec. 1981.Google Scholar
  131. [22]
    H. Lev-Ari, “Modeling and Parametrization of Nonstationary Stochastic Processes”, Ph.D. Dissertation, Stanford Univ., Dept. of Elec., Engg., Stanford, CA, 1982.Google Scholar
  132. [23]
    B. Porat, “Contributions to the Theory and Applications of Lattice Filters”, Ph.D. Dissertation, Stanford Univ., Dept. of Elec. Eny:., Stanford, CA, Dec. 1982.Google Scholar
  133. [24]
    T. Kailath, “Equations of Wiener-Hopf Type in Filtering Theory and Related Applications”, in Norbert Wiener: Collected Works, vol. III, edited by P.R. Masani MIT Press, 1982.Google Scholar
  134. [25]
    C.W. Helstrom, “Solution of the Detection Integral Equation for Stationary Filtered White Noise”, IEEE Trans. on Inform. Thy., IT-11, pp. 335–339, 1965.Google Scholar
  135. [26]
    T. Kailath, “Fredholm Resolvents, Wiener-Hopf Equations and Riccati Differential’ Equations”, IEEE Trans. on Inform. Thy., IT-15, no. 6, pp. 655–672, November 1969.Google Scholar
  136. [27]
    B. Anderson and T. Kailath, “Some Integral Equations with Nonsymmetric Separable Kernels”, SIAM J. Appl. Math., 20, no. 4, pp. 659–669, June 1971.MathSciNetGoogle Scholar
  137. [28]
    T. Kailath, R. Geesey and H. Weinert, “Some Relations Between RKHS Norms, Innovation Representations and Fredholm Equations”, IEEE Trans. on Inform. Thy., IT-18, no. 3, pp. 341–348, May 1972.MathSciNetGoogle Scholar
  138. [29]
    T. Kailath, L. Ljung and M. Morf, “Generalized Krein-Levinson Equations for Efficient Calculation of Fredholm Resolvents of Nondisplacement Kernels”, in Topics in Functional Analysis, pp. 169–184, I.C. Gohberg and M. Kac, eds., Academic Press, New York, 1978.Google Scholar
  139. [30]
    A.J. Berkhout, “Stability and Least-Squares Estimation”, Automatica, 11, pp. 633–638, 1975.Google Scholar
  140. [31]
    J.C. Willems, “Dissipative Dynamical Systems I: General Theory; II: Linear Systems with Quadratic Supply Rates”, Archive for Rational Mechanics and Analysis, 45, pp. 321–343, 1972.MathSciNetMATHADSGoogle Scholar
  141. [321.
    A. Vieira and T. Kailath, “On Another Approach to the Schur-Cohn Criterion”, IEEE Trans. Circuits and Systems, 24, pp. 218–220, April 1977.MathSciNetMATHGoogle Scholar
  142. [33]
    F.J. Dyson, “Old and New Approaches to the Inverse Scattering Problem”, in Studies in Mathematical Physics, Essays in Honor of Valentine Bargmann, pp. 151–167, E.H. Lieb, B. Simon and A.S. Wightman, eds., Princeton Univ. Press, 1976.Google Scholar
  143. [34]
    B.D. Anderson and T. Kailath, “Fast Algorithms for the Integral Equations of the Inverse Scattering Problems”, Integral Eqs. and Operator Thy., 1, no. 1, pp. 132–136, 1978.MathSciNetMATHGoogle Scholar
  144. [35]
    K.M. Case, “Inverse Scattering, Orthogonal Polynomials and Linear-Estimation”, in Topics in Functional Analysis, pp. 25–43, I. Gohberg and M. Kac, eds., Academic Press, New York, 1978.Google Scholar
  145. H. Dym, “Applications of Factorization Theory to the Inverse Spectral Problem”, Proc. Intl. Symp. on Math. Thy. of Networks and Systems, pp. 188–193, Delft, Holland, July 1979. Published by Western Periodicals Co., North Hollywood, CA.Google Scholar
  146. [37]
    P. Dewilde, J. Fokkema and I. Widya, “Inverse Scattering and Linear Prediction: The Continuous-Time Case”, in Stochastic Systems, M. Hazewinkel and J.C. Willems, eds., Reidel, 1981.Google Scholar
  147. [38]
    H. Akaike, “Stochastic Theory of Minimal Realization”, IEEE Trans. Autom. Contr., AC-19, pp. 667–674, Dec. 1974.Google Scholar
  148. [39]
    H. Akaike, “Markovian Representation of Stochastic Processes by Canonical Variables”, SIAM J. Contr., 13, pp. 162–173, 1975.MathSciNetMATHGoogle Scholar
  149. [40]
    B.D.O. Anderson and T. Kailath, “The Choice of Signal Process Models in Kalman-Bucy Filtering”, J. Math. Anal. and Appls., 35, pp. 659–668, Sept. 1971.MathSciNetMATHGoogle Scholar
  150. [41]
    A. Lindquist and G. Picci, “On the Stochastic Realization Problem”, SIAM J. Contr. Optim., 17, pp. 365–389, 1979.MathSciNetMATHGoogle Scholar
  151. [42]
    G. Ruckebusch, “On the Theory of Markovian Representation”, in Lecture Notes in Mathematics, 695, pp. 77–88, Springer-Verlag, New York, 1978.Google Scholar
  152. [43]
    T. Kailath and L. Ljung, “Strict-Sense State-Space Realizations of Nonstationary Gaussian Processes”, to appear.Google Scholar
  153. [44]
    B.D.O. Anderson and S. Vongpanitlerd, Network Analysis and Synthesis, A Modern Systems Theory Approach, Prentice-Hall, New Jersey, 1973.Google Scholar
  154. [45]
    P. Dewilde, A. Vieira and T. Kailath, “On a Generalized Szego-Levinson Realization Algorithm for Optimal Linear Prediction Based on a Network Synthesis Approach”, IEEE Trans. Circuits and Systems, CAS-25, pp. 663–675, Sept. 1978.Google Scholar
  155. [46]
    B.D.O. Anderson and T. Kailath, “Forwards and Backwards Models for Finite-State Markov Processes”, Adv. Appl. Probl., 11, pp. 118–133, 1979.MathSciNetMATHGoogle Scholar
  156. [47]
    P. Dewilde and H. Dym, “Schur Recursions, Error Formulas and Convergence of Rational Estimators for Stationary Stochastic Sequences”, IEEE Trans. Inform. Thy., IT-pp. 446–461, 1981.Google Scholar
  157. [48]
    Y. Genin and P. Van Dooren, “On E -Lossless Transfer Functions and Related Questions”, Philips Res. Lab. Tech. Rept. R447, Brussels, Belgium, Nov. 1980.Google Scholar
  158. [49]
    D.Z. Arov, “Passive Linear Stationary Dynamic Systems”, Siberian Mat. J., 20, pp. 149–162, 1979.MATHGoogle Scholar
  159. [50]
    G. Kimeldorf and G. Wahba, “Spline Functions and Stochastic Processes”, Sankhya, 132, pp. 173–180, 1970.MathSciNetGoogle Scholar
  160. [51]
    H.L. Weinert and T. Kailath, “Stochastic Interpretations and Recursive Algorithms for Spline Functions”, Annals of Stat., 2, pp. 787–794, 1974.MathSciNetMATHGoogle Scholar
  161. [52]
    H.L. Weinert and G.S. Sidhu, “A Stochastic Framework for Recursive Computation of Spline Functions, Pt. I, Interpolating Splines”, IEEE Trans. Inform. Thy, IT-24, pp. 45–50, Jan. 1978.Google Scholar
  162. [53]
    R.J.P. de Figueireido and A. Caprihan, “An Algorithm for the Generalized Smoothing Spline with Application to System Identification”, Proc. 1977 Conf. Inform. Sciences and Systems, John Hopkins Univ., Baltimore, MD, April 1977.Google Scholar
  163. [54]
    E. Parzen, “Multiple Time-Series Modeling”, in Multivariate Analysis — II, P.R. Krishnaiah, ed., pp. 389–409, Academic Press, New York, 1969.Google Scholar
  164. 55.
    J.P. Burg, “Maximum Entropy Spectral Analysis”, Ph.D. Dissertation, Dept. of Geophysics, Stanford Univ., Stanford, CA, 1975.Google Scholar
  165. [56]
    M. Morf, A. Vieira, D. Lee and T. Kailath, “Recursive Multichannel Maximum Entropy Spectral Estimation”, IEEE Trans. Geoscience Elect., GE-16, no. 2, pp. 85–94, April 1978.MathSciNetADSGoogle Scholar
  166. [57]
    D.G. Childers, ed., Modern Spectrum Analysis, IEEE Press, New York, 1978.Google Scholar
  167. [58]
    S. Haykin, ed., Monlinear Spectral Analysis, Sprjnger-Verlag, New York, 1979.Google Scholar
  168. [59]
    K.J. Astrom and P. Eykhoff, “System Identification–A Survey”, Automatica, 7, pp. 123–162, 1971.MathSciNetGoogle Scholar
  169. M. Morf, T. Kailath and L. Ljung, “Fast Algorithms for Recursive Identifcation”, Proc. 1975 IEEE Conf. on Decision and Contr., pp. 916–921, Florida, Dec. 1976.Google Scholar
  170. [61]
    V. Solo, “Time Series Recursions and Stochastic Approximation”, Ph.D. Dissertation, Dept. of Statistics, Australian National Univ., Camberra, Australia, 1978.Google Scholar
  171. [62]
    R. Isermann, ed., Special Issue on Identification and System Parameter Estimation, Automatica, 17, pp. 1–259, Jan. 1981.Google Scholar
  172. [63]
    V. Strejc, “Trends in Identification”, Automatica, 17, pp. 1–22, Jan 1981.MathSciNetGoogle Scholar
  173. [64]
    P. Young, “Parameter Estimation for Continuous-Time Models — A Survey”, Automatica, 17, pp. 23–40, Jan. 1981.MATHGoogle Scholar
  174. [65]
    L. Ljung and T. Soderstrom, Theory and Practice of Recursive Identification, MIT Press, Cambridge, MA, 1982.Google Scholar
  175. [66]
    D. Messerschmitt, “A Geometric Theory of Intersymbol Interference, Pt. I: Zero-Forcing and Decision-Feedback Equalization”, Bell Syst. Tech. J., pp. 1483–1519, 1973.Google Scholar
  176. [67]
    D. Godard, “Channel Equalization Using a Kalman Filer for Fast Data Transmission”, IBM J. Res. Develop., pp. 267–273, May 1974.Google Scholar
  177. [68]
    D.D. Falconer and L. Ljung, “Application of Fast Kalman, Estimation to Adaptive Equalization”, IEEE Trans. Communications, COM-26, pp. 1439–1446, Oct. 1978.Google Scholar
  178. [69]
    L.E. Franks, ed., Data Communication, Benchmark Papers in Electrical Engineering and Computer Science, Halsted Press, New York, 1974.Google Scholar
  179. [70]
    E.H. Satorius and J.D. Pack, “Application of Least-Squares Lattice Algorithms to Adaptive Equalization”, IEEE Trans. on Communication, COM-29, pp. 136–142, Feb. 1981.Google Scholar
  180. [71]
    M.S. Mueller and J. Salz, “A Unified Theory of Data-Aided Equalization” Bell Syst. Tech. J., 1981.Google Scholar
  181. [72]
    V.U. Reddy, A.M. Peterson and T. Kailath, “Application of Modified Least-Squares Algorithms to Adaptive Echo Cancellation”, Proc. Int’L Symp. on Microwaves and Communication, Kharagpur, INDIA, Dec. 1981.Google Scholar
  182. [73]
    L.C. Wood and S. Treitel, “Seismic Signal Processing”, Proc. IEEE, 63, pp. 649–661, A April 1975.Google Scholar
  183. [74]
    J.D. Markel and A.H. Gray, Linear Prediction of Speech, Springer-Verlag, New York, 1976.MATHGoogle Scholar
  184. [75]
    J. Makhoul, “Linear Prediction: A Tutorial Review”, Proc IEEE, 63, pp. 561–580, April 1975.ADSGoogle Scholar
  185. M. Morf and D.T. Lee, “Recursive Least-Squares Ladder Forms for Fast Parameter Tracking”, Proc. 1978 IEEE Conf. on Decision and Contr., pp. 1362–1367, San Diego, CA, Jan. 1979.Google Scholar
  186. [77]
    S. Wood and M. Morf, “A Fast Implementation of a Minimum Variance Estimator for Computerized Tomography Image Reconstruction”, Trans. Bio-Medical Engineering, BME-28, pp. 56–88, Feb. 1981.Google Scholar
  187. [78]
    S.R. Parker and L.J. Griffiths, eds., Joint Special Issue on Adaptive Signal Processing, IEEE Trans. on Acoustics, Speech and Signal Processing, ASSP-29, June 1981.Google Scholar
  188. [79]
    A. Benveniste and P. Dewilde, eds., Fast Algorithms for Linear Dynamical Systems, published by INRIA/CNRS, Paris, France, 1982. 48Google Scholar

Copyright information

© Springer-Verlag Wien 1981

Authors and Affiliations

  • Thomas Kailath
    • 1
  1. 1.Department of Electrical EngineeringStanford UniversityStanfordUSA

Personalised recommendations