A Convex Optimization Approach to Generalized Moment Problems

  • Christopher I. Byrnes
  • Anders Lindquist
Part of the Trends in Mathematics book series (TM)


In this paper we present a universal solution to the generalized moment problem, with a nonclassical complexity constraint. We show that this solution can be obtained by minimizing a strictly convex nonlinear functional. This optimization problem is derived in two different ways. We first derive this intrinsically, in a geometric way, by path integration of a one-form which defines the generalized moment problem. It is observed that this one-form is closed and defined on a convex set, and thus exact with, perhaps surprisingly, a strictly convex primitive function. We also derive this convex functional as the dual problem of a problem to maximize a cross entropy functional. In particular, these approaches give a constructive parameterization of all solutions to the Nevanlinna-Pick interpolation problem, with possible higher-order interpolation at certain points in the complex plane, with a degree constraint as well as all solutions to the rational covariance extension problem — two areas which have been advanced by the work of Hidenori Kimura. Illustrations of these results in system identification and probability are also mentioned.


Moment problems Convex optimization Nevanlinna-Pick interpolation Covariance extension Systems identification Kullback-Leibler distance 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    N. I. Akhiezer, The Classical Moment Problem and Some Related Questions in Analysis. Hafner Publishing, 1965.zbMATHGoogle Scholar
  2. [2]
    N. I. Ahiezer and M. KreinSome Questions in the Theory of MomentsProvidence, Rhode Island: American Mathematical Society, 1962.Google Scholar
  3. [3]
    C. I. Byrnes and A. Lindquist, “On the geometry of the KimuraGeorgiou parameterization of modelling filter,”Int. J. Controlvol. 50, pp. 99–105, 1989.MathSciNetCrossRefGoogle Scholar
  4. [4]
    C. I. Byrnes, A. Lindquist, and T. McGregor, “Predictability and unpredictability in Kalman filtering,”IEEE Trans. Automatic Controlvol. 36, pp. 563–579, 1991.MathSciNetzbMATHCrossRefGoogle Scholar
  5. [5]
    C. I. Byrnes, A. Lindquist, and Y. Zhou, “On the nonlinear dynamics of fast filtering algorithms,”SIAM J. Control and Optimizationvol. 32, pp. 744–789, 1994.MathSciNetzbMATHCrossRefGoogle Scholar
  6. [6]
    C. I. Byrnes, A. Lindquist, S. V. Gusev and A. S. Matveev, “A complete parametrization of all positive rational extensions of a covariance sequence,”IEEE Trans.Automatic Controlvol. AC-40, pp. 1841–1857, 1995.MathSciNetCrossRefGoogle Scholar
  7. [7]
    C. I. Byrnes, S. V. Gusev, and A. Lindquist, “A convex optimization approach to the rational covariance extension problem,”SIAM J. Control and Optimizationvol. 37, pp. 211–229, 1999.MathSciNetzbMATHCrossRefGoogle Scholar
  8. [8]
    C. I. Byrnes and A. Lindquist, “On the duality between filtering and Nevanlinna¡ªPick interpolation,”SIAM J. Control and Optimizationvol. 39, pp. 757–775, 2000.MathSciNetzbMATHCrossRefGoogle Scholar
  9. [9]
    C. I. Byrnes, T. T. Georgiou and A. Lindquist, “A generalized entropy criterion for Nevanlinna¡ªPick interpolation with degree constraint,”IEEE Trans. Automatic Controlvol. AC-46, pp. 822–839, 2001.MathSciNetCrossRefGoogle Scholar
  10. [10]
    C. I. Byrnes, T. T. Georgiou, and A. Lindquist, “A new approach to spectral estimation: A tunable high-resolution spectral estimator,”IEEE Trans. Signal Processingvol. SP-49, pp. 3189–3205, Nov. 2000.MathSciNetCrossRefGoogle Scholar
  11. [11]
    C. I. Byrnes, T. T. Georgiou, and A. Lindquist, “Generalized interpolation inH¡ã¡ã:Solutions of bounded complexity,” preprint.Google Scholar
  12. [12]
    C. I. Byrnes, T. T. Georgiou, and A. Lindquist, forthcoming paper.Google Scholar
  13. [13]
    C. I. Byrnes, P. Enqvist, and A. Lindquist, “Cepstral coefficients, covariance lags and pole-zero models for finite data strings,”IEEE Trans. Signal Processingvol.SP-50, pp. 677–693, 2001.MathSciNetGoogle Scholar
  14. [14]
    C. I. Byrnes, P. Enqvist, and A. Lindquist, “Identifiability and wellposedness of shaping-filter parameterizations: A global analysis approach,”SIAM J. Control and Optimization vol.41, pp. 23–59, 2002.MathSciNetzbMATHCrossRefGoogle Scholar
  15. [15]
    C. I. Byrnes, S. V. Gusev, and A. Lindquist, “From finite covariance windows to modeling filters: A convex optimization approach,”SIAM Review vol.43, pp. 645–675, Dec. 2001.MathSciNetzbMATHCrossRefGoogle Scholar
  16. [16]
    C. I. Byrnes and A. Lindquist, “Interior point solutions of variational problems and global inverse function theorems,” submitted for publication.Google Scholar
  17. [17]
    I. Csiszár“I-divergence geometry of probability distributions and minimization problems,”Ann. Probab.vol. 3, pp. 146–158, 1975.zbMATHCrossRefGoogle Scholar
  18. [18]
    I. Csiszár and F. Matús, “Information projections revisited,” preprint, 2001.Google Scholar
  19. [19]
    Ph. Delsarte, Y. Genin, and Y. Kamp, “On the role of the NevanlinnaPick problem in circuits and system theory,”Circuit Theory and Applications vol.9, pp. 177–187, 1981.MathSciNetzbMATHCrossRefGoogle Scholar
  20. [20]
    Ph. Delsarte, Y. Genin, Y. Kamp, and P. van Dooren, “Speech modelling and the trigonometric moment problem,”Philips J. Res. vol.37, pp. 277–292, 1982.MathSciNetzbMATHGoogle Scholar
  21. [21]
    J.C. Doyle, B. A. Frances, and A. R. TannenbaumFeedback Control Theory.New York: Macmillan Publ. Co., 1992.Google Scholar
  22. [22]
    P. Enqvist, “Spectral estimation by geometric, topological and optimization Methods,” Ph.D. thesis, Optimization and Systems Theory, KTH, Stockholm, Sweden, 2001.Google Scholar
  23. [23]
    T. T. Georgiou, “Realization of power spectra from partial covariance sequences,” IEEE Trans. Acoustics, Speech, and Signal Processingvol.ASSP-35, pp. 438–449, 1987.CrossRefGoogle Scholar
  24. [24]
    T. T. Georgiou, “A topological approach to Nevanlinna¡ªPick interpolation,”SIAM J. Math. Analysis vol.18, pp. 1248–1260, 1987.MathSciNetzbMATHCrossRefGoogle Scholar
  25. [25]
    T. T. Georgiou, “The interpolation problem with a degree constraint,”IEEE Trans. Automatic Control vol.44, no. 3, pp. 631–635, 1999.MathSciNetzbMATHCrossRefGoogle Scholar
  26. [26]
    I.J.Good, “Maximum entropy for hypothesis formulation, especially for multidimensional contingency tables,”Annals Math. Stat.vol. 34, pp. 911–934, 1963.MathSciNetzbMATHCrossRefGoogle Scholar
  27. [27]
    U. Grenander and G. Szegö, “Toeplitz forms and their applications,” Univ. California Press, 1958.Google Scholar
  28. [28]
    J. W. Helton, “Non-Euclidean analysis and electronics,”Bull. Amer. Math. Soc.vol. 7, pp. 1–64, 1982.MathSciNetzbMATHCrossRefGoogle Scholar
  29. [29]
    P. S. C. Heuberger, T. J. de Hoog, P. M. J. Van den Hof, Z. Szab¨®, and J. Bokar, “Minimal partial realization from orthonormal basis function expansions,” inProc. 40th IEEE Conf Decision and ControlOrlando, Florida, USA, December 2001, pp. 3673–3678.Google Scholar
  30. [30]
    R. E. Kalman, “Realization of covariance sequences,” inProc. Toeplitz Memorial Conference (1981)Tel Aviv, Israel, 1981.Google Scholar
  31. [31]
    H. Kimura, “Positive partial realization of covariance sequences,” inModelling Identification and Robust ControlC. I. Byrnes and A. Lindquist, Eds. North-Holland, 1987, pp. 499–513.Google Scholar
  32. [32]
    H. Kimura, “Robust stabilizability for a class of transfer functions,” IEEE Trans. Automatic Control, vol. AC-29, pp. 788–793, 1984.CrossRefGoogle Scholar
  33. [33]
    H. Kimura, “State space approach to the classical interpolation problem and its applications,” inThree Decades of Mathematical System TheoryLecture Notes in Control and Inform. Sci., vol. 135, Berlin: Springer, 1989, pp. 243–275.Google Scholar
  34. [34]
    H. Kimura and H. Iwase, “On directional interpolation inH¡ã¡ã inLinear circuits systems and signal processing: theory and application(Phoenix, AZ, 1987), Amsterdam: North-Holland, 1988, pp. 551–560.Google Scholar
  35. [35]
    H. Kimura, “Conjugation, interpolation and model-matching inH¡ã¡ã Int. J. Controlvol. 49, pp. 269–307, 1989.zbMATHGoogle Scholar
  36. [36]
    H. Kimura, “Directional interpolation in the state space,”Systems Control Lett.vol. 10, pp. 317–324, 1988.MathSciNetzbMATHCrossRefGoogle Scholar
  37. [37]
    H. Kimura, “Directional interpolation approach to H¡ã¡ã-optimization and robust stabilization,”IEEE Trans. Automatic Controlvol. AC-32, pp. 1085–1093, 1987.CrossRefGoogle Scholar
  38. [38]
    M. G. Krein and A. A. NudelmanThe Markov Moment Problem and Extremal ProblemsProvidence, Rhode Island: American Mathematical Society, 1977.Google Scholar
  39. [39]
    S. KullbackInformation Theory and StatisticsNew York John Wiley, 1959.zbMATHGoogle Scholar
  40. [40]
    A. Lindquist, “A new algorithm for optimal filtering of discrete-time stationary processes,”SIAM J. Control, vol. 12 pp.736–746, 1974.MathSciNetzbMATHCrossRefGoogle Scholar
  41. [41]
    A. Lindquist, “Some reduced-order non-Riccati equations for linear least-squares estimation: the stationary, single-output case,”Int. J. Controlvol. 24, pp. 821–842, 1976.MathSciNetzbMATHCrossRefGoogle Scholar
  42. [42]
    R. Nagamune and A. Lindquist, “Sensitivity shaping in feedback control and analytic interpolation theory,” inOptimal Control and Partial Differential Equations J.L. Medaldi et al. Eds. Amsterdam: IOS Press, 2001, pp. 404–413.Google Scholar
  43. [43]
    R. Nagamune, “A robust solver using a continuation method for Nevanlinna¡ªPick interpolation with degree constraint,”IEEE Trans. Automatic Controlto appear.Google Scholar
  44. [44]
    R. Nagamune, “Closed-loop shaping based on Nevanlinna¡ªPick interpolation with a degree bound,” submitted toIEEE Trans. Automatic Control. Google Scholar
  45. [45]
    R. Nagamune, “Robust control with complexity constraint,” Ph.D. thesis, Optimization and Systems Theory, Royal Institute of Technology, Stockholm, Sweden, 2002.Google Scholar
  46. [46]
    A. RenyiProbability Theory.North-Holland, 1970.Google Scholar
  47. [47]
    Z. Szab¨®, P. Heuberger, J. Bokar, and P. Van den Hof, “Extended Ho¡ªKalman algorithm for systems represented in generalized orthonormal bases,”Automatica vol.36, pp. 1809–1818, 2000.Google Scholar
  48. [48]
    W. RudinReal and Complex Analysis.McGraw-Hill, 1966.zbMATHGoogle Scholar
  49. [49]
    J. A. Shohat and J. D. TamarkinThe Problem of MomentsMathematical Surveys I. New York: American Mathematical Society, 1943.Google Scholar
  50. [50]
    A. Tannenbaum, “Feedback stabilization of linear dynamical plants with uncertainty in the gain factor,”Int. J. Control vol.32, pp. 1–16, 1980.MathSciNetzbMATHCrossRefGoogle Scholar
  51. [51]
    B. Wahlberg “Systems identification using Laguerre models,” IEEE Trans. Automatic Controlvol.AC-36, pp. 551–562, 1991.MathSciNetCrossRefGoogle Scholar
  52. [52]
    B. Wahlberg “System identification using Kautz models,” IEEE Trans. Automatic Controlvol.AC-39, pp. 1276–1282, 1994.MathSciNetCrossRefGoogle Scholar
  53. [53]
    K. ZhouEssentials of Robust Control.Prentice-Hall, 1998.Google Scholar

Copyright information

© Springer Science+Business Media New York 2003

Authors and Affiliations

  • Christopher I. Byrnes
    • 1
  • Anders Lindquist
    • 2
  1. 1.Department of Systems Science and Mathematics Washington UniversityWashington UniversitySt. LouisUSA
  2. 2.Department of MathematicsDivision of Optimization and Systems Theory Royal Institute of TechnologyStockholmSweden

Personalised recommendations