# A Convex Optimization Approach to Generalized Moment Problems

- 21 Citations
- 314 Downloads

## Abstract

In this paper we present a universal solution to the generalized moment problem, with a nonclassical complexity constraint. We show that this solution can be obtained by minimizing a strictly convex nonlinear functional. This optimization problem is derived in two different ways. We first derive this intrinsically, in a geometric way, by path integration of a one-form which defines the generalized moment problem. It is observed that this one-form is closed and defined on a convex set, and thus exact with, perhaps surprisingly, a strictly convex primitive function. We also derive this convex functional as the dual problem of a problem to maximize a cross entropy functional. In particular, these approaches give a constructive parameterization of all solutions to the Nevanlinna-Pick interpolation problem, with possible higher-order interpolation at certain points in the complex plane, with a degree constraint as well as all solutions to the rational covariance extension problem — two areas which have been advanced by the work of Hidenori Kimura. Illustrations of these results in system identification and probability are also mentioned.

## Keywords

Moment problems Convex optimization Nevanlinna-Pick interpolation Covariance extension Systems identification Kullback-Leibler distance## Preview

Unable to display preview. Download preview PDF.

## References

- [1]N. I. Akhiezer, The Classical Moment Problem and Some Related Questions in Analysis. Hafner Publishing, 1965.zbMATHGoogle Scholar
- [2]N. I. Ahiezer and M. Krein
*Some Questions in the Theory of Moments*Providence, Rhode Island: American Mathematical Society, 1962.Google Scholar - [3]C. I. Byrnes and A. Lindquist, “On the geometry of the KimuraGeorgiou parameterization of modelling filter,”
*Int. J. Control*vol. 50, pp. 99–105, 1989.MathSciNetCrossRefGoogle Scholar - [4]C. I. Byrnes, A. Lindquist, and T. McGregor, “Predictability and unpredictability in Kalman filtering,”
*IEEE Trans. Automatic Control*vol. 36, pp. 563–579, 1991.MathSciNetzbMATHCrossRefGoogle Scholar - [5]C. I. Byrnes, A. Lindquist, and Y. Zhou, “On the nonlinear dynamics of fast filtering algorithms,”
*SIAM J. Control and Optimization*vol. 32, pp. 744–789, 1994.MathSciNetzbMATHCrossRefGoogle Scholar - [6]C. I. Byrnes, A. Lindquist, S. V. Gusev and A. S. Matveev, “A complete parametrization of all positive rational extensions of a covariance sequence,”
*IEEE Trans*.*Automatic Control*vol. AC-40, pp. 1841–1857, 1995.MathSciNetCrossRefGoogle Scholar - [7]C. I. Byrnes, S. V. Gusev, and A. Lindquist, “A convex optimization approach to the rational covariance extension problem,”
*SIAM J. Control and Optimization*vol. 37, pp. 211–229, 1999.MathSciNetzbMATHCrossRefGoogle Scholar - [8]C. I. Byrnes and A. Lindquist, “On the duality between filtering and Nevanlinna¡ªPick interpolation,”
*SIAM J. Control and Optimization*vol. 39, pp. 757–775, 2000.MathSciNetzbMATHCrossRefGoogle Scholar - [9]C. I. Byrnes, T. T. Georgiou and A. Lindquist, “A generalized entropy criterion for Nevanlinna¡ªPick interpolation with degree constraint,”
*IEEE Trans. Automatic Control*vol. AC-46, pp. 822–839, 2001.MathSciNetCrossRefGoogle Scholar - [10]C. I. Byrnes, T. T. Georgiou, and A. Lindquist, “A new approach to spectral estimation: A tunable high-resolution spectral estimator,”
*IEEE Trans. Signal Processing*vol. SP-49, pp. 3189–3205, Nov. 2000.MathSciNetCrossRefGoogle Scholar - [11]C. I. Byrnes, T. T. Georgiou, and A. Lindquist, “Generalized interpolation in
*H¡ã¡ã:*Solutions of bounded complexity,” preprint.Google Scholar - [12]C. I. Byrnes, T. T. Georgiou, and A. Lindquist, forthcoming paper.Google Scholar
- [13]C. I. Byrnes, P. Enqvist, and A. Lindquist, “Cepstral coefficients, covariance lags and pole-zero models for finite data strings,”
*IEEE Trans. Signal Processing*vol.*SP*-50, pp. 677–693, 2001.MathSciNetGoogle Scholar - [14]C. I. Byrnes, P. Enqvist, and A. Lindquist, “Identifiability and wellposedness of shaping-filter parameterizations: A global analysis approach,”
*SIAM J. Control and Optimization**vol.*41, pp. 23–59, 2002.MathSciNetzbMATHCrossRefGoogle Scholar - [15]C. I. Byrnes, S. V. Gusev, and A. Lindquist, “From finite covariance windows to modeling filters: A convex optimization approach,”
*SIAM Review**vol.*43, pp. 645–675, Dec. 2001.MathSciNetzbMATHCrossRefGoogle Scholar - [16]C. I. Byrnes and A. Lindquist, “Interior point solutions of variational problems and global inverse function theorems,” submitted for publication.Google Scholar
- [17]I. Csiszár“I-divergence geometry of probability distributions and minimization problems,”
*Ann. Probab.*vol. 3, pp. 146–158, 1975.zbMATHCrossRefGoogle Scholar - [18]I. Csiszár and F. Matús, “Information projections revisited,” preprint, 2001.Google Scholar
- [19]Ph. Delsarte, Y. Genin, and Y. Kamp, “On the role of the NevanlinnaPick problem in circuits and system theory,”
*Circuit Theory and Applications**vol.*9, pp. 177–187, 1981.MathSciNetzbMATHCrossRefGoogle Scholar - [20]Ph. Delsarte, Y. Genin, Y. Kamp, and P. van Dooren, “Speech modelling and the trigonometric moment problem,”
*Philips J. Res.**vol.*37, pp. 277–292, 1982.MathSciNetzbMATHGoogle Scholar - [21]
*J.*C. Doyle, B. A. Frances, and A. R. Tannenbaum*Feedback Control Theory.*New York: Macmillan Publ. Co., 1992.Google Scholar - [22]P. Enqvist, “Spectral estimation by geometric, topological and optimization Methods,” Ph.D. thesis, Optimization and Systems Theory, KTH, Stockholm, Sweden, 2001.Google Scholar
- [23]T. T. Georgiou, “Realization of power spectra from partial covariance sequences,” IEEE Trans. Acoustics, Speech, and Signal Processing
*vol.*ASSP-35, pp. 438–449, 1987.CrossRefGoogle Scholar - [24]T. T. Georgiou, “A topological approach to Nevanlinna¡ªPick interpolation,”
*SIAM J. Math. Analysis**vol.*18, pp. 1248–1260, 1987.MathSciNetzbMATHCrossRefGoogle Scholar - [25]T. T. Georgiou, “The interpolation problem with a degree constraint,”
*IEEE Trans. Automatic Control**vol.*44, no. 3, pp. 631–635, 1999.MathSciNetzbMATHCrossRefGoogle Scholar - [26]I.
*J.*Good, “Maximum entropy for hypothesis formulation, especially for multidimensional contingency tables,”*Annals Math. Stat.*vol. 34, pp. 911–934, 1963.MathSciNetzbMATHCrossRefGoogle Scholar - [27]U. Grenander and G. Szegö, “Toeplitz forms and their applications,” Univ. California Press, 1958.Google Scholar
- [28]J. W. Helton, “Non-Euclidean analysis and electronics,”
*Bull. Amer. Math.**Soc.*vol. 7, pp. 1–64, 1982.MathSciNetzbMATHCrossRefGoogle Scholar - [29]P. S. C. Heuberger, T. J. de Hoog, P. M. J. Van den Hof, Z. Szab¨®, and J. Bokar, “Minimal partial realization from orthonormal basis function expansions,” in
*Proc. 40th IEEE Conf Decision and Control*Orlando, Florida, USA, December 2001, pp. 3673–3678.Google Scholar - [30]R. E. Kalman, “Realization of covariance sequences,” in
*Proc. Toeplitz Memorial Conference (1981)*Tel Aviv, Israel, 1981.Google Scholar - [31]H. Kimura, “Positive partial realization of covariance sequences,” in
*Modelling**Identification and Robust Control*C. I. Byrnes and A. Lindquist, Eds. North-Holland, 1987, pp. 499–513.Google Scholar - [32]H. Kimura, “Robust stabilizability for a class of transfer functions,” IEEE Trans. Automatic Control, vol. AC-29, pp. 788–793, 1984.CrossRefGoogle Scholar
- [33]H. Kimura, “State space approach to the classical interpolation problem and its applications,” in
*Three Decades of Mathematical System Theory*Lecture Notes in Control and Inform. Sci., vol. 135, Berlin: Springer, 1989, pp. 243–275.Google Scholar - [34]H. Kimura and H. Iwase, “On directional interpolation in
*H¡ã¡ã**”*in*Linear circuits**systems and signal processing: theory and application*(Phoenix, AZ, 1987), Amsterdam: North-Holland, 1988, pp. 551–560.Google Scholar - [35]H. Kimura, “Conjugation, interpolation and model-matching in
*H¡ã¡ã**”**Int. J. Control*vol. 49, pp. 269–307, 1989.zbMATHGoogle Scholar - [36]H. Kimura, “Directional interpolation in the state space,”
*Systems Control Lett.*vol. 10, pp. 317–324, 1988.MathSciNetzbMATHCrossRefGoogle Scholar - [37]H. Kimura, “Directional interpolation approach to H¡ã¡ã-optimization and robust stabilization,”
*IEEE Trans. Automatic Control*vol. AC-32, pp. 1085–1093, 1987.CrossRefGoogle Scholar - [38]M. G. Krein and A. A. Nudelman
*The Markov Moment Problem and Extremal Problems*Providence, Rhode Island: American Mathematical Society, 1977.Google Scholar - [39]S. Kullback
*Information Theory and Statistics*New York John Wiley, 1959.zbMATHGoogle Scholar - [40]A. Lindquist, “A new algorithm for optimal filtering of discrete-time stationary processes,”
*SIAM J. Control, vol. 12**pp.*736–746, 1974.MathSciNetzbMATHCrossRefGoogle Scholar - [41]A. Lindquist, “Some reduced-order non-Riccati equations for linear least-squares estimation: the stationary, single-output case,”
*Int. J. Control*vol. 24, pp. 821–842, 1976.MathSciNetzbMATHCrossRefGoogle Scholar - [42]R. Nagamune and A. Lindquist, “Sensitivity shaping in feedback control and analytic interpolation theory,” in
*Optimal Control and Partial Differential Equations**J.*L. Medaldi et al. Eds. Amsterdam: IOS Press, 2001, pp. 404–413.Google Scholar - [43]R. Nagamune, “A robust solver using a continuation method for Nevanlinna¡ªPick interpolation with degree constraint,”
*IEEE Trans. Automatic Control*to appear.Google Scholar - [44]R. Nagamune, “Closed-loop shaping based on Nevanlinna¡ªPick interpolation with a degree bound,” submitted to
*IEEE Trans. Automatic Control.*Google Scholar - [45]R. Nagamune, “Robust control with complexity constraint,” Ph.D. thesis, Optimization and Systems Theory, Royal Institute of Technology, Stockholm, Sweden, 2002.Google Scholar
- [46]A. Renyi
*Probability Theory.*North-Holland, 1970.Google Scholar - [47]Z. Szab¨®, P. Heuberger, J. Bokar, and P. Van den Hof, “Extended Ho¡ªKalman algorithm for systems represented in generalized orthonormal bases,”
*Automatica**vol.*36, pp. 1809–1818, 2000.Google Scholar - [48]W. Rudin
*Real and Complex Analysis.*McGraw-Hill, 1966.zbMATHGoogle Scholar - [49]J. A. Shohat and J. D. Tamarkin
*The Problem of Moments*Mathematical Surveys I. New York: American Mathematical Society, 1943.Google Scholar - [50]A. Tannenbaum, “Feedback stabilization of linear dynamical plants with uncertainty in the gain factor,”
*Int. J. Control**vol.*32, pp. 1–16, 1980.MathSciNetzbMATHCrossRefGoogle Scholar - [51]B. Wahlberg “Systems identification using Laguerre models,” IEEE Trans. Automatic Control
*vol.*AC-36, pp. 551–562, 1991.MathSciNetCrossRefGoogle Scholar - [52]B. Wahlberg “System identification using Kautz models,” IEEE Trans. Automatic Control
*vol.*AC-39, pp. 1276–1282, 1994.MathSciNetCrossRefGoogle Scholar - [53]K. Zhou
*Essentials of Robust Control.*Prentice-Hall, 1998.Google Scholar