Abstract
There are now many proofs that the maximum entropy stationary stochastic process, subject to a finite number of autocorrelation constraints, is the Gauss Markov process of appropriate order. The associated spectrum is Burg’s maximum entropy spectral density. We pose a somewhat broader entropy maximization problem, in which stationarity, for example, is not assumed, and shift the burden of proof from the previous focus on the calculus of variations and time series techniques to a string of information theoretic inequalities. This results in a simple proof.
Expanded version of a paper published originally in Proceedings of the IEEE 72, pp. 1094–1095 (1984).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Akaike, H. (1977), “An entropy maximization principle,” in P. Krishnaiah, ed., Proceedings of the Symposium on Applied Statistics, North-Holland, Amsterdam.
Ash, R. (1965), Information Theory, Wiley Interscience, New York.
Berger, T. (1971), Rate Distortion Theory, A Mathematical Basis for Data Compression, Prentice-Hall, N.J.
Burg, J. P. (1967), “Maximum entropy spectral analysis,” presented at the 37th Meeting of the Society of Exploration Geophysicists; reprinted in D. G. Childers, ed. (1978), Modern Spectrum Analysis, IEEE Press, pp. 34–41.
Burg, J. P. (1975), “Maximum Entropy Spectral Analysis,” Ph.D. dissertation, Department of Geophysics, Stanford University, Stanford, Calif.
Choi, B. S. (1983), “A Conditional Limit Characterization of the Maximum Entropy Spectral Density in Time Series Analysis,” Ph.D. dissertation, Statistics Department, Stanford University.
Choi, B. S., and T. M. Cover (1984), “An information-theoretic proof of Burg’s maximum entropy spectrum” (letter), Proc. IEEE 72, pp. 1094–1095.
Choi, B. S., and T. M. Cover (1987), “A conditional limit characterization of Gauss Markov processes,” submitted to JASA.
Edward, J. A., and M. M. Fitelson (1973), “Notes on maximum-entropy processing,” IEEE Trans. Inf. Theory IT-19, pp. 232–234; reprinted in D. G. Childers, ed. (1978), Modern Spectrum Analysis, IEEE Press, pp. 94–96.
Feder, M., and E. Weinstein (1984), “On the finite maximum entropy extrapolation,” Proc. IEEE 72, pp. 1660–1662.
Gallager, R. (1968), Information Theory and Reliable Communication, Wiley, New York.
Grandell, J., M. Hamrud, and P. Toll (1980), “A remark on the correspondence between the maximum entropy method and the autoregressive model,” IEEE Trans. Inf. Theory IT-26, pp. 750–751.
Haykin, S., and S. Kesler (1979), “Prediction-error filtering and maximum entropy spectral estimation,” in S. Haykin, ed., Nonlinear Methods of Spectral Analysis, Springer, New York, pp. 9–72.
Kolmogorov, A. N. (1941), “Interpolation und Extrapolation von Stationären Zufalligen Folgen,” Bull. Acad. Sci. URSS, Ser. Math. 5, pp. 3–41.
McDonough, R. N. (1979), “Application of the maximum-likelihood method and the maximum entropy method to array processing,” in S. Haykin, ed., Nonlinear Methods of Spectral Analysis, Springer, New York, pp. 181–244.
Priestley, M. B. (1981), Spectral Analysis and Time Series, Vol. 1, Academic Press, New York.
Robinson, E. A. (1982), “A historical perspective of spectrum estimation,” Proc. IEEE 70, pp. 885–907.
Smylie, D. G., G. K. C. Clarke, and T. J. Ulrych (1973), “Analysis of irregularities in the earth’s rotation,” Meth. Comp. Phys. 13i, pp. 391–430.
Ulrych, T., and T. Bishop (1975), “Maximum entropy spectral analysis and autoregressive decomposition,” Rev. Geophys. and Space Phys., 13, pp. 183–200; reprinted in D. G. Childers, ed. (1978), Modern Spectrum Analysis, IEEE Press, pp. 54–71.
Ulrych, T., and M. Ooe (1979), “Autoregressive and mixed autoregressive-moving average models and spectra,” in S. Haykin, ed., Nonlinear Methods of Spectral Analysis, Springer, New York, pp. 73–126.
Van den Bos, A. (1971), “Alternative interpretation of maximum entropy spectral analysis,” IEEE Trans. Inf. Theory IT-17, pp. 493–494; reprinted in D. G. Childers, ed. (1978), Modern Spectrum Analysis, IEEE Press, pp. 92–93.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1987 D. Reidel Publishing Company
About this paper
Cite this paper
Choi, B.S., Cover, T.M. (1987). A Proof of Burg’s Theorem. In: Smith, C.R., Erickson, G.J. (eds) Maximum-Entropy and Bayesian Spectral Analysis and Estimation Problems. Fundamental Theories of Physics, vol 21. Springer, Dordrecht. https://doi.org/10.1007/978-94-009-3961-5_5
Download citation
DOI: https://doi.org/10.1007/978-94-009-3961-5_5
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-010-8257-0
Online ISBN: 978-94-009-3961-5
eBook Packages: Springer Book Archive