Skip to main content
SpringerLink
Log in

A Survey of Convergence Results for Maximum Entropy Methods

  • Chapter
Maximum Entropy and Bayesian Methods

Part of the book series: Fundamental Theories of Physics ((FTPH,volume 53))

Abstract

Maximum entropy methods seek to estimate an unknown density function, typically nonnegative, on the basis of some known moments — integrals of the function with respect to given weights. The estimate is chosen to minimize some measure of entropy, a convex integral functional of the density, subject to the given moment constraints. A desirable feature of such a method is that the estimates should converge to the unknown density as the number of known moments grows. We survey recent results demonstrating how various types of convergence (weak-star, weak and norm in L 1 and L p , measure, and uniform) result from various properties of the entropy (strict convexity, smoothness, and growth conditions). This investigation may be seen as an extension of the classical study of the convergence of Fourier series to the framework of maximum entropy problems. Rather than present the most general theorems known, the unified pattern of the results is illustrated on a single fairly general model problem. References are given for the more general results and their proofs.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 259.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 329.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 329.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. D. Amir and Z. Ziegler. Polynomials of extremal LP-norm on the L,,-Unit sphere. Jounal of Approximation Theory,18:86-98, 1976.

    Google Scholar 

  2. E. Asplund. Fréchet differentiability of convex functions. Acta Mathematica, 121: 31 - 47, 1968.

    Article  MathSciNet  MATH  Google Scholar 

  3. J.M. Borwein and A.S. Lewis. Convergence in measure, mean and max norm for sequences in L1. Forthcoming.

    Google Scholar 

  4. J.M. Borwein and A.S. Lewis. Convergence of best entropy estimates. SIAM Journal on Optimization, 1: 191 - 205, 1991.

    Article  MathSciNet  MATH  Google Scholar 

  5. J.M. Borwein and A.S. Lewis. Duality relationships for entropy-like minimization problems. SIAM Journal on Control and Optimization, 29: 325 - 338, 1991.

    Article  MathSciNet  MATH  Google Scholar 

  6. J.M. Borwein and A.S. Lewis. On the convergence of moment problems. Transactions of the American Mathematical Society, 325: 249 - 271, 1991.

    Article  MathSciNet  MATH  Google Scholar 

  7. J.M. Borwein and A.S. Lewis. Partially finite convex programming, Part I, Duality theory. Mathematical Programming B, pages 15 - 48, 1992.

    Google Scholar 

  8. J.M. Borwein and A.S. Lewis. Partially finite convex programming, Part II, Explicit lattice models. Mathematical Programming B, pages 49 - 84, 1992.

    Google Scholar 

  9. J.M. Borwein and A.S. Lewis. Partially-finite programming in L 1 and the existence of maximum entropy estimates. SIAM Journal on Optimization,1992. To appear. CORR 91-05, University of Waterloo.

    Google Scholar 

  10. J.M. Borwein and A.S. Lewis. Strong convexity and optimization. SIAM Journal on Optimization, 1993. To appear.

    Google Scholar 

  11. J.M. Borwein, A.S. Lewis, and M.A. Limber. Entropy minimization with lattice bounds. Technical Report CORR 92-05, University of Waterloo, 1992. Submitted to Journal of Approximation Theory.

    Google Scholar 

  12. J.M. Borwein and H. Wolkowicz. A simple constraint qualification in infinite dimensional programming. Mathematical Programming, 35: 83 - 96, 1986.

    Article  MathSciNet  MATH  Google Scholar 

  13. Peter Borwein and A.S. Lewis. Moment-matching and best entropy estimation. Technical Report CORR 91-03, University of Waterloo, 1991. Submitted to Journal of Mathematical Analysis and Applications.

    Google Scholar 

  14. B. Buck and V.A. Macaulay, editors. Maximum entropy in action. Oxford University Press, Oxford, 1991.

    Google Scholar 

  15. J.P. Burg. Maximum entropy spectral analysis. Paper presented at 37th meeting of the Society of Exploration Geophysicists, Oklahoma City, 1967.

    Google Scholar 

  16. E.W. Cheney. Introduction to approximation theory. McGraw-Hill, New York, 1966.

    MATH  Google Scholar 

  17. D. Dacunha-Castelle and F. Gamboa. Maximum dentropie et problème des moments. Annales de lInstitut Henri Poincaré, 26: 567 - 596, 1990.

    MathSciNet  MATH  Google Scholar 

  18. J.E. Dennis and R.B. Schnabel. Numerical methods for unconstrained optimization and nonlinear equations. Prentice-Hall, New Jersey, 1983.

    MATH  Google Scholar 

  19. J. Diestel. Sequences and Series in Banach Spaces. Springer-Verlag, New York, 1984.

    Book  Google Scholar 

  20. N. Dunford and J.T. Schwartz. Linear operators, volume 1. Interscience, New York, 1958.

    MATH  Google Scholar 

  21. B. Forte, W. Hughes, and Z. Pales. Maximum entropy estimators and the problem of moments. Rendiconti di Matematica, Serie VII, 9: 689 - 699, 1989.

    MathSciNet  MATH  Google Scholar 

  22. F. Gamboa. Methode du Maximum dEntropie sur la Moyenne et Applications. PhD thesis, Universite Paris Sud, Centre dOrsay, 1989.

    Google Scholar 

  23. E. Gassiat. Probléme sommatoire par maximum dentropie. Comptes Rendues de lAcadémie des Sciences de Paris, Série I, 303: 675 - 680, 1986.

    MathSciNet  MATH  Google Scholar 

  24. R.K. Goodrich and A. Steinhardt. L2 spectral estimation. SIAM Journal on Applied Mathematics, 46: 417 - 428, 1986.

    Article  MathSciNet  MATH  Google Scholar 

  25. R.B. Holmes. Geometric functional analysis and its applications. Springer-Verlag, New York, 1975.

    Book  MATH  Google Scholar 

  26. E.T. Jaynes. Prior probabilites. IEEE Transactions, SSC-4: 227 - 241, 1968.

    Google Scholar 

  27. A.S. Lewis. Pseudo-Haar functions and partially-finite programming. Manuscript.

    Google Scholar 

  28. A.S. Lewis. The convergence of entropic estimates for moment problems. In S. Fitzpatrick and J. Giles, editors, Workshop/Miniconference on Functional Analysis/Optimization, pages 100 - 115, Canberra, 1989. Centre for Mathematical Analysis, Australian National University.

    Google Scholar 

  29. A.S. Lewis. The convergence of Burg and other entropy estimates. Technical Report CORR 92-08, University of Waterloo, 1992.

    Google Scholar 

  30. A.S. Lewis. Facial reduction in partially finite convex programming. Technical Report CORR 92-07, University of Waterloo, 1992.

    Google Scholar 

  31. L.R. Mead and N. Papanicolaou. Maximum entropy in the problem of moments. Journal of Mathematical Physics, 25: 2404 - 2417, 1984.

    Article  MathSciNet  Google Scholar 

  32. J. Navaza. The use of non-local constraints in maximum-entropy electron density reconstruction. Acta Crystallographica, A42: 212 - 223, 1986.

    Google Scholar 

  33. R. Nityananda and R. Narayan. Maximum entropy image reconstruction — a practical non-information-theoretic approach. Journal of Astrophysics and Astronomy, 3: 419450, 1982.

    Google Scholar 

  34. R.T. Rockafellar. Integrals which are convex functionals. Pacific Journal of Mathematics, 24: 525 - 539, 1968.

    Article  MathSciNet  MATH  Google Scholar 

  35. R.T. Rockafellar. Convex Analysis. Princeton University Press, Princeton, N.J., 1970.

    MATH  Google Scholar 

  36. W. Rudin. Real and complex analysis. McGraw-Hill, New York, 1966.

    MATH  Google Scholar 

  37. J. Skilling and S.F. Gull. The entropy of an image. SIAM-AMS Proceedings, 14: 167189, 1984.

    Google Scholar 

  38. M. Teboulle and I. Vajda. Convergence of best 0-entropy estimates. IEEE Transactions on Information Theory, 1992. To appear.

    Google Scholar 

  39. A. Visintin. Strong convergence results related to strict convexity. Communications on partial differential equations, 9: 439 - 466, 1984.

    Article  MathSciNet  MATH  Google Scholar 

  40. Z. Ziegler. Minimizing the 4,,,o-distortion of trigonometric polynomials. Journal of Mathematical Analysis and Applications, 61: 426 - 431, 1977.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Combinatorics and Optimization, University of Waterloo, Waterloo, N2L 3G1, Ontario, Canada

    J. M. Borwein & A. S. Lewis

Authors
  1. J. M. Borwein
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. S. Lewis
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Groupe Problèmes Inverses, Laboratoire des Signaux et Systèmes (CNRS-ESE-UPS), 91192, Gif-sur-Yvette Cedex, France

    Ali Mohammad-Djafari  & Guy Demoment  & 

Rights and permissions

Reprints and permissions

Copyright information

© 1993 Springer Science+Business Media Dordrecht

About this chapter

Cite this chapter

Borwein, J.M., Lewis, A.S. (1993). A Survey of Convergence Results for Maximum Entropy Methods. In: Mohammad-Djafari, A., Demoment, G. (eds) Maximum Entropy and Bayesian Methods. Fundamental Theories of Physics, vol 53. Springer, Dordrecht. https://doi.org/10.1007/978-94-017-2217-9_5

Download citation

  • DOI: https://doi.org/10.1007/978-94-017-2217-9_5

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-90-481-4272-9

  • Online ISBN: 978-94-017-2217-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation