Skip to main content
SpringerLink
Log in
Book cover

Maximum Entropy and Bayesian Methods pp 253–264Cite as

Maximum Entropy and Linear Inverse Problems. A Short Review

  • Chapter

Part of the book series: Fundamental Theories of Physics ((FTPH,volume 53))

Abstract

In this paper we give a short review of the Maximum Entropy (ME) principle used to solve inverse problems. We distinguish three fundamentally different approaches for solving inverse problems when using the ME principle: a) Classical ME in which the unknown function is considered to be or to have the properties of a probability density function, b) ME in mean in which the unknown function is assumed to be a random function and the data are assumed to be the expected values of some finite number of known constraints on the unknown function, and finally, c) Bayesian approach with ME priors. In this case the ME principle is used only for assigning a probability distribution to the unknown function to translate our prior knowledge about it. In each approach, we describe the main ideas and give explicitly the hypothesis, the practical and the theoretical limitations.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   259.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   329.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   329.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. N. Agmon, Y. Alhassid and D. Levine, “An Algorithm for Finding the Distribution of Maximal Entropy,” Journal of Computational Physics, Vol. 30,pp: 250–258, 1979.

    Google Scholar 

  2. J. M. Borwein and A. S. Lewis, “Duality relationships for entropy-like minimization problems,” SIAM J. Control and Optimization, Vol. 29 No. 2, Mar:,h 1991.

    Google Scholar 

  3. R. K. Bryan, M. Bansal, W. Folkhard., C. Nave and D. A. Marvin, “Maximum—entropy calculation of the electron density at 4 A resolution of Pfl filamentous bacteriophage,” Proc.Natl. Acad. Sci. USA, 80, pp. 4 728–4 731, 1983.

    Google Scholar 

  4. S. F. Burch, S. F. Gull and J. Skilling. Gull and J. Skilling, “Image Restoration by a Powerful Maximum Entropy Method, ” Comput. Vis. Graph. Im. Process., 23, pp. 113–128, 1983.

    Article  Google Scholar 

  5. G. J. Daniell and S. F. Gull, “Maximum Entropy algorithm applied to image enhancement,” Proc. IEE, 127E, pp. 170–172, 1980.

    Google Scholar 

  6. T. Elfwing, “On some Methods for Entropy Maximization and Matrix Scaling,” Linear Algebra and its Applications, 34, pp. 321–339, 1980.

    Article  MathSciNet  Google Scholar 

  7. J. Eriksson, “A note on solution of large sparse maximum entropy problems with linear equality constraints,” Math. Programming,18, pp. 146–154, 1980.

    Google Scholar 

  8. S. Erlander, “Entropy in linear programs,” Mathematical Programming, 21, pp. 137151, 1981.

    Google Scholar 

  9. Frieden B. R., “Band—Unlimited Reconstruction of Optical Objects and Spectra,” J. Opt. Soc. Am., 57, pp. 1 013–1 019, 1967.

    Google Scholar 

  10. Frieden B. R., “Restoring with Maximum Likelihood and Maximum Entropy,” J. Opt. Soc. Am.,62 pp. 511–518, 1972.

    Google Scholar 

  11. Frieden B. R., “Statistical Estimates of Bounded Optical Scenes by the Method of Prior Probabilities,” IEEE Trans.,IT- pp. 118–119, 1973.

    Google Scholar 

  12. Frieden B. R., “Restoring with maximum entropy. III. Poisson sources and backgrounds,” J. Opt. Soc. Am., 68, pp. 93–103, 1978.

    Article  Google Scholar 

  13. Frieden B. R., “Statistical Models for the Image Restoration Problem,” Comput. Graph. Im. Process., 12, pp. 40–59, 1980.

    Article  Google Scholar 

  14. Frieden B. R., “Statistical Models for the Image Restoration Problem,” Comput. Graph. Im. Process., 12, pp. 40–59, 1980.

    Article  Google Scholar 

  15. Frieden B. R. and Zoltani C. K., “Maximum bounded entropy: application to tomo-graphic reconstruction,” Applied Optics, 24, pp. 201–207, 1985.

    Article  Google Scholar 

  16. F. Gamboa, “Méthode du maximum d’entropie sur la moyenne et applications,” Thèse, Dépt. de Mathématiques, Univ. de Paris-Sud, 1989.

    Google Scholar 

  17. S. F. Gull and J. Skilling, “Maximum entropy method in image processing,” IEE Proc., 131-F, pp. 646–659, 1984.

    Google Scholar 

  18. E. T. Jaynes, “On the Rationale of Maximum-Entropy Methods,” Proc. IEEE, 70, pp. 939–952, 1982.

    Article  Google Scholar 

  19. E. T. Jaynes, “Where do we go from here?,” Maximum-Entropy and Bayesian Methods in Inverse Problems, C. R. Smith T. Grandy, Jr. (eds.), pp. 21–58, 1985.

    Google Scholar 

  20. R. Johnson and J. Shore, “Which is Better Entropy Expression for Speech Processing: -SlogS or logS?,” IEEE Trans, ASSP-32, pp. 129–137, 1984.

    Google Scholar 

  21. R. Kikuchi and B. H. Soifer, “Maximum entropy image restoration. I. The entropy expression,” J. Opt. Soc. Am., 67, pp. 1 656–1 665, 1977.

    Google Scholar 

  22. G. Le Besnerais, J. Navaza, G. Demoment, “Aperture synthesis using maximum entropy on the mean in radio-astronomy,” GRETSI, 1991.

    Google Scholar 

  23. V. A. Macaulay and B. Buck, “Linear inversion by the method of maximum entropy,” Maximum-Entropy and Bayesian Methods in Inverse Problems, C.R. Smith T. Grandy, Jr. (eds.), 1985.

    Google Scholar 

  24. A. Mohammad-Djafari et G. Demoment, “Maximum entropy Fourier synthesis with application to diffraction tomography,” Applied Optics, Vol.26, No. 10,pp: 1745–1754, (1987).

    Google Scholar 

  25. A. Mohammad-Djafari et G. Demoment, “Tomographie de diffraction et synthèse de Fourier à maximum d’entropie,” Revue Phys. Appl., Vol. 22,pp: 153–167, (1987).

    Google Scholar 

  26. A. Mohammad-Djafari and G. Demoment, “Maximum entropy reconstruction in X ray and diffraction tomography,” IEEE Trans. on Medical Imaging, Vol. 7, No. 4 pp: 345–354, (1988).

    Google Scholar 

  27. A. Mohammad—Djafari and G. Demoment, “Image restoration and reconstruction using entropy as a regularization functional,” Maximum Entropy and Bayesian Methods in Science and Engineering,Vol. 2, pp:341–355, (1988) by Kluwer Academic Publishers.

    Google Scholar 

  28. A. Mohammad—Djafari et G. Demoment, “Utilisation de l’entropie dans les problèmes de restauration et de reconstruction d’images,” Traitement du Signal, Vol. 5, No. 4,pp: 235–248, (1988).

    Google Scholar 

  29. A. Mohammad—Djafari et G. Demoment, “Maximum Entropy and Bayesian Approach in Tomographic Image Reconstruction and Restoration,” Maximum Entropy and Bayesian Methods, pp:195–201, J. Skilling (ed.) by Kluwer Academic Publishers, 1989.

    Google Scholar 

  30. A. Mohammad—Djafari et G. Demoment, “Estimating Priors in Maximum Entropy Image Processing,” Proc. of ICASSP 1990,pp: 2069–2072

    Google Scholar 

  31. A. Mohammad—Djafari et J. Idier, “Maximum entropy prior laws of images and estimation of their parameters,” Proc. of The 10th Int. MaxEnt Workshop, Laramie, Wyoming, publié dans Maximum-entropy and Bayesian methods, T.W. Grandy ed., 1990.

    Google Scholar 

  32. A. Mohammad—Djafari, “Maximum Likelihood Estimation of the Lagrange Parameters of the Maximum Entropy Distributions,” Proc. of The 11th Int. MaxEnt Workshop, Seattle, USA, 1991.

    Google Scholar 

  33. A. Mohammad—Djafari, “A Matlab Program to Calculate the Maximum Entropy Distributions,” Proc. of The 11th Int. MaxEnt Workshop, Seattle, U:’A, 1991.

    Google Scholar 

  34. A. Mohammad—Djafari, “Maximum entropy Methods and Algorithmes for solving linear inverse problems,” Rapport interne LSS No. GPI/02/91.

    Google Scholar 

  35. D. Mukherjee and D. C. Hurst, “Maximum Entropy Revisited,” Statistica Neerlandica38, 1984.

    Google Scholar 

  36. R. Narayan and R. Nityananda “Maximum entropy image restoration in astronomy,” Ann. Rev. Astron. Astrophys., 24, pp. 127–170, 1986.

    Article  MathSciNet  Google Scholar 

  37. J. Navaza, “On the maximum entropy estimate of electron density function,” Acta Cryst.A-41, pp. 232–244, 1985.

    Google Scholar 

  38. J. P. Noonan, N. S. Tzannes and T. Costello, “On the Inverse Problem of Entropy Maximizations.,” IEEE Trans. Information Theory„ pp. 120–123, 1976.

    Google Scholar 

  39. J. E. Shore and R. W. Johnson, “Properties of Cross—Entropy Minimization,” IEEE Trans.IT-27, pp. 472–482, 1981.

    Google Scholar 

  40. S. Sibisi, “Two—dimensional reconstructions from one—dimensional data by maximum entropy,” Nature, 301, pp. 134–136, 1983.

    Article  Google Scholar 

  41. J. Skilling and R. K. Bryan, “Maximum entropy image reconstruction: general algorithm,” Mon. Not. R. astr. Soc.211, pp. 111–124, 1984.

    Google Scholar 

  42. J. Skilling and S. F. Gull, “Maximum entropy method in image processing,” IEE Proceedings131, pp. 646–659, 1984.

    Google Scholar 

  43. J. Skilling and S. F. Gull, “Algorithms and Applications,” Maximum-Entropy and Bayesian Methods in Inverse Problems, C.R. Smith T. Grandy, Jr. (eds.), pp. 83–132, 1985.

    Google Scholar 

  44. J. Skilling and S. F. Gull, “The Entropy of an Image,” SIAM-AMS Proceedings, 14, 1984.

    Google Scholar 

  45. C. B. Smith, “A dual method for maximum entropy restoration,” IEEE Trans. Pattern Anal. Machine IntelligencePAMI-1, pp. 411–414, 1979.

    Google Scholar 

  46. H. J. Trussell, “The Relationship Between Image Restoration by the Maximum A Posteriori Method and a Maximum Entropy Method,” IEEE Trans., ASSP-28, pp. 114–117, 1980.

    Google Scholar 

  47. S. J. Wernecke and L. R. D’Addario, “Maximum Entropy Image Reconstruction,” IEEE Trans.C-26, pp. 351–364, 1977.

    Google Scholar 

  48. A. Wehrl, “General properties of entropy,” Reviews of Modern Physics, Vol. 50 No. 2, April 1978.

    Google Scholar 

  49. X. Zhuang, K. B. Yu and R. M. Haralick, “A Differential Equation Approach to Maximum Entropy Image Reconstruction,” IEEE Trans. on Acoustics, Speech, and Signal Processing, Vol. ASSP-35, No. 2, pp. 208–218, 1987.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Laboratoire des Signaux et Systèmes (CNRS-ESE-UPS), École Supérieure d’Électricité, Plateau de Moulon, 91192, Gif-sur-Yvette Cédex, France

    Ali Mohammad-Djafari

Authors
  1. Ali Mohammad-Djafari
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Groupe Problèmes Inverses, Laboratoire des Signaux et Systèmes (CNRS-ESE-UPS), 91192, Gif-sur-Yvette Cedex, France

    Ali Mohammad-Djafari  & Guy Demoment  & 

Rights and permissions

Reprints and permissions

Copyright information

© 1993 Springer Science+Business Media Dordrecht

About this chapter

Cite this chapter

Mohammad-Djafari, A. (1993). Maximum Entropy and Linear Inverse Problems. A Short Review. In: Mohammad-Djafari, A., Demoment, G. (eds) Maximum Entropy and Bayesian Methods. Fundamental Theories of Physics, vol 53. Springer, Dordrecht. https://doi.org/10.1007/978-94-017-2217-9_31

Download citation

  • DOI: https://doi.org/10.1007/978-94-017-2217-9_31

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-90-481-4272-9

  • Online ISBN: 978-94-017-2217-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation