Journal of Statistical Physics

, Volume 170, Issue 4, pp 731–747 | Cite as

Exact Maximum-Entropy Estimation with Feynman Diagrams

  • Amitai Netser Zernik
  • Tomer M. Schlank
  • Ran J. Tessler


A longstanding open problem in statistics is finding an explicit expression for the probability measure which maximizes entropy with respect to given constraints. In this paper a solution to this problem is found, using perturbative Feynman calculus. The explicit expression is given as a sum over weighted trees.


Feynman calculus Maximum entropy Perturbative expansion Weighted trees 



We thank O. Bozo, B. Gomberg, R.S. Melzer, A. Moscovitch-Eiger, R. Schweiger, A. Solomon and D. Zernik for discussions related to the work presented here. R.T. was partially supported by Dr. Max Rössler, the Walter Haefner Foundation and the ETH Zurich Foundation.

Compliance with Ethical Standards

Conflict of interest

The authors declare no conflict of interest.


  1. 1.
    Avellaneda, M., Friedman, C., Holmes, R., Samperi, D.: Calibrating volatility surfaces via relative-entropy minimization. App. Math. Financ. 4(1), 37–64 (1997)CrossRefMATHGoogle Scholar
  2. 2.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, New York (2012)MATHGoogle Scholar
  3. 3.
    Etingof, P.: Geometry and quantum field theory. MIT OpenCourseware 18.238 (2002)Google Scholar
  4. 4.
    Frisch, H.L., Lebowitz, J.L.: The Equilibrium Theory of Classical Fluids. Benjamin, New York (1964)MATHGoogle Scholar
  5. 5.
    Jaynes, E.T.: Information theory and statistical mechanics. Phys. Rev. 106(4), 620–630 (1957)ADSMathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Jaynes, E.T.: Information theory and statistical mechanics. II. Phys. Rev. 108(2), 171–190 (1957)ADSMathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Lin, H.W., Tegmark, M., Rolnick. D.: Why does deep and cheap learning work so well? arXiv preprint. arXiv:1608.08225 (2016)
  8. 8.
    Shell, S.M.: The relative entropy is fundamental to multiscale and inverse thermodynamic problems. J. Chem. Phys. 129(14), 144108 (2008)ADSCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Amitai Netser Zernik
    • 1
  • Tomer M. Schlank
    • 2
  • Ran J. Tessler
    • 3
  1. 1.Institute for Advanced StudyPrincetonUSA
  2. 2.Department of MathematicsThe Hebrew University of JerusalemJerusalemIsrael
  3. 3.Institute for Theoretical StudiesETH ZürichZurichSwitzerland

Personalised recommendations