Theory of Probability

  • Walter T. GrandyJr.
Part of the Fundamental Theories of Physics book series (FTPH, volume 19)


The traditional thought that probability theory has its origins in gambling and games of chance seems well established, although many of the general ideas can already be found in the Old Testament (e.g., Sheynin, 1974). According to Todhunter (1949), one can find reference to probabilities for different throws of the dice in contemporary comments on Dante’s Divine Comedy, and Cardano apparently had calculated numerical values of probabilities in the mid-sixteenth century (e.g., Wilks, 1961). It is generally conceded, however, that Pascal originated the first systematic studies of probabilities about a century later, at the instigation of the Chevalier de Méré, a reputed gambler. Several questions were posed to Pascal, and he corresponded at some length with Fermat regarding the solutions to these and related problems (e.g., Ore, 1960). From the beginning it is clear that the theory of probability was related to pragmatic matters of some importance to those concerned.


Prior Probability Maximum Entropy Probability Assignment Rational Belief Probable Inference 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Abel, N.H.: 1826, ‘Untersuchung der Funktionen zweier abhängig veränderlicher Größen x und y, wie f(x,y) welche die Eigenschaft haben daß f(z,f(x,y)) eine symmetrische Funktion von z,x und y ist’, Crelle’s Journal 1, 11.zbMATHCrossRefGoogle Scholar
  2. Aczel, J.: 1966, Lectures on Functional Equations and Their Applications, Academic Press, New York, Sec.6.2.zbMATHGoogle Scholar
  3. Aristotle: Rhetoric, Book 1, Chap.2, 1357a.Google Scholar
  4. Bayes, T.: 1763, ‘An Essay Towards Solving a Problem in the Doctrine of Chances’, Phil. Trans. Roy. Soc. (London) 53, 370. [Reprinted with a biographical note in G.A. Barnard: 1958, Biometrika 45, 293.].CrossRefGoogle Scholar
  5. Beckenstein, J.D.: 1975, ‘Statistical Blackhole Thermodynamics’, Phys. Rev. D 12, 3077.ADSGoogle Scholar
  6. Bernoulli, J.: 1713, Ars Conjectandi. Google Scholar
  7. Boltzmann, L.: 1895, ‘On Certain Questions of the Theory of Gases’,Nature 51, 413.ADSCrossRefGoogle Scholar
  8. Boole, G.: 1854, An Investigation of the Laws of Thought, Walton, London.Google Scholar
  9. Bricogne, G.: 1982, ‘Generalized Density-Modification Methods’, in D. Sayre (ed.),Computational Crystallography, Oxford Univ. Press, New York, p.258.Google Scholar
  10. Bryan, R.K., M. Bansal, W. Folkhard, C. Nave, and D.A. Marvin: 1983, ‘Maximum-Entropy Calculation of the Electron Density at 4 Å Resolution of Pfl Filamentous Bacteriophage’, Proc. Natl. Acad. Sei. (U.S.A.) 80, 4728.ADSCrossRefGoogle Scholar
  11. Collins, D.M.: 1982, ‘Electron Density Images from Imperfect Data by Iterative Entropy Maximization’, Nature 298, 49.ADSCrossRefGoogle Scholar
  12. Cox, R.T: 1946, ‘Probability, Frequency and Reasonable Expectation’,Am. J. Phys. 14, 7.ADSGoogle Scholar
  13. Cox, R.T.: 1961, The Algebra of Probable Inference, Johns Hopkins Univ. Press, Baltimore.zbMATHGoogle Scholar
  14. Cozzolino, J., and M. Zahner: 1973, ‘The Maximum Entropy Distribution of the Future Market Price of a Stock’, Operations Research 21, 1200.MathSciNetzbMATHCrossRefGoogle Scholar
  15. Currie, R.G.: 1980, ‘Detection of the 11-Year Sunspot Cycle Signal in Earth Rotation’, Geophys. J. Roy. Astron. Soc. 46, 513.MathSciNetCrossRefGoogle Scholar
  16. David, F.N.: 1965, ‘Some Notes on Laplace’, in J. Neyman and L.M. LeCam (eds.),, Springer-Verlag, Heidelberg.Google Scholar
  17. de Finetti, B.: 1937, ‘Prevision: ses lois logiques, ses Sources Subjectives’,Ann. Inst. Henri Poincaré 7, 1.[English translation, H.E. Kyburg, Jr.: 1980, ‘Foresight: Its Logical Laws, Its Subjective Sources’, in H.E. Kyburg, Jr. and H.E. Smokier (eds.), Studies in Subjective Probability, Wiley, New York.].zbMATHGoogle Scholar
  18. Deutsch, D.: 1983, ‘Uncertainty in Quantum Measurements’, Phys. Rev. Letters 50, 631.MathSciNetADSCrossRefGoogle Scholar
  19. Ellis, R.L.: 1843, ‘On the Foundations of the Theory of Probabilities’,Trans. Camb. Phil. Soc. 8, 1.CrossRefGoogle Scholar
  20. Ellis, R.L.: 1854, ‘Remarks on the Fundamental Principles of the Theory of Probabilities’, Trans. Camb. Phil. Soc. 9, 1.Google Scholar
  21. Feller, W.: 1957, An Introduction to Probability Theory and Its Applications, Vol.1, 2nd ed., Wiley, New York, p.3.zbMATHGoogle Scholar
  22. Fisher, R.A.: 1932, Statistical Methods for Research Workers, Hafner, New York.zbMATHGoogle Scholar
  23. Friedman, K., and A. Shimony: 1971, ‘Jaynes’s Maximum Entropy Prescription and Probability Theory’, J.Stat. Phys. 3, 381.ADSCrossRefGoogle Scholar
  24. Gardner, M.: 1970, ‘Mathematical Games’, Scientifìc American 223, 110 (December).Google Scholar
  25. Hobson, A.: 1971, Concepts in Statistical Mechanics, Gordon and Breach, New York.Google Scholar
  26. Jaynes, E.T.: 1957, ‘Information Theory and Statistical Mechanics’,Phys. Rev. 106, 620.MathSciNetADSCrossRefGoogle Scholar
  27. Jaynes, E.T.: 1963, ‘Information Theory and Statistical Mechanics’, in K.W. Ford (ed.), Statistical Physics, Benjamin, New York, p. 181.Google Scholar
  28. Jaynes, E.T.: 1968, ‘Prior Probabilities’, IEEE Trans. SSC-4, 227.Google Scholar
  29. Jaynes, E.T.: 1978, ‘Where Do We Stand on Maximum Entropy?’, in R.D. Levine and M. Tribus (eds.), The Maximum Entropy Formalism, M.I.T. Press, Cambridge, MA.Google Scholar
  30. Jaynes, E.T.: 1980, ‘Marginalization and Prior Probabilities’, in A. Zellner (ed.), Bayesian Analysis in Econometrics and Statistics, North-Holland, Amsterdam.Google Scholar
  31. Jaynes, E.T.: 1982, ‘On the Rationale of Maximum-Entropy Methods’,Proc. IEEE 70, 939.ADSCrossRefGoogle Scholar
  32. Jaynes, E.T.: 1984, ‘The Intuitive Inadequacy of Classical Statistics’,Epistemologica 7, 43. (Special issue on probability, statistics, and inductive logic.).Google Scholar
  33. Jaynes, E.T.: 1985a, ‘Some Random Observations’, Synthese 63, 115.MathSciNetCrossRefGoogle Scholar
  34. Jaynes, E.T.: 1985b, ‘Entropy and Search Theory’, in C.R. Smith and W.T. Grandy, Jr. (eds.), Maximum-Entropy and Bayesian Methods in Inverse Problems, Reidel, Dordrecht.Google Scholar
  35. Jeffreys, H.: 1939, Theory of Probability, Oxford Univ. Press, Oxford.Google Scholar
  36. Keynes, J.M.: 1921, A Treatise on Probability, Macmillan, London.zbMATHGoogle Scholar
  37. Kullback, S.: 1959, Information Theory and Statistics, Wiley, New York.zbMATHGoogle Scholar
  38. Landsberg, P.T.: 1984, ‘Is Equilibrium Always an Entropy Maximum?’, J. Stat. Phys. 35, 159.MathSciNetADSCrossRefGoogle Scholar
  39. Landsberg, P.T., and D. Tranah: 1980, ‘Entropies Need Not Be Concave’, Phys. Letters 78A, 219.MathSciNetADSGoogle Scholar
  40. Laplace, P.S.: 1820, Théorie analytique des Probabilités, 3rd ed.,, Paris first published in 1812.Google Scholar
  41. Laplace, P.S.: 1825, Essai philosophique sur les Probabilités, 5th ed., Bachelier, Paris [English translation, F.W. Truscott and F.L. Emory: 1952, A Philosophical Essay on Probabilities, 2nd ed., Dover, New York.].Google Scholar
  42. Livesey, A.K., and J. Skilling: 1985, ‘Maximum Entropy Theory’, Acta Cryst. A41 Pt.2, 113.Google Scholar
  43. Mead, L.R., and N. Papanicolaou: 1984, ‘Maximum Entropy in the Problem of Moments’, J.Math. Phys. 25, 2404.MathSciNetADSCrossRefGoogle Scholar
  44. Ore, O.: 1960, ‘Pascal and the Invention of Probability Theory’,Am. Math. Monthly 67, 409.MathSciNetzbMATHCrossRefGoogle Scholar
  45. Partovi, M.H.: 1983, ‘Entropie Formulation of Uncertainty for Quantum Measurements’,Phys. Rev. Letters 50, 1883.MathSciNetADSCrossRefGoogle Scholar
  46. Rietsch, E.: 1977, ‘The Maximum Entropy Approach to Inverse Problems’, J. Geophys. 42, 489.Google Scholar
  47. Rosen, J.M.: 1984, ‘A Statistical Description of Coagulation’, J. Colloid Interface Sci. 19, 9.CrossRefGoogle Scholar
  48. Rothstein, J.: 1978, ‘Generalized Entropy, Boundary Conditions, and Biology’, in R.D. Levine and M. Tribus (eds.), The Maximum Entropy Formalism, M.I.T. Press, Cambridge, MA.Google Scholar
  49. Schrödinger, E.: 1960, Statistical Thermodynamics, Cambridge Univ. Press, Cambridge.Google Scholar
  50. Shannon, C.E.: 1948, ‘A Mathematical Theory of Communication’,Bell System Tech. J. 27, 379, 623.MathSciNetGoogle Scholar
  51. Sheynin, O.B.: 1974, ‘On the Prehistory of the Theory of Probability’,Arch. Hist. Exact Sei. 12, 97.MathSciNetzbMATHCrossRefGoogle Scholar
  52. Sheynin, O.B.: 1976, ‘P.S. Laplace’s Work on Probability’, Arch. Hist. Exact Sei. 16, 137.MathSciNetzbMATHCrossRefGoogle Scholar
  53. Shore, J.E., and R.W. Johnson: 1980, ‘Axiomatic Derivation of the Principle of Maximum Entropy and the Principle of Minimum Cross-Entropy’, IEEE Trans. Inf. Th. IT-26, 26.MathSciNetADSCrossRefGoogle Scholar
  54. Smith, C.R., and W.T. Grandy, Jr. (eds.): 1985, Maximum-Entropy and Bayesian Methods in Inverse Problems, Reidel, Dordrecht.zbMATHGoogle Scholar
  55. Tikoshinsky, Y., N.Z. Tishby, and R.D. Levine: 1984, ‘Consistent Inference of Probabilities for Reproducible Experiments’, Phys. Rev. Letters 52, 1357.ADSCrossRefGoogle Scholar
  56. Todhunter, I.: 1949, A History of the Mathematical Theory of Probability, Chelsea, New York (first published in 1865.).Google Scholar
  57. Van Campenhout, J.M., and T.M. Cover: 1981, ‘Maximum Entropy and Conditional Probability’, IEEE Trans. Inf. Th. IT-27, 483.Google Scholar
  58. Venn, J.: 1888, The Logic of Chance, 3rd ed., Macmillan, London.Google Scholar
  59. von Mises, R.: 1957, Probability, Statistics and Truth, 2nd ed., Macmillan, New York.zbMATHGoogle Scholar
  60. Wilkins, S.W.: 1983a, ‘Statistical Geometry.II. Numerical Solution via the Single Pixel Equation’, Acta Cryst. A39, 892.MathSciNetGoogle Scholar
  61. Wilkins, S.W.: 1983b, ‘Statistical Geometry.III. Accelerated Convergence Using Contrast Amplifi- cation’, Acta Cryst. A39, 896.MathSciNetGoogle Scholar
  62. Wilkins, S.W., J.N. Varghese, and M.S. Lehmann: 1983, ‘Statistical Geometry.I. A Self-Consistent Approach to the Crystallographic Inversion Problem Based on Information Theory’, Acta Cryst. A39, 47. MathSciNetGoogle Scholar
  63. Wilks, S.S.: 1961, Forward to: G. Cardano, The Book on Games of Chance (translated by S.H. Gould), Holt, Rinehart, and Winston, New York.Google Scholar
  64. Zellner, A.: 1984, Basic Issues in Econometrics, Univ. Chicago Press, Chicago.Google Scholar

Copyright information

© D. Reidel Publishing Company, Dordrecht, Holland 1987

Authors and Affiliations

  • Walter T. GrandyJr.
    • 1
  1. 1.Department of Physics and AstronomyUniversity of WyomingUSA

Personalised recommendations