Foundations of Probability

Abstract

This chapter introduces probability theory as a system of models, based on measure theory, of some real-world phenomena. The models are measure spaces of total measure 1 and usually have certain distinguished measurable functions defined on them.

Section 1 begins by establishing the measure-theoretic framework and a short dictionary for passing back and forth between terminology in measure theory and terminology in probability theory. The latter terminology includes events, random variables, expectation, distribution of a random variable, and joint distribution of several random variables. An important feature of probability is that it is possible to work with random variables without any explicit knowledge of the underlying measure space, the joint distributions of random variables being the objects of importance.

Section 2 introduces conditional probability and uses that to motivate the mathematical definition of independence of events. In turn, independence of events leads naturally to a definition of independent random variables. Independent random variables are of great importance in the subject and play a much larger role than their counterparts in abstract measure theory.

Section 3 states and proves the Kolmogorov Extension Theorem, a foundational result allowing one to create stochastic processes involving infinite sets of times out of data corresponding to finite subsets of those times. A special case of the theorem provides the existence of infinite sets of independent random variables with specified distributions.

Section 4 establishes the celebrated Strong Law of Large Numbers, which says that the Cesàro sums of a sequence of identically distributed independent random variables with finite expectation converge almost everywhere to the expectation. This is a theorem that is vaguely known to the general public and is widely misunderstood. The proof is based on Kolmogorov’s inequality.