Random Variables: Fundamentals of Probability Theory and Statistics

Abstract

A fundamental concept for any statistical treatment is that of the random variable. Thus this concept and various other closely related ideas are presented at the beginning of this book. Section 2.1 will introduce event spaces, probabilities, probability distributions, and density distributions within the framework of the Kolmogorov axioms. The concept of random variables will then be introduced, initially in a simplified form. In Sect. 2.2 these concepts will be extended to multidimensional probability densities and conditional probabilities. This will allow us to define independent random variables and to discuss the Bayes’ theorem. Section 2.3 will deal with characteristic quantities of a probability density, namely, the expectation value, variance, and quantiles. Entropy is also a quantity characterising a probability density, and because of its significance a whole section, Sect. 2.4, is devoted to entropy. In particular, relative entropy, i.e., the entropy of one density relative to another, will be introduced and the maximum entropy principle will be discussed. In Sect. 2.5, the reader will meet the calculus of random variables; the central limit theorem in a first simple version is proven, stressing the importance of the normal random variable; various other important random variables are also presented here.