Source Coding

Abstract

Let (X¹, Y ¹),…, (Xⁿ, Yⁿ) be n independent chance variables with the common distribution given by

$$ P\left\{ {{X^1} = i,{Y^1} = j} \right\} = {\pi _i}w\left( {j\left| i \right.} \right),i = 1,...,\;a;j = 1,...,b $$

((12.1.1))

. As usual, the distribution of X¹ will be written as π, that of Y¹ as π’. We will write their joint distribution for short as πw and also as #x03C0;’ w’, where the definition of w’ is obvious (see also (11.2.5)). Define X _n = (X¹,…, Xⁿ) and Y _n = (Y¹,…, Yⁿ). The spaces on which they are defined will be denoted by A ^* _n and B ^* _n , respectively, as in Chapter 11. The statement that a certain subset of a space of sequences, on which a given probability distribution has been defined, “covers” the space, is to mean that the probability of the subset can be made > 1 - λ, with λ > 0 arbitrarily small.