This chapter extends many of the basic properties of relative entropy to sequences of random variables and to processes. Several limiting properties of entropy rates are proved and a mean ergodic theorem for relative entropy densities is given. The principal ergodic theorems for relative entropy and information densities in the general case are given in the next chapter.
KeywordsRelative Entropy Ergodic Theorem Entropy Density Process Distribution Finite Alphabet
Unable to display preview. Download preview PDF.