Information in Language

Abstract

We have already seen from chapter 1 that the idea of probability is absolutely basic in Information Theory. It is necessary to develop some of the important rules and results of probability theory before we can attempt to analyse the information content of a language such as English. In the mathematical sections of this book a fairly intuitive approach will be adopted, based on an understanding of the processes involved, rather than a strictly rigorous mathematical presentation. Mathematics will be viewed as a tool for the efficient representation and analysis of ideas, rather than a study for its own sake. We will avoid formal proofs except where such proofs are instructive or add to our understanding of the subject.