Abstract
This chapter briefly introduces the necessary concepts from the theory of stochastic processes (see for example Lamperti 1977; Doob 1953) that are needed for a proper definition of information rate and channel capacity, following Shannon.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
\(\mathbf{c} {=\log }_{2}10 +{ \frac{8} {10}\log }_{2}8 +{ \frac{2} {10}\log }_{2}2 {-\log }_{2}10 = 2.6\text{ bit}\).
- 2.
\(\text{ bit-rate} {=\log }_{2}10 + \frac{68} {100}\left ({\frac{16} {17}\log }_{2}\frac{16} {17} +{ \frac{1} {17}\log }_{2} \frac{1} {17}\right ) {=\log }_{2}10 + \frac{68} {100}\left (\frac{64} {17} {-\log }_{2}17\right ) {=\log }_{2}10\ -\ 0.322757 \cdot 0.68 = 3.10245\).
References
Ash, R. B. (1965). Information theory. New York, London, Sidney: Interscience.
Attneave, F. (1959). Applications of information theory to psychology. New York: Holt, Rinehart and Winston.
Billingsley, P. (1978). Ergodic theory and information. Huntington, NY: Robert E. Krieger Publishing Co.
Cover, T. M. & Thomas, J. A. (1991). Elements of information theory. New York: Wiley.
Doob, J. L. (1953). Stochastic processes. New York: Wiley.
Friedman, N. A. (1970). Introduction to ergodic theory. New York: Van Nostrand Reinhold Company.
Gray, R. M. (1990). Entropy and information theory. New York: Springer.
Khinchin, A. (1957). Mathematical foundations of information theory. New York: Dover Publications, Inc.
Lamperti, J. (1966). Probability : A survey of the mathematical theory. Reading, Massachusetts: Benjamin/Cummings.
Lamperti, J. (1977). Stochastic processes - a survey of the mathematical theory. Applied Mathematical Sciences 23, New York: Springer.
Shannon, C. E. (1948). A mathematical theory of communication. Bell Systems Technical Journal, 27, 379–423, 623–656.
Topsøe, F. (1974). Informationstheorie: eine Einführung. Stuttgart: Teubner Verlag.
Walters, P. (1982). An introduction to ergodic theory. New York, Berlin, Heidelberg: Springer.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Palm, G. (2012). Stationary Processes and Their Information Rate. In: Novelty, Information and Surprise. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29075-6_6
Download citation
DOI: https://doi.org/10.1007/978-3-642-29075-6_6
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-29074-9
Online ISBN: 978-3-642-29075-6
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)