Abstract
In this chapter, we briefly review the basic facts of the classical information communication processes. The fundamental aspects of information theory according to Shannon (Bell Syst. Tech. J. 27:379–423, 623–656, 1948) are composed of the following concepts: message; entropy describing the amount of information; communication channel, mutual entropy, coding, and capacity of the channel. We will discuss some coding theorems which are important results of the classical information theory.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Arndt, C.: Information Measures. Springer, Berlin (2001)
Ash, R., Information Theory. Wiley, New York (1965)
Billingsley, P.: Ergodic Theory and Information. Wiley, New York (1968)
Breiman, L.: On achieving channel capacity in finite-memory channels. Ill. J. Math. 4, 246–252 (1960)
Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, New York (1991)
Feinstein, A.: Foundations of Information Theory. McGraw-Hill, New York (1958)
Feinstein, A.: On the coding theorem and its converse for finite memory channels. Inf. Control 2, 25–44 (1959)
Gelfand, I.M., Yaglom, A.M.: Calculation of the amount of information about a random function contained in another such function. Am. Math. Soc. Transl. 12, 199–246 (1959)
Guiasu, S.: Information Theory with Applications. McGraw-Hill, New York (1977)
Ihara, S.: Stochastic Process and Entropy. Iwanami, Tokyo (1984) (in Japanese)
Ingarden, R.S.: Simplified axioms for information without probability. Pr. Mat. 9, 273–282 (1965)
Ingarden, R.S., Kossakowski, A., Ohya, M.: Information Dynamics and Open Systems. Kluwer Academic, Dordrecht (1997)
Jumarie, G.: Relative Information. Springer, Berlin (1990)
Khinchin, A.I.: Mathematical Foundations of Information Theory. Dover, New York (1958), English Translation
Kullback, S., Leibler, R.: On information and sufficiency. Ann. Math. Stat. 22, 79–86 (1951)
Kunisawa, K., Umegaki, H.: Progress of Information Theory. Iwanami, Tokyo (1965) (in Japanese)
Kunisawa, K.: Information Theory, vol. I. Kyoritsu, Tokyo (1983) (in Japanese)
McEliece, R.J.: The Theory of Information Theory and Coding. Addison-Wesley, Reading (1977)
McMillan, B.: The basic theorem of information theory. Ann. Math. Stat. 24, 196–219 (1953)
Ohya, M., Watanabe, N.: A new treatment of communication processes with Gaussian channels. Jpn. J. Appl. Math. 3(1), 197–206 (1986)
Parthasarathy, K.R.: On the integral representation of the rate of transmission of a stationary channel. Ill. J. Math. 5, 299–305 (1961)
Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423 (1948), 623–656
Taki, Y., Information Theory, vol. I. Iwanami, Tokyo (1978) (in Japanese)
Umegaki, H.: A functional method on amount of entropy. Kodai Math. Semin. Rep. 15, 162–175 (1963)
Umegaki, H.: General treatment of alphabet message space and integral representation of entropy. Kodai Math. Semin. Rep. 16, 8–26 (1964)
Umegaki, H., Ohya, M.: Probabilistic Entropy. Kyoritsu, Tokyo (1983) (in Japanese)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2011 Springer Science+Business Media B.V.
About this chapter
Cite this chapter
Ohya, M., Volovich, I. (2011). Fundamentals of Classical Information Theory. In: Mathematical Foundations of Quantum Information and Computation and Its Applications to Nano- and Bio-systems. Theoretical and Mathematical Physics. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-0171-7_6
Download citation
DOI: https://doi.org/10.1007/978-94-007-0171-7_6
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-007-0170-0
Online ISBN: 978-94-007-0171-7
eBook Packages: Physics and AstronomyPhysics and Astronomy (R0)