Abstract
This chapter presents a review of the classical information theory which plays a crucial role in this thesis. We introduce the various types of informational measures such as Shannon entropy, the relative entropy, the mutual information and the transfer entropy. We also briefly discuss the noisy-channel coding theorem which represents the meaning of informational measures in the artificial information transmission over a communication channel.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
C.E. Shannon, A mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423 (1948)
T. Schreiber, Measuring information transfer. Phys. Rev. Lett. 85, 461 (2000)
T.M. Cover, J.A. Thomas, Elements of Information Theory (Wiley, New York, 1991)
R.V.L. Hartley, Transmission of information. Bell Syst. Tech. J. 7, 535–563 (1928)
J. Massey, Causality, feedback and directed information. in Proceedings of International Symposium on Information Theory and Its Applications, pp. 303–305 (1990)
K. Kaneko, Lyapunov analysis and information flow in coupled map lattices. Physica D 23, 436 (1986)
J.A. Vanstano, H.L. Swinney, Information transport in spatiotemporal systems. Phys. Rev. Lett. 60, 1733 (1988)
L. Barnett, A.B. Barrett, A.K. Seth, Granger causality and transfer entropy are equivalent for Gaussian variables. Phys. Rev. Lett. 103, 238701 (2009)
C.W.J. Granger, Investigating causal relations by econometric and cross-spectral methods. Econometrica 37, 424438 (1969)
A. Hlavackova-Schindler, M. Palus, M. Vejmelka, Causality detection based on information-theoretic approaches in time series analysis. Phys. Rep. 441, 1–46 (2007)
M. Staniek, K. Lehnertz, Symbolic transfer entropy. Phys. Rev. Lett. 100, 158101 (2008)
L. Barnett, J.T. Lizier, M. Harre, A.K. Seth, T. Bossomaier, Information flow in a kinetic Ising model peaks in the disordered phase. Phys. Rev. Lett. 111, 177203 (2013)
A.E. Allahverdyan, J. Dominik, M. Guenter, Thermodynamic efficiency of information and heat flow. J. Stat. Mech. P09011 (2009)
M. Prokopenko, J.T. Lizier, D.C. Price, On thermodynamic interpretation of transfer entropy. Entropy 15, 524 (2013)
N. Ay, D. Polani, Information flows in causal networks. Adv. Complex Syst. 11, 17 (2008)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2016 Springer Science+Business Media Singapore
About this chapter
Cite this chapter
Ito, S. (2016). Review of Classical Information Theory. In: Information Thermodynamics on Causal Networks and its Application to Biochemical Signal Transduction. Springer Theses. Springer, Singapore. https://doi.org/10.1007/978-981-10-1664-6_2
Download citation
DOI: https://doi.org/10.1007/978-981-10-1664-6_2
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-1662-2
Online ISBN: 978-981-10-1664-6
eBook Packages: Physics and AstronomyPhysics and Astronomy (R0)