Skip to main content
SpringerLink
Log in
Book cover

Directed Information Measures in Neuroscience pp 3–36Cite as

Transfer Entropy in Neuroscience

  • Chapter

Part of the book series: Understanding Complex Systems ((UCS))

Abstract

Information transfer is a key component of information processing, next to information storage and modification. Information transfer can be measured by a variety of directed informationmeasures of which transfer entropy is themost popular, andmost principled one. This chapter presents the basic concepts behind transfer entropy in an intuitive fashion, including graphical depictions of the key concepts. It also includes a special section devoted to the correct interpretation of the measure, especially with respect to concepts of causality. The chapter also provides an overview of estimation techniques for transfer entropy and pointers to popular open source toolboxes. It also introduces recent extensions of transfer entropy that serve to estimate delays involved in information transfer in a network. By touching upon alternative measures of information transfer, such as Massey’s directed information transfer and Runge’s momentary information transfer, it may serve as a frame of reference for more specialised treatments and as an overview over the field of studies in information transfer in general.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Amblard, P.O., Michel, O.J.J.: On directed information theory and granger causality graphs. J. Comput. Neurosci. 30(1), 7–16 (2011)

    Article  MathSciNet  Google Scholar 

  2. Amblard, P.O., Michel, O.J.J.: The relation between granger causality and directed information theory: A review. Entropy 15(1), 113–143 (2012)

    Article  MathSciNet  Google Scholar 

  3. Ay, N., Polani, D.: Information flows in causal networks. Adv. Complex Syst. 11, 17 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  4. Barnett, L., Barrett, A.B., Seth, A.K.: Granger causality and transfer entropy are equivalent for Gaussian variables. Phys. Rev. Lett. 103(23), 238–701 (2009)

    Article  Google Scholar 

  5. Battaglia, D., Witt, A., Wolf, F., Geisel, T.: Dynamic effective connectivity of inter-areal brain circuits. PLoS Comput. Biol. 8(3), e1002438 (2012)

    Google Scholar 

  6. Besserve, M., Schlkopf, B., Logothetis, N.K., Panzeri, S.: Causal relationships between frequency bands of extracellular signals in visual cortex revealed by an information theoretic analysis. J. Comput. Neurosci. 29(3), 547–566 (2010)

    Article  Google Scholar 

  7. Bühlmann, A., Deco, G.: Optimal information transfer in the cortex through synchronization. PLoS Comput. Biol. 6(9), e1000934 (2010)

    Google Scholar 

  8. Chávez, M., Martinerie, J., Le Van Quyen, M.: Statistical assessment of nonlinear causality: application to epileptic EEG signals. J. Neurosci. Methods 124(2), 113–128 (2003)

    Article  Google Scholar 

  9. Chicharro, D., Ledberg, A.: When two become one: the limits of causality analysis of brain dynamics. PLoS One 7(3), e32466 (2012)

    Google Scholar 

  10. Cover, T.M., Thomas, J.A.: Elements of information theory. Wiley-Interscience, New York (1991)

    Book  MATH  Google Scholar 

  11. Faes, L., Nollo, G.: Bivariate nonlinear prediction to quantify the strength of complex dynamical interactions in short-term cardiovascular variability. Med. Biol. Eng. Comput. 44(5), 383–392 (2006)

    Article  Google Scholar 

  12. Faes, L., Nollo, G., Porta, A.: Information-based detection of nonlinear granger causality in multivariate processes via a nonuniform embedding technique. Phys. Rev. E Stat. Nonlin. Soft. Matter Phys. 83(5 Pt. 1), 051112 (2011)

    Google Scholar 

  13. Faes, L., Nollo, G., Porta, A.: Non-uniform multivariate embedding to assess the information transfer in cardiovascular and cardiorespiratory variability series. Comput. Biol. Med. 42(3), 290–297 (2012)

    Article  Google Scholar 

  14. Faes, L., Nollo, G., Porta, A.: Compensated transfer entropy as a tool for reliably estimating information transfer in physiological time series. Entropy 15(1), 198–219 (2013)

    Article  MathSciNet  Google Scholar 

  15. Felts, P.A., Baker, T.A., Smith, K.J.: Conduction in segmentally demyelinated mammalian central axons. J. Neurosci. 17(19), 7267–7277 (1997)

    Google Scholar 

  16. Freiwald, W.A., Valdes, P., Bosch, J., Biscay, R., Jimenez, J.C., Rodriguez, L.M., Rodriguez, V., Kreiter, A.K., Singer, W.: Testing non-linearity and directedness of interactions between neural groups in the macaque inferotemporal cortex. J. Neurosci. Methods 94(1), 105–119 (1999)

    Article  Google Scholar 

  17. Garofalo, M., Nieus, T., Massobrio, P., Martinoia, S.: Evaluation of the performance of information theory-based methods and cross-correlation to estimate the functional connectivity in cortical networks. PLoS One 4(8), e6482 (2009)

    Google Scholar 

  18. Gomez-Herrero, G., Wu, W., Rutanen, K., Soriano, M.C., Pipa, G., Vicente, R.: Assessing coupling dynamics from an ensemble of time series. arXiv preprint arXiv:1008.0539 (2010)

    Google Scholar 

  19. Gourevitch, B., Eggermont, J.J.: Evaluating information transfer between auditory cortical neurons. J. Neurophysiol. 97(3), 2533–2543 (2007)

    Article  Google Scholar 

  20. Gray, C.M., Knig, P., Engel, A.K., Singer, W.: Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature 338(6213), 334–337 (1989)

    Article  Google Scholar 

  21. Griffith, V., Koch, C.: Quantifying synergistic mutual information. In: Prokopenko, M. (ed.) Guided Self-Organization: Inception, pp. 159–190. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  22. Hadjipapas, A., Hillebrand, A., Holliday, I.E., Singh, K.D., Barnes, G.R.: Assessing interactions of linear and nonlinear neuronal sources using MEG beamformers: a proof of concept. Clin. Neurophysiol. 116(6), 1300–1313 (2005)

    Article  Google Scholar 

  23. Hahs, D.W., Pethel, S.D.: Distinguishing anticipation from causality: anticipatory bias in the estimation of information flow. Phys. Rev. Lett. 107(12), 128701 (2011)

    Article  Google Scholar 

  24. Hahs, D.W., Pethel, S.D.: Transfer entropy for coupled autoregressive processes. Entropy 15(3), 767–788 (2013)

    Article  MathSciNet  Google Scholar 

  25. Harder, M., Salge, C., Polani, D.: Bivariate measure of redundant information. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 87(1), 012130 (2013)

    Google Scholar 

  26. Hebb, D.O.: The organization of behavior: A neuropsychological theory. Wiley, New York (1949)

    Google Scholar 

  27. Ito, S., Hansen, M.E., Heiland, R., Lumsdaine, A., Litke, A.M., Beggs, J.M.: Extending transfer entropy improves identification of effective connectivity in a spiking cortical network model. PLoS One 6(11), e27431 (2011)

    Google Scholar 

  28. Kaiser, A., Schreiber, T.: Information transfer in continuous processes. Physica D 166, 43 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  29. Kim, J., Kim, G., An, S., Kwon, Y.K., Yoon, S.: Entropy-based analysis and bioinformatics-inspired integration of global economic information transfer. PLoS One 8(1), e51986 (2013)

    Google Scholar 

  30. Kozachenko, L., Leonenko, N.: Sample estimate of entropy of a random vector. Probl. Inform. Transm. 23, 95–100 (1987)

    MATH  MathSciNet  Google Scholar 

  31. Kraskov, A., Stoegbauer, H., Grassberger, P.: Estimating mutual information. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 69(6 Pt. 2), 066138 (2004)

    Google Scholar 

  32. Kwon, O., Yang, J.S.: Information flow between stock indices. EPL (Europhysics Letters) 82(6), 68003 (2008)

    Article  Google Scholar 

  33. Lapidoth, A., Pete, G.: On the entropy of the sum and of the difference of independent random variables. In: IEEE 25th Convention of Electrical and Electronics Engineers in Israel, IEEEI 2008, pp. 623–625. IEEE (2008)

    Google Scholar 

  34. Leistritz, L., Hesse, W., Arnold, M., Witte, H.: Development of interaction measures based on adaptive non-linear time series analysis of biomedical signals. Biomed. Tech (Berl.) 51(2), 64–69 (2006)

    Article  Google Scholar 

  35. Li, X., Ouyang, G.: Estimating coupling direction between neuronal populations with permutation conditional mutual information. NeuroImage 52(2), 497–507 (2010)

    Article  Google Scholar 

  36. Lindner, M., Vicente, R., Priesemann, V., Wibral, M.: Trentool: A Matlab open source toolbox to analyse information flow in time series data with transfer entropy. BMC Neurosci. 12(119), 1–22 (2011)

    Google Scholar 

  37. Lizier, J.: The Local Information Dynamics of Distributed Computation in Complex Systems. Springer theses. Springer (2013)

    Google Scholar 

  38. Lizier, J.T., Atay, F.M., Jost, J.: Information storage, loop motifs, and clustered structure in complex networks. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 86(2 Pt. 2), 026110 (2012)

    Google Scholar 

  39. Lizier, J.T., Flecker, B., Williams, P.L.: Towards a synergy-based approach to measuring information modification. In: Proceedings of the 2013 IEEE Symposium on Artificial Life (ALIFE), pp. 43–51. IEEE (2013)

    Google Scholar 

  40. Lizier, J.T., Heinzle, J., Horstmann, A., Haynes, J.D., Prokopenko, M.: Multivariate information-theoretic measures reveal directed information structure and task relevant changes in fMRI connectivity. J. Comput. Neurosci. 30(1), 85–107 (2011)

    Article  MathSciNet  Google Scholar 

  41. Lizier, J.T., Mahoney, J.R.: Moving frames of reference, relativity and invariance in transfer entropy and information dynamics. Entropy 15(1), 177–197 (2013)

    Article  MathSciNet  Google Scholar 

  42. Lizier, J.T., Pritam, S., Prokopenko, M.: Information dynamics in small-world Boolean networks. Artif. Life 17(4), 293–314 (2011)

    Article  Google Scholar 

  43. Lizier, J.T., Prokopenko, M.: Differentiating information transfer and causal effect. Eur. Phys. J. B 73, 605–615 (2010)

    Article  Google Scholar 

  44. Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: Local information transfer as a spatiotemporal filter for complex systems. Phys. Rev. E 77(2 Pt. 2), 026110 (2008)

    Google Scholar 

  45. Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: Information modification and particle collisions in distributed computation. Chaos 20(3), 037109 (2010)

    Google Scholar 

  46. Lizier, J.T., Rubinov, M.: Multivariate construction of effective computational networks from observational data. Max Planck Preprint 25/2012. Max Planck Institute for Mathematics in the Sciences (2012)

    Google Scholar 

  47. Lüdtke, N., Logothetis, N.K., Panzeri, S.: Testing methodologies for the nonlinear analysis of causal relationships in neurovascular coupling. Magn. Reson. Imaging 28(8), 1113–1119 (2010)

    Article  Google Scholar 

  48. Marko, H.: The bidirectional communication theory–a generalization of information theory. IEEE Transactions on Communications 21(12), 1345–1351 (1973)

    Article  Google Scholar 

  49. Marr, D.: Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. Henry Holt and Co. Inc., New York (1982)

    Google Scholar 

  50. Massey, J.: Causality, feedback and directed information. In: Proc. Int. Symp. Information Theory Application (ISITA 1990), pp. 303–305 (1990)

    Google Scholar 

  51. Merkwirth, C., Parlitz, U., Lauterborn, W.: Fast nearest-neighbor searching for nonlinear signal processing. Phys. Rev. E Stat. Phys. Plasmas. Fluids Relat. Interdiscip. Topics 62(2 Pt. A), 2089–2097 (2000)

    Google Scholar 

  52. Neymotin, S.A., Jacobs, K.M., Fenton, A.A., Lytton, W.W.: Synaptic information transfer in computer models of neocortical columns. J. Comput. Neurosci. 30(1), 69–84 (2011)

    Article  MathSciNet  Google Scholar 

  53. Nolte, G., Ziehe, A., Nikulin, V.V., Schlogl, A., Kramer, N., Brismar, T., Muller, K.R.: Robustly estimating the flow direction of information in complex physical systems. Phys. Rev. Lett. 100(23), 234101 (2008)

    Article  Google Scholar 

  54. Oostenveld, R., Fries, P., Maris, E., Schoffelen, J.M.: Fieldtrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput. Intell. Neurosci. 2011, 156869 (2011)

    Article  Google Scholar 

  55. Paluš, M.: Synchronization as adjustment of information rates: detection from bivariate time series. Phys. Rev. E 63, 046211 (2001)

    Google Scholar 

  56. Pearl, J.: Causality: models, reasoning, and inference. Cambridge University Press (2000)

    Google Scholar 

  57. Pompe, B., Runge, J.: Momentary information transfer as a coupling measure of time series. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 83(5 Pt. 1), 051122 (2011)

    Google Scholar 

  58. Ragwitz, M., Kantz, H.: Markov models from data by simple nonlinear time series predictors in delay embedding spaces. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 65(5 Pt. 2), 056201 (2002)

    Google Scholar 

  59. Sabesan, S., Good, L.B., Tsakalis, K.S., Spanias, A., Treiman, D.M., Iasemidis, L.D.: Information flow and application to epileptogenic focus localization from intracranial EEG. IEEE Trans. Neural. Syst. Rehabil. Eng. 17(3), 244–253 (2009)

    Article  Google Scholar 

  60. Schreiber, T.: Measuring information transfer. Phys. Rev. Lett. 85(2), 461–464 (2000)

    Article  Google Scholar 

  61. Small, M., Tse, C.: Optimal embedding parameters: a modelling paradigm. Physica D: Nonlinear Phenomena 194, 283–296 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  62. Staniek, M., Lehnertz, K.: Symbolic transfer entropy: inferring directionality in biosignals. Biomed. Tech (Berl.) 54(6), 323–328 (2009)

    Article  Google Scholar 

  63. Stetter, O., Battaglia, D., Soriano, J., Geisel, T.: Model-free reconstruction of excitatory neuronal connectivity from calcium imaging signals. PLoS Comput. Biol. 8(8), e1002653 (2012)

    Google Scholar 

  64. Sun, L., Grützner, C., Bölte, S., Wibral, M., Tozman, T., Schlitt, S., Poustka, F., Singer, W., Freitag, C.M., Uhlhaas, P.J.: Impaired gamma-band activity during perceptual organization in adults with autism spectrum disorders: evidence for dysfunctional network activity in frontal-posterior cortices. J. Neurosci. 32(28), 9563–9573 (2012)

    Article  Google Scholar 

  65. Takens, F.: Detecting Strange Attractors in Turbulence. In: Dynamical Systems and Turbulence, Warwick. Lecture Notes in Mathematics, vol. 898, pp. 366–381. Springer (1980)

    Google Scholar 

  66. Vakorin, V.A., Kovacevic, N., McIntosh, A.R.: Exploring transient transfer entropy based on a group-wise ica decomposition of EEG data. Neuroimage 49(2), 1593–1600 (2010)

    Article  Google Scholar 

  67. Vakorin, V.A., Krakovska, O.A., McIntosh, A.R.: Confounding effects of indirect connections on causality estimation. J. Neurosci. Methods 184(1), 152–160 (2009)

    Article  Google Scholar 

  68. Vakorin, V.A., Mii, B., Krakovska, O., McIntosh, A.R.: Empirical and theoretical aspects of generation and transfer of information in a neuromagnetic source network. Front Syst. Neurosci. 5, 96 (2011)

    Article  Google Scholar 

  69. Vicente, R., Wibral, M., Lindner, M., Pipa, G.: Transfer entropy – a model-free measure of effective connectivity for the neurosciences. J. Comput. Neurosci. 30(1), 45–67 (2011)

    Article  MathSciNet  Google Scholar 

  70. Victor, J.: Binless strategies for estimation of information from neural data. Phys. Rev. E 72, 051903 (2005)

    Google Scholar 

  71. Whitford, T.J., Ford, J.M., Mathalon, D.H., Kubicki, M., Shenton, M.E.: Schizophrenia, myelination, and delayed corollary discharges: a hypothesis. Schizophr Bull. 38(3), 486–494 (2012)

    Google Scholar 

  72. Wibral, M., Pampu, N., Priesemann, V., Siebenhhner, F., Seiwert, H., Lindner, M., Lizier, J.T., Vicente, R.: Measuring information-transfer delays. PLoS One 8(2), e55809 (2013)

    Google Scholar 

  73. Wibral, M., Rahm, B., Rieder, M., Lindner, M., Vicente, R., Kaiser, J.: Transfer entropy in magnetoencephalographic data: Quantifying information flow in cortical and cerebellar networks. Prog. Biophys. Mol. Biol. 105(1-2), 80–97 (2011)

    Article  Google Scholar 

  74. Wibral, M., Wollstadt, P., Meyer, U., Pampu, N., Priesemann, V., Vicente, R.: Revisiting wiener’s principle of causality – interaction-delay reconstruction using transfer entropy and multivariate analysis on delay-weighted graphs. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2012, 3676–3679 (2012)

    Google Scholar 

  75. Wollstadt, P., Martinéz-Zarzuela, M., Vicente, R., Díaz-Pernas, F., Wibral, M.: Efficient transfer entropy analysis of non-stationary neural time series. arXiv preprint arXiv:1401.4068 (2014)

    Google Scholar 

  76. Wiener, N.: The theory of prediction. In: Beckmann, E.F. (ed.) In Modern Mathematics for the Engineer. McGraw-Hill, New York (1956)

    Google Scholar 

  77. Williams, P.L., Beer, R.D.: Nonnegative decomposition of multivariate information. arXiv preprint arXiv:1004.2515 (2010)

    Google Scholar 

  78. Williams, P.L., Beer, R.D.: Generalized measures of information transfer. arXiv preprint arXiv:1102.1507 (2011)

    Google Scholar 

  79. Wolfram, S.: A new kind of science. Wolfram Media, Champaign (2002)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. MEG Unit, Brain Imaging Center, Goethe University, Heinrich-Hoffmann Strasse 10, 60528, Frankfurt am Main, Germany

    Michael Wibral

  2. Max-Planck Institute for Brain Research, 60385, Frankfurt am Main, Germany

    Raul Vicente

  3. School of Psychology and Clinical Language Science, University of Reading, Reading, UK

    Michael Lindner

Authors
  1. Michael Wibral
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Raul Vicente
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Lindner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Wibral .

Editor information

Editors and Affiliations

  1. Brain Imaging Center, Frankfurt am Main, Germany

    Michael Wibral

  2. Max-Planck Institute for Brain Research, Frankfurt am Main, Germany

    Raul Vicente

  3. CSIRO Computational Informatics, Marsfield, Sydney, Australia

    Joseph T. Lizier

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Wibral, M., Vicente, R., Lindner, M. (2014). Transfer Entropy in Neuroscience. In: Wibral, M., Vicente, R., Lizier, J. (eds) Directed Information Measures in Neuroscience. Understanding Complex Systems. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-54474-3_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-54474-3_1

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-54473-6

  • Online ISBN: 978-3-642-54474-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation