Encyclopedia of Computational Neuroscience

2015 Edition
| Editors: Dieter Jaeger, Ranu Jung

Directed Information Flow and Causality in Neural Systems

Reference work entry
DOI: https://doi.org/10.1007/978-1-4614-6675-8_141

Definition

In the human experience, information typically flows from one place to another. By contrast, the notion of mutual information introduced by Shannon (1948) is perfectly symmetric in its arguments and does not distinguish between “input” and “output.” In this sense, it is perhaps surprising that this very measure of information indeed captures the capacity of any communication channel – though we should recall that the proof of this fundamental fact is not merely a simple consequence of Shannon’s definition.

In spite of Shannon’s strong and fundamental results, it has been tempting to define a notion of directed information. This was first proposed in (Marko 1973) for stationary processes. The more general and useful definition was given in the brief and beautiful note by Massey (1990). Moreover, Massey (1990), Kramer (1998) and subsequent work revealed that directed information has a natural place in the study of information transmission with feedback from the output to the...

This is a preview of subscription content, log in to check access

References

  1. Granger CWJ (1969) Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37(3):424438Google Scholar
  2. Kramer G (1998) Directed information for channels with feedback, vol 11, ETH series in information processing. HartungGorre, KonstanzGoogle Scholar
  3. Marko H (1973) The bidirectional communication theory a generalization of information theory. IEEE Trans Commun 21:1345–1351Google Scholar
  4. Massey JL (1990) Causality, feedback and directed information. In: Proceedings of the 1990 international symposium on information theory and its applications, Hawaii, pp 303–305Google Scholar
  5. Massey JL, Massey PC (2005) Conservation of mutual and directed information. In: Proceedings of the 2005 international symposium on information theory, Adelaide, pp 157–158Google Scholar
  6. Quinn C, Coleman TP, Kiyavash N, Hatsopoulos NG (2011) Estimating the directed information to infer causal relationships in ensemble neural spike train recordings. J Comput Neurosci. doi:10.1007/s10827-010-0247-2PubMedCentralPubMedGoogle Scholar
  7. Schreiber T (2000) Measuring information transfer. Phys Rev Lett 85:461–464PubMedGoogle Scholar
  8. Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27:379–423, and 623–656Google Scholar
  9. So K, Koralek AC, Ganguly K, Gastpar MC, Carmena JM (2012) Assessing functional connectivity of neural ensembles using directed information. J Neural Eng 9:026004PubMedGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.University of CaliforniaBerkeleyUSA
  2. 2.Ecole Polytechnique Fédérale de Lausanne (EPFL)LausanneSwitzerland