A Robust Estimation of Information Flow in Coupled Nonlinear Systems

Part of the Springer Optimization and Its Applications book series (SOIA, volume 38)


Transfer entropy (TE) is a recently proposed measure of the information flow between coupled linear or nonlinear systems. In this study, we first suggest improvements in the selection of parameters for the estimation of TE that significantly enhance its accuracy and robustness in identifying the direction and the level of information flow between observed data series generated by coupled complex systems. Second, a new measure, the net transfer of entropy (NTE), is defined based on TE. Third, we employ surrogate analysis to show the statistical significance of the measures. Fourth, the effect of measurement noise on the measures’ performance is investigated up to \(S/N = 3\) dB. We demonstrate the usefulness of the improved method by analyzing data series from coupled nonlinear chaotic oscillators. Our findings suggest that TE and NTE may play a critical role in elucidating the functional connectivity of complex networks of nonlinear systems.


Autocorrelation Function Information Flow Surrogate Data Transfer Entropy Drive Oscillator 



This work was supported in part by NSF (Grant ECS-0601740) and the Science Foundation of Arizona (Competitive Advantage Award CAA 0281-08).


  1. 1.
    Bharucha-Reid, A. Elements of the Theory of Markov Processes and Their Applications. Courier Dover Publications, Chemsford, MA (1997)Google Scholar
  2. 2.
    Chen, Y., Rangarajan, G., Feng, J., Ding, M. Analyzing multiple nonlinear time series with extended Granger causality. Phys Lett A 324(1), 26–35 (2004)MathSciNetMATHCrossRefGoogle Scholar
  3. 3.
    Eckmann, J., Ruelle, D. Ergodic theory of chaos and strange attractors. In: Ruelle, D. (ed.) Turbulence, Strange Attractors, and Chaos, pp. 365–404. World Scientific, Singapore (1995)CrossRefGoogle Scholar
  4. 4.
    Efron, B., Tibshirani, R. An Introduction to the Bootstrap. CRC Press, Boca Raton (1993)Google Scholar
  5. 5.
    Franaszczuk, P., Bergey, G. Application of the directed transfer function method to mesial and lateral onset temporal lobe seizures. Brain Topogr 11(1), 13–21 (1998)CrossRefGoogle Scholar
  6. 6.
    Friston, K. Brain function, nonlinear coupling, and neuronal transients. Neuroscientist 7(5), 406–418 (2001)CrossRefGoogle Scholar
  7. 7.
    Gaspard, P., Nicolis, G. What can we learn from homoclinic orbits in chaotic dynamics? J Stat Phys 31(3), 499–518 (1983)MathSciNetMATHCrossRefGoogle Scholar
  8. 8.
    Grassberger, P. Finite sample corrections to entropy and dimension estimates. Phys Lett A 128(6–7), 369–373 (1988)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Hlaváčková-Schindler, K., Paluş, M., Vejmelka, M., Bhattacharya, J. Causality detection based on information-theoretic approaches in time series analysis. Phys Rep 441(1), 1–46 (2007)CrossRefGoogle Scholar
  10. 10.
    Iasemidis, L.D., Sackellares, J.C., Savit, R. Quantification of hidden time dependencies in the EEG within the framework of nonlinear dynamics. In: Jansen, B., Brandt, M. (eds.) Nonlinear Dynamical Analysis of the EEG, pp. 30–47. World Scientific, Singapore (1993)Google Scholar
  11. 11.
    Kaiser, A., Schreiber, T. Information transfer in continuous processes. Physica D 166, 43–62 (2002)MathSciNetMATHCrossRefGoogle Scholar
  12. 12.
    Katz, R. On some criteria for estimating the order of a Markov chain. Technometrics 23(3), 243–256 (1981)MathSciNetMATHCrossRefGoogle Scholar
  13. 13.
    Martinerie, J., Albano, A., Mees, A., Rapp, P. Mutual information, strange attractors, and the optimal estimation of dimension. Phys Rev A 45(10), 7058–7064 (1992)CrossRefGoogle Scholar
  14. 14.
    Pawelzik, K., Schuster, H. Generalized dimensions and entropies from a measured time series. Phys Rev A 35(1), 481–484 (1987)CrossRefGoogle Scholar
  15. 15.
    Pereda, E., Quiroga, R.Q., Bhattacharya, J. Nonlinear multivariate analysis of neurophysiological signals. Prog Neurobiol 77(1–2), 1–37 (2005)CrossRefGoogle Scholar
  16. 16.
    Politis, D., Romano, J., Wolf, M. Subsampling, Springer Series in Statistics, Springer Verlag, New York (1999)Google Scholar
  17. 17.
    Quiroga, R.Q., Arnhold, J., Lehnertz, K., Grassberger, P. Kulback-Leibler and renormalized entropies: Applications to electroencephalograms of epilepsy patients. Phys Rev E 62(6), 8380–8386 (2000)CrossRefGoogle Scholar
  18. 18.
    Sabesan, S., Narayanan, K., Prasad, A., Spanias, A. and Iasemidis, L. Improved measure of information flow in coupled nonlinear systems. In: Proceedings of International Association of Science and Technology for Development, pp. 24–26 (2003)Google Scholar
  19. 19.
    Sabesan, S., Narayanan, K., Prasad, A., Tsakalis, K., Spanias, A., Iasemidis, L. Information flow in coupled nonlinear systems: Application to the epileptic human brain. In: Pardalos, P., Boginski, V., Vazacopoulos, A. (eds.) Data Mining in Biomedicine, Springer Optimization and Its Applications Series, Springer, New York, pp. 483–504 (2007)CrossRefGoogle Scholar
  20. 20.
    Schiff, S., So, P., Chang, T., Burke, R., Sauer, T. Detecting dynamical interdependence and generalized synchrony through mutual prediction in a neural ensemble. Phys Rev E 54(6), 6708–6724 (1996)CrossRefGoogle Scholar
  21. 21.
    Schreiber, T. Determination of the noise level of chaotic time series. Phys Rev E 48(1), 13–16 (1993)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Schreiber, T. Measuring information transfer. Phys Rev Lett 85(2), 461–464 (2000)CrossRefGoogle Scholar
  23. 23.
    Theiler, J. Spurious dimension from correlation algorithms applied to limited time-series data. Phys Rev A 34(3), 2427–2432 (1986)CrossRefGoogle Scholar
  24. 24.
    Wiener, N. Modern Mathematics for the Engineers [Z]. Series 1. McGraw-Hill, New York (1956)Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.The Harrington Department of BioengineeringArizona State UniversityTempeUSA
  2. 2.Barrow Neurological InstitutePhoenixUSA
  3. 3.Department of Electrical EngineeringArizona State UniversityTempeUSA
  4. 4.Mayo ClinicPhoenixUSA

Personalised recommendations