Nonlinear Dynamics

, Volume 68, Issue 3, pp 431–444 | Cite as

Entropy measures for biological signal analyses

  • Jianbo Gao
  • Jing Hu
  • Wen-wen Tung
Original Paper


Entropies are among the most popular and promising complexity measures for biological signal analyses. Various types of entropy measures exist, including Shannon entropy, Kolmogorov entropy, approximate entropy (ApEn), sample entropy (SampEn), multiscale entropy (MSE), and so on. A fundamental question is which entropy should be chosen for a specific biological application. To solve this issue, we focus on scaling laws of different entropy measures and introduce an ensemble forecasting framework to find the connections among them. One critical component of the ensemble forecasting framework is the scale-dependent Lyapunov exponent (SDLE), whose scaling behavior is found to be the richest among all the entropy measures. In fact, SDLE contains all the essential information of other entropy measures, and can act as a unifying multiscale complexity measure. Furthermore, SDLE has a unique scale separation property to aptly deal with nonstationarity and characterize high-dimensional and intermittent chaos. Therefore, SDLE can often be the first choice for exploratory studies in biology. The effectiveness of SDLE and the ensemble forecasting framework is illustrated by considering epileptic seizure detection from EEG.


Approximate entropy Sample entropy Scale-dependent Lyapunov exponent Seizure detection 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Pincus, S.M.: Approximate entropy as a measure of system complexity. Proc. Natl. Acad. Sci. USA 88, 2297–2301 (1991) MathSciNetMATHCrossRefGoogle Scholar
  2. 2.
    Cavanaugh, J.T., Guskiewicz, K.M., Giuliani, C., Marshal, S., Mercer, V., Stergiou, N.: Detecting altered postural-control after cerebral concussion in athletes with normal postural stability. Br. J. Sports Med. 39, 805–811 (2005) CrossRefGoogle Scholar
  3. 3.
    Sosnoff, J.J., Broglio, S.P., Shin, S.H., Ferrara, M.S.: Previous mild traumatic brain injury and postural-control dynamics. J. Athl. Train. 46, 85–91 (2011) CrossRefGoogle Scholar
  4. 4.
    National Center for Injury Prevention and Control: Report to Congress on Mild Traumatic Brain Injury in the United States: Steps to Prevent a Serious Public Health Problem. Centers for Disease Control and Prevention, Atlanta (2003) Google Scholar
  5. 5.
    Finkelstein, E.A., Corso, P.S., Miller, T.R.: The Incidence and Economic Burden of Injuries in the United States. Oxford University Press, New York (2006) CrossRefGoogle Scholar
  6. 6.
    Aubry, M., Cantu, R., Dvorak, J., Graf-Baumann, T., Johnston, K.M., Kelly, J., Lovell, M., McCrory, P., Meeuwiasse, W.H., Schamasch, P., Concussion in Sport (CIS) Group: Summary and agreement statement of the 1st international symposium on concussion in sport. Clin. J. Sport Med. 12, 6–11 (2002) CrossRefGoogle Scholar
  7. 7.
    Richman, J.S., Moorman, J.R.: Physiological time-series analysis using approximate entropy and sample entropy. Am. J. Physiol., Heart Circ. Physiol. 278, H2039–H2049 (2000) Google Scholar
  8. 8.
    McCrory, P., Meeuwisse, W., Johnston, K., Dvorak, J., Aubry, M., Molloy, M., Cantu, R.: Consensus statement on concussion in sport: The 3rd International Conference on Concussion in Sport held in Zurich, November 2008. J. Athl. Train. 44, 434–448 (2009) CrossRefGoogle Scholar
  9. 9.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, New York (1991) MATHCrossRefGoogle Scholar
  10. 10.
    Grassberger, P., Procaccia, I.: Estimation of the Kolmogorov entropy from a chaotic signal. Phys. Rev. A 28, 2591–2593 (1983) CrossRefGoogle Scholar
  11. 11.
    Costa, M., Goldberger, A.L., Peng, C.K.: Multiscale entropy analysis of biological signals. Phys. Rev. E 71, 021906 (2005) MathSciNetCrossRefGoogle Scholar
  12. 12.
    Cao, Y.H., Tung, W.W., Gao, J.B., Protopopescu, V.A., Hively, L.M.: Detecting dynamical changes in time series using the permutation entropy. Phys. Rev. E 70, 046217 (2004) MathSciNetCrossRefGoogle Scholar
  13. 13.
    Lempel, A., Ziv, J.: On the complexity of finite sequences. IEEE Trans. Inf. Theory 22, 75–81 (1976) MathSciNetMATHCrossRefGoogle Scholar
  14. 14.
    Hu, J., Gao, J.B., Principe, J.C.: Analysis of biomedical signals by the Lempel-Ziv complexity: The effect of finite data size. IEEE Trans. Biomed. Eng. 53, 2606–2609 (2006) CrossRefGoogle Scholar
  15. 15.
    Gao, J.B., Cao, Y.H., Tung, W.W., Hu, J.: Multiscale Analysis of Complex Time Series—Integration of Chaos and Random Fractal Theory, and Beyond. Wiley, New York (2007) MATHGoogle Scholar
  16. 16.
    Gao, J.B., Hu, J., Tung, W.-W., Cao, Y.H.: Distinguishing chaos from noise by scale-dependent Lyapunov exponent. Phys. Rev. E 74, 066204 (2006) MathSciNetCrossRefGoogle Scholar
  17. 17.
    Andrzejak, R.G., Lehnertz, K., Rieke, C., Mormann, F., David, P., Elger, C.E.: Indications of nonlinear deterministic and finite dimensional structures in time series of brain electrical activity: Dependence on recording region and brain state. Phys. Rev. E 64, 061907 (2001) CrossRefGoogle Scholar
  18. 18.
    Adeli, H., Ghosh-Dastidar, S., Dadmehr, N.: A wavelet-chaos methodology for analysis of EEGs and EEG subbands to detect seizure and epilepsy. IEEE Trans. Biomed. Eng. 54, 205–211 (2007) CrossRefGoogle Scholar
  19. 19.
    Gao, J.B., Hu, J., Tung, W.W.: Complexity measures of brain wave dynamics. Cogn. Neurodyn. 5, 171–182 (2011) CrossRefGoogle Scholar
  20. 20.
    Gao, J.B., Hu, J., Tung, W.W.: Facilitating joint chaos and fractal analysis of biosignals through nonlinear adaptive filtering. PLoS ONE 6(9), e24331 (2011). doi: 10.1371/journal.pone.0024331 CrossRefGoogle Scholar
  21. 21.
    Hu, J., Gao, J.B., Wang, X.S.: Multifractal analysis of sunspot time series: The effects of the 11-year cycle and Fourier truncation. J. Stat. Mech. 2009/02/P02066 Google Scholar
  22. 22.
    Gao, J.B., Sultan, H., Hu, J., Tung, W.W.: Denoising nonlinear time series by adaptive filtering and wavelet shrinkage: A comparison. IEEE Signal Process. Lett. 17, 237–240 (2010) CrossRefGoogle Scholar
  23. 23.
    Tung, W.W., Gao, J.B., Hu, J., Yang, L.: Recovering chaotic signals in heavy noise environments. Phys. Rev. E 83, 046210 (2011) CrossRefGoogle Scholar
  24. 24.
    Gao, J.B., Hu, J., Buckley, T., White, K., Hass, C.: Shannon and Renyi entropies to classify effects of mild traumatic brain injury on postural sway. PLoS ONE 6(9), e24446 (2011). doi: 10.1371/journal.pone.0024446 CrossRefGoogle Scholar
  25. 25.
    Packard, N.H., Crutchfield, J.P., Farmer, J.D., Shaw, R.S.: Geometry from a time series. Phys. Rev. Lett. 45, 712–716 (1980) CrossRefGoogle Scholar
  26. 26.
    Takens, F.: Detecting strange attractors in turbulence. In: Rand, D.A., Young, L.S. (eds.) Dynamical Systems and Turbulence, Lecture Notes in Mathematics, vol. 898, pp. 366–381. Springer, Berlin (1981) Google Scholar
  27. 27.
    Sauer, T., Yorke, J.A., Casdagli, M.: Embedology. J. Stat. Phys. 65, 579–616 (1991) MathSciNetMATHCrossRefGoogle Scholar
  28. 28.
    Liebert, W., Pawelzik, K., Schuster, H.G.: Optimal embedding of chaotic attractors from topological considerations. Europhys. Lett. 14, 521–526 (1991) MathSciNetCrossRefGoogle Scholar
  29. 29.
    Kennel, M.B., Brown, R., Abarbanel, H.D.I.: Determining embedding dimension for phase-space reconstruction using a geometrical construction. Phys. Rev. A 45, 3403–3411 (1992) CrossRefGoogle Scholar
  30. 30.
    Gao, J.B., Zheng, Z.M.: Local exponential divergence plot and optimal embedding of a chaotic time series. Phys. Lett. A 181, 153–158 (1993) CrossRefGoogle Scholar
  31. 31.
    Gao, J.B., Zheng, Z.M.: Direct dynamical test for deterministic chaos and optimal embedding of a chaotic time series. Phys. Rev. E 49, 3807–3814 (1994) CrossRefGoogle Scholar
  32. 32.
    Atmanspacher, H., Scheingraber, H.: A fundamental link between system theory and statistical mechanics. Found. Phys. 17, 939–963 (1987) MathSciNetCrossRefGoogle Scholar
  33. 33.
    Gaspard, P., Wang, X.J.: Noise, chaos, and (ε,τ)-entropy per unit time. Phys. Rep. 235, 291–343 (1993) MathSciNetCrossRefGoogle Scholar
  34. 34.
    Cohen, A., Procaccia, I.: Computing the Kolmogorov entropy from time series of dissipative and conservative dynamical systems. Phys. Rev. A 31, 1872–1882 (1985) CrossRefGoogle Scholar
  35. 35.
    Eckmann, J.P., Ruelle, D.: Ergodic theory of chaos and strange attractors. Rev. Mod. Phys. 57, 617–656 (1985) MathSciNetCrossRefGoogle Scholar
  36. 36.
    Gao, J.B., Hu, J., Tung, W.W.: Multiscale entropy analysis of biological signals: A fundamental bi-scaling law. Europhys. Lett. (submitted) Google Scholar
  37. 37.
    Lorenz, E.Z.: Deterministic nonperiodic flow. J. Atmos. Sci. 20, 130–141 (1963) CrossRefGoogle Scholar
  38. 38.
    Gao, J.B., Tung, W.W., Hu, J.: Quantifying dynamical predictability: The pseudo-ensemble approach (in honor of Professor Andrew Majda’s 60th birthday). Chin. Ann. Math., Ser. B 30, 569–588 (2009) MathSciNetMATHCrossRefGoogle Scholar
  39. 39.
    Kleeman, R.: Measuring dynamical prediction utility using relative entropy. J. Atmos. Sci. 59, 2057–2072 (2002) CrossRefGoogle Scholar
  40. 40.
    Abramov, R., Majda, A., Kleeman, R.: Information theory and predictability for low frequency variability. J. Atmos. Sci. 62, 65–87 (2005) MathSciNetCrossRefGoogle Scholar
  41. 41.
    Haven, K., Majda, A., Abramov, R.: Quantifying predictability through information theory: Small sample estimation in a non-Gaussian framework. J. Comput. Phys. 206, 334–362 (2005) MathSciNetMATHCrossRefGoogle Scholar
  42. 42.
    Gao, J.B., Hwang, S.K., Liu, J.M.: When can noise induce chaos? Phys. Rev. Lett. 82, 1132–1135 (1999) CrossRefGoogle Scholar
  43. 43.
    Gao, J.B., Chen, C.C., Hwang, S.K., Liu, J.M.: Noise-induced chaos. Int. J. Mod. Phys. B 13, 3283–3305 (1999) CrossRefGoogle Scholar
  44. 44.
    Hwang, K., Gao, J.B., Liu, J.M.: Noise-induced chaos in an optically injected semiconductor laser. Phys. Rev. E 61, 5162–5170 (2000) CrossRefGoogle Scholar
  45. 45.
    Gao, J.B., Hu, J., Tung, W.W., Cao, Y.H., Sarshar, N., Roychowdhury, V.P.: Assessment of long range correlation in time series: How to avoid pitfalls. Phys. Rev. E 73, 016117 (2006) CrossRefGoogle Scholar
  46. 46.
    Gao, J.B., Hu, J., Tung, W.W., Zheng, Y.: Multiscale analysis of economic time series by scale-dependent Lyapunov exponent. Quant. Finance (2011). doi: 10.1080/14697688.2011.580774 Google Scholar
  47. 47.
    Gao, J.B., Hu, J., Mao, X., Tung, W.W.: Detecting low-dimensional chaos by the “noise titration” technique: Possible problems and remedies. Chaos Solitons Fractals (in press) Google Scholar
  48. 48.
    Hu, J., Gao, J.B., Tung, W.W.: Characterizing heart rate variability by scale-dependent Lyapunov exponent. Chaos 19, 028506 (2009) MathSciNetCrossRefGoogle Scholar
  49. 49.
    Hu, J., Gao, J.B., Tung, W.W., Cao, Y.H.: Multiscale analysis of heart rate variability: A comparison of different complexity measures. Ann. Biomed. Eng. 38, 854–864 (2010) CrossRefGoogle Scholar
  50. 50.
    Cellucci, C.J., Albano, A.M., Rapp, P.E., Pittenger, R.A., Josiassen, R.C.: Detecting noise in a time series. Chaos 7, 414–422 (1997) MATHCrossRefGoogle Scholar
  51. 51.
    Hu, J., Gao, J.B., White, K.D.: Estimating measurement noise in a time series by exploiting nonstationarity. Chaos Solitons Fractals 22, 807–819 (2004) MATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2011

Authors and Affiliations

  1. 1.PMB Intelligence LLCWest LafayetteUSA
  2. 2.Mechanical and Materials EngineeringWright State UniversityDaytonUSA
  3. 3.Affymetrix, Inc.Santa ClaraUSA
  4. 4.Department of Earth & Atmospheric SciencesPurdue UniversityWest LafayetteUSA

Personalised recommendations