Skip to main content
SpringerLink
Log in

Memory

  • Chapter
  • First Online:
A Primer on Complex Systems

Part of the book series: Lecture Notes in Physics ((LNP,volume 943))

  • 1211 Accesses

Abstract

Memory is a term that often appears in discussions about complex dynamics. What is it meant when a scientist says that a certain complex system exhibits ‘memory’? More often than not it is meant that the past history of the system has some meaningful influence on its future evolution. Let’s consider, for instance, the case of earthquakes. It has often been claimed that the process of stress relaxation at a fault that can lead to the triggering of earthquakes exhibits long-term memory in the sense that when and where the next earthquake will happen, or how much stress and energy it will release, are somehow affected by how large past earthquakes were, as well as when and where they took place (Hergarten, Self-Organized Criticality in Earth Systems. Springer, Heidelberg, 2002).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 16.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Expected values were introduced in Sect. 2.2.5 (see Eq. 2.11).

  2. 2.

    However, the reverse is not true in general, since the covariance may also vanish in cases in which the processes are not independent [3].

  3. 3.

    That is, p(A ∪ B) = p(A)p(B). In fact, the degree of dependence between two events is measured through the probability [4]: p(A ∩ B) := p(A ∪ B) − p(A)p(B), that somewhat resembles the covariances defined in Eq. 4.1. Naturally, p(A ∩ B) = 0 if A and B are independent.

  4. 4.

    It may happen for τ ≠ 0 for other processes that are not stationary and random; for instance, for any periodic process when the lag τ = nP, being n any integer and P the period of the process.

  5. 5.

    Again, the extreme negative value can be reached for periodic processes when the lag is equal to a semiperiod, τ = ±(n + 1/2)P, with n integer.

  6. 6.

    The autocorrelation function (as does the covariance) quantifies statistical dependence, not causal dependence. That is, it only tells us whether it is statistically more probable that two values share a sign, not if the sign of the first value causes that of the second value. Assuming causal instead of statistical dependence is a common misconception. It must always be kept in mind that although causal dependence often translates into statistical dependence, the opposite is not always true. Think, for instance, in the case in which two processes are caused by a third one. The first two processes are statistically dependent, but there is clearly no causal dependence between them.

  7. 7.

    The equivalence of Eqs. 4.7 and 4.3 for stationary processes can be made apparent by replacing the ensemble averages in the latter by temporal averages:

    $$\displaystyle \begin{aligned} \begin{array}{rcl}{} \mathrm{Ac}_{[\,y]}( \tau) &\displaystyle = &\displaystyle \frac{ \left[ \lim_{T\rightarrow\infty} ~~\frac{1}{2T } \int_{-T}^{T} dt ~ y(t)y(t+\tau) - \left(\frac{1}{2T} \int_{-T}^{T} dt ~ y(t)\right)^2\right]}{ \left[ \lim_{T\rightarrow\infty} ~~\frac{1}{2T } \int_{-T}^{T} dt ~ y(t)^2 - \left(\frac{1}{2T} \int_{-T}^{T} dt ~ y(t)\right)^2\right]}. \end{array} \end{aligned} $$
    (4.6)

    Then, one can easily reorder the terms in order to convert this expression into Eq. 4.7.

  8. 8.

    In practice, T will be finite. We will make this dependence explicit, when needed, by adding a T superscript to the symbol (i.e., \( \mathrm {Ac}^T_{[\,y]}( \tau )\) or \(\bar y^T\)). To obtain meaningful results, T must be sufficiently large so that the dependence of the autocorrelation on T becomes negligible.

  9. 9.

    Naturally, the process y ∗ has zero mean and unit variance.

  10. 10.

    The precise way in which this autocorrelation function (or the power spectrum to be discussed later) is obtained will be discussed in Sect. 4.4. Here, to keep the discussion fluid, we just simply discuss the final results without dwelling too much on the details of how they are computed.

  11. 11.

    We will argue soon that one has to be careful with such hasty interpretation of the tail of the autocorrelation function, though.

  12. 12.

    Physically, this time would be interpreted as the average amount of time that turbulent structures take to pass by the Langmuir probe tip, if they are moving, or the average life of the local turbulent structures, if they remain relatively at rest with respect to the probe tip.

  13. 13.

    In our case, we will prefer to use the value of the integral of the autocorrelation function over the extended temporal range (see Eq. 4.11, that will be discussed next), that yields τ d  ∼ 60 μs.

  14. 14.

    Things will never be as clear-cut as we discuss them in this section. The direct determination of the autocorrelation tail from a finite record is usually quite inaccurate. Tails usually exhibit irregular oscillations and are contaminated by statistical noise, making it rather difficult to tell exponential from power-law decays by direct inspection. Other methods must be usually called upon for this task. Be it as it may, the discussion of the tails that follows will still illustrate several important concepts quite clearly.

  15. 15.

    In fact, it is the origin of the previously mentioned prescription, Ac[ y](τ d ) = 1/e.

  16. 16.

    For a = 1, the integral would diverge instead logarithmically.

  17. 17.

    Note that B must be positive in the case of the divergent tail; otherwise, Eq. 4.13 would not exhibit the non-negativeness property we discussed earlier.

  18. 18.

    It must be remembered that exact scale-invariance does not happen in nature, as we discussed in Chap. 3, being always limited to a mesorange set by finite-size effects. Thus, one should never expect an autocorrelation function with a perfect power-law scaling extending to infinitely long times, even if long-term memory is present in the system. The scaling should be eventually overcome by noise.

  19. 19.

    The definition of τ m given by Eq. 4.14 is intimately related to the so-called Green-Kubo relation that appears in stochastic transport theory [6]. The Green-Kubo relation states that, if v(t) is a random variable that represents the instantaneous velocity of a particle, any population of these particles will diffuse, at long times, according to the famous diffusive transport equation,

    $$\displaystyle \begin{aligned} \frac{\partial n}{\partial t} = D\frac{\partial^2 n}{\partial x^2} \end{aligned} $$
    (4.17)

    where n is the particle density, and with the diffusivity D given by the integral:

    $$\displaystyle \begin{aligned} D = \sigma^2\left( \lim_{T\rightarrow\infty} \int_0^T \mathrm{Ac}_{[v]} (\tau)d\tau\right). \end{aligned} $$
    (4.18)

    σ 2 is the velocity variance. There are cases, however, when the velocity fluctuations are such that this integral may vanish or diverge. We will discuss the type of transport that appears in these cases in Chap. 5 under the respective names of subdiffusive and superdiffusive transport. Both correspond to typical behaviours often found in the context of complex dynamics.

  20. 20.

    Indeed, one should expect that, if a longer record was available, one could calculate the autocorrelation function for larger values of τ. The divergent power-law scaling, if maintained for these larger lags, would then make the value of τ m to increase towards infinity as the record length is increased. This is in contrast to what would happen for an exponential decay, for which τ m will eventually become independent of the record length.

  21. 21.

    We consider here that the process is such that the interchange of limits, averages and integrals is allowed.

  22. 22.

    It is only by introducing the truncated process that we can extend the limits of the last integral in Eq. 4.20 to infinity, allowing us to invoke the convolution theorem (see Appendix 1 of Chap. 2) to get the final result for the power spectrum in terms of \(y^*_T(\omega )\).

  23. 23.

    If one uses the temporal-average formulation instead, the signature of the non-existence of the finite second moment is that \(\mathrm {Ac}^T_{[\,y]} (\tau )\), the autocorrelation computed with a record of finite length T, will never become independent of T. Instead, the autocorrelation will diverge as T →∞.

  24. 24.

    The autodifference is a particular case of more general function known as the s-codifference, that is defined for two arbitrary time processes with zero mean, x and y, as [7]:

    $$\displaystyle \begin{aligned} \mathrm{Cd}^s_{[x,y]}(\tau) = \left\langle\left| x(t + \tau) \right|{}^s\right\rangle + \left\langle \left|\,y(t) \right|{}^s\right\rangle - \left\langle\left| x(t+\tau) - y(t) \right|{}^s\right\rangle. \end{aligned} $$
    (4.28)

    The s-codifference reduces to (twice) the standard covariance for s = 2. In contrast to the cross-correlation, it can be calculated for processes x and y with divergent variance, as long as the moments of order s, \(\left \langle | ~\cdot ~|{ }^s\right \rangle \), do exist. It vanishes for any s for which it is finite when x and y are independent processes.

  25. 25.

    As it happened in the case of the autocorrelation, the inverse is not true in general.

  26. 26.

    As it also happened with the autocorrelation, T should be chosen large enough as to guarantee that the result becomes independent of it.

  27. 27.

    The derivation of Eq. 4.35 can be done either by considering the limit of truncated versions of fGn, along the lines of what we did to derive Eq. 4.20, or by moving to a generalization of the Fourier representation known as a Fourier-Stieltjes representation [9], that exists for stationary random processes [10].

  28. 28.

    It is worth mentioning at this point that some authors often use many of the methods discussed in Chap. 3 to characterize scale-invariance (i.e., moment methods, rescaling methods, DFA, etc) to characterize the presence of memory. The rationale for their use is that, for the monofractal fBm and fLm processes, the self-similarity exponent H also characterizes completely the memory properties of their associated noises. However, one must be careful when adhering to this philosophy since monofractal behaviour does not usually extend over all scales in real data, being instead restricted to within the mesorange (either because of finite-size effects, or by having different physics governing the dynamics at different scales). As a result, the self-similarity exponent H obtained say, from DFA, and the exponent c of the power spectrum of a real set of data, will often will not verify the relation c = 1 − 2H that we found for fBm in Sect. 4.3.1.

  29. 29.

    That is the reason why, in the autocorrelation of the TJ-II data we discussed at the beginning of this chapter, the values only run up to 103 lags, although the data set contained 104 points.

  30. 30.

    In fact, another way of estimating the autocorrelation of any time series is by computing its discrete power spectrum first, and then Fourier-inverting it to get the discrete autocorrelation.

  31. 31.

    One could also use one of the many fast Fourier transform (FFT) canned routines available [11] to evaluate Eq. 4.43. In that case, subroutines perform the sum,

    $$\displaystyle \begin{aligned} \sum_{k=0}^{N-1}y_k e^{2\pi \i nk/N},~~~~n= 0, 1, \cdots, N-1. \end{aligned} $$
    (4.44)

    that provides the Fourier transform at frequencies \(\omega ^{\prime }_n = 2\pi (n/N)\) instead of ω n  = n/N.

  32. 32.

    However, the interpretation of the power spectrum becomes much more involved when dealing with non-monofractal signals. We will see an example when applying the technique to the analysis of the running sandpile, carried out as always at the end of the chapter (see Sect. 4.5).

  33. 33.

    Indeed, second moments are infinite for an infinite series; they are finite, for a finite record, but their values diverge with the length of the series.

  34. 34.

    These difficulties were also apparent when computing the discrete autocorrelation function, but are made more apparent as s becomes smaller.

  35. 35.

    The \(\tilde y\)-series is thus formed by the increments of the z-series.

  36. 36.

    Hurst always assumed the statistics of the increment process (i.e., \(\tilde Y\)), to be near-Gaussian. As a result, his method only applies, as originally formulated, to processes with statistics with finite second moments.

  37. 37.

    In fact, Mandelbrot showed that this scaling property of the range requires only that the random process be self-similar with exponent H and with stationary increments. As such, the same range scaling also applies to fLm.

  38. 38.

    Although the average becomes less effective as k grows larger, since the number of blocks in the signal, roughly N/k, becomes smaller. This is apparent in the increasing perturbations that appear at the end of each of the curves in Fig. 4.11.

  39. 39.

    These two properties are quite important in the context of plasmas [18], due to the fact that signals are always contaminated by all kinds of noise and, quite often, also by mid- to low-frequency MHD modes. The R/S analysis provides a robust tool with which self-similarity and memory can be looked for in these contexts. On the other hand, plasma signals typically have tens (or even hundreds) of thousands of values, which makes the imprecisions of the R/S technique at small record lengths less worrisome.

  40. 40.

    This comes extremely handy in some contexts, such as plasma turbulence, where magnetohydrodynamic modes may sometime coexist with the background turbulence.

  41. 41.

    For instance, for fLn with α = 1, H R/S varies within (−1/2, 1/2]; for α = 0.5, H R/S varies within (−3/2, 1/2].

  42. 42.

    That is, one that does not scale with the record length!

  43. 43.

    Thus implying slight antipersistence, since H R/S < 0.5, what is consistent with the self-similarity exponent (H = 0.6 < 1/α = 0.625).

  44. 44.

    In some cases, non-exponential waiting-time pdfs have been interpreted as evidence of non-stationarity, instead of memory. That is, of processes in which the triggering rate varies with time, generally known under the name of inhomogeneous Poisson processes [20]. When the triggering rate is itself a random variable, the process is known as a Cox process. As usual, it is important to know the physics of the problem at hand, and to contrast any result with other analysis tools, so that one can distinguish between any of these non-stationary possibilities and truly self-similar behaviour.

  45. 45.

    Although these three regions are the most relevant regions, it can be shown that other regions can also appear in the sandpile, particularly as p 0 is varied [23] (see also Problem 4.6).

  46. 46.

    In fact, as the sandpile size increases, the timescale at which the anticorrelated region appears is pushed to larger timescales (or smaller frequencies)[23].

  47. 47.

    In fact, it was at first believed that such exponential pdf was a fundamental feature of SOC models [25]. It was later shown that this was not the case (see Sect. 1.3), and that very similar SOC dynamics could also be obtained in a sandpile driven with coloured noises, which yielded a power-law waiting-time pdf at small thresholds that reflected the correlation in the drive [24], or by choosing the location where a random drive is applied in a correlated manner [26].

References

  1. Hergarten, S.: Self-Organized Criticality in Earth Systems. Springer, Heidelberg (2002)

    Book  Google Scholar 

  2. Box, G.E.P., Jenkins, G.M., Reinsel, G.C.: Time Series Analysis: Forecasting and Control. Prentice-Hall, New York (1994)

    MATH  Google Scholar 

  3. Feller, W.: Probability Theory and Its Applications. Wiley, New York (1950)

    MATH  Google Scholar 

  4. Fuller, W.A.: Introduction to Statistical Time Series Analysis. Wiley, New York (1996)

    MATH  Google Scholar 

  5. Hidalgo, C., Balbin, R., Pedrosa, M.A., Garcia-Cortes, I., Anabitarte, E., Sentíes, J.M., San Jose, M., Bustamante, E.G., Giannone, L., Niedermeyer, H.: Edge Plasma Turbulence Diagnosis by Langmuir Probes. Contrib. Plasma Phys. 36, 139 (1996)

    Article  Google Scholar 

  6. Kubo, R.: Fluctuation-Dissipation Theorem. Rep. Prog. Phys. 29, 255 (1966)

    Article  Google Scholar 

  7. Samorodnitsky, G., Taqqu, M.S.: Stable Non-Gaussian Processes. Chapman and Hall, New York (1994)

    MATH  Google Scholar 

  8. Mandelbrot, B.B., van Ness, J.W.: Fractional Brownian Motions, Fractional Noises and Applications. SIAM Rev. 10, 422 (1968)

    MATH  Google Scholar 

  9. Katznelson, Y.: An Introduction to Harmonic Analysis. Dover, New York (1976)

    MATH  Google Scholar 

  10. Yaglom, A.M.: Correlation Theory of Stationary and Related Random Functions: Basic Results. Springer, New York (1987)

    MATH  Google Scholar 

  11. Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes. Cambridge University Press, Cambridge (2007)

    MATH  Google Scholar 

  12. Rosadi, D., Deistler, M.: Estimating the Codifference Function of Linear Time Series Models with Infinite Variance. Metrika 73, 395 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  13. Yu, J.: Empirical Characteristic Function Estimators and Its Applications. Econ. Rev. 23, 93 (2004)

    Article  MATH  Google Scholar 

  14. Hurst, H.E.: Long Term Storage Capacity of Reservoirs. Trans. Am. Soc. Civ. Eng. 116, 770 (1951)

    Google Scholar 

  15. Kantelhardt, J.W., Zschiegner, S.A., Koscielny-Bunde, E., Havlin, S., Bunde, A., Stanley, H.E.: Multifractal Detrended Fluctuation Analysis of Nonstationary Time Series. Physica A 316, 87 (2002)

    Article  ADS  MATH  Google Scholar 

  16. Mercik, S., Weron, R., Burnecki, K.: Enigma of Self-Similarity of Fractional Lévy Stable Motions. Acta Phys. Pol. B 34, 3773 (2003)

    ADS  MATH  Google Scholar 

  17. Weron, R.: Estimating Long Range Dependence: Finite Sample Properties and Confidence Intervals. Physica A 312, 285 (2002)

    Article  ADS  MathSciNet  MATH  Google Scholar 

  18. Carreras, B.A., van Milligen, B.P., Pedrosa, M.A., Balbin, R., Hidalgo, C., Newman, D.E., Sanchez, E., Frances, M., Garcia-Cortes, I., Bleuel, J., Endler, M., Riccardi, C., Davies, S., Matthews, G.F., Martines, E., Antoni, V., Latten, A., Klinger, T.: Self-Similarity of the Plasma Edge Fluctuations. Phys. Plasmas 5, 3632 (1998)

    Article  ADS  Google Scholar 

  19. Chechnik, A.V., Gonchar, V.Y.: Self and Spurious Multi-Affinity of Ordinary Lévy Motion, and Pseudo-Gaussian Relations. Chaos Solitons Fractals 11, 2379 (2000)

    Article  ADS  MATH  Google Scholar 

  20. Cox, D.R., Isham, V.: Point Processes. Chapman and Hall, New York (1980)

    MATH  Google Scholar 

  21. Spada, E., Carbone, V., Cavazzana, R., Fattorini, L., Regnoli, G., Vianello, N., Antoni, V., Martines, E., Serianni, G., Spolaore, M., Tramontin, L.: Search of Self-Organized Criticality Processes in Magnetically Confined Plasmas: Hints from the Reversed Field Pinch Configuration. Phys. Rev. Lett. 86, 3032 (2001)

    Article  ADS  Google Scholar 

  22. Hwa, T., Kardar, M.: Avalanches, Hydrodynamics and Discharge Events in Models of Sand Piles. Phys. Rev. A 45, 7002 (1992)

    Article  ADS  Google Scholar 

  23. Woodard, R., Newman, D.E., Sanchez, R., Carreras, B.A.: Persistent Dynamic Correlations in Self-Organized Critical Systems Away from Their Critical Point. Physica A 373, 215 (2007)

    Article  ADS  Google Scholar 

  24. Sanchez, R., Newman, D.E., Carreras, B.A.: Waiting-Time Statistics of Self-Organized-Criticality Systems. Phys. Rev. Lett. 88, 068302 (2002)

    Article  ADS  Google Scholar 

  25. Boffetta, G., Carbone, V., Giuliani, P., Veltri, P., Vulpiani, A.: Power Laws in Solar Flares: Self-Organized Criticality or Turbulence? Phys. Rev. Lett. 83, 4662 (1999)

    Article  ADS  Google Scholar 

  26. Sattin, F., Baiesi, M.: Self-Organized-Criticality Model Consistent with Statistical Properties of Edge Turbulence in a Fusion Plasma. Phys. Rev. Lett. 96, 105005 (2006)

    Article  ADS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Departamento de Física, Grupo de Fusion, Universidad Carlos III de Madrid, Madrid, Spain

    Raúl Sánchez

  2. Physics Department, University of Alaska-Fairbanks, Fairbanks, Alaska, USA

    David Newman

Authors
  1. Raúl Sánchez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Newman
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Science+Business Media B.V.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Sánchez, R., Newman, D. (2018). Memory. In: A Primer on Complex Systems. Lecture Notes in Physics, vol 943. Springer, Dordrecht. https://doi.org/10.1007/978-94-024-1229-1_4

Download citation

Publish with us

Policies and ethics

Search

Navigation