Journal of Statistical Physics

, Volume 177, Issue 1, pp 95–118 | Cite as

Fokker–Planck Equations for Time-Delayed Systems via Markovian Embedding

  • Sarah A. M. LoosEmail author
  • Sabine H. L. Klapp


For stochastic systems with discrete time delay, the Fokker–Planck equation (FPE) of the one-time probability density function (PDF) does not provide a complete, self-contained probabilistic description. It explicitly involves the two-time PDF, and represents, in fact, only the first member of an infinite hierarchy. We here introduce a new approach to obtain a Fokker–Planck description by using a Markovian embedding technique and a subsequent limiting procedure. On this way, we find a closed, complete FPE in an infinite-dimensional space, from which one can derive a hierarchy of FPEs. While the first member is the well-known FPE for the one-time PDF, we obtain, as second member, a new representation of the equation for the two-time PDF. From a conceptual point of view, our approach is simpler than earlier derivations and it yields interesting insight into both, the physical meaning, and the mathematical structure of delayed processes. We further propose an approximation for the two-time PDF, which is a central quantity in the description of these non-Markovian systems as it directly gives the correlation between the present and the delayed state. Application to a prototypical bistable system reveals that this approximation captures the non-trivial effects induced by the delay remarkably well, despite its surprisingly simple form. Moreover, it outperforms earlier approaches for the one-time PDF in the regime of large delays.


Time delay Stochastic delay differential equations Fokker–Planck equations Non-Markovian processes Novikov’s theorem 



This work was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - Projektnummer 163436311 - SFB 910. We further thank Christian Kuehn for fruitful discussions.


  1. 1.
    Bao, J.D., Hänggi, P., Zhuo, Y.Z.: Non-Markovian Brownian dynamics and nonergodicity. Phys. Rev. E 72(6), 061107 (2005)ADSCrossRefGoogle Scholar
  2. 2.
    Bruot, N., Damet, L., Kotar, J., Cicuta, P., Lagomarsino, M.C.: Noise and synchronization of a single active colloid. Phys. Rev. Lett. 107(9), 094101 (2011)ADSCrossRefGoogle Scholar
  3. 3.
    Cabral, J.R., Luckhoo, H., Woolrich, M., Joensson, M., Mohseni, H., Baker, A., Kringelbach, M.L., Deco, G.: Exploring mechanisms of spontaneous functional connectivity in MEG: how delayed network interactions lead to structured amplitude envelopes of band-pass filtered oscillations. Neuroimage 90, 423–435 (2014)CrossRefGoogle Scholar
  4. 4.
    Callen, J.L., Khan, M., Lu, H.: Accounting quality, stock price delay, and future stock returns. Contemp. Acc. Res. 30(1), 269–295 (2013)CrossRefGoogle Scholar
  5. 5.
    Carmele, A., Kabuss, J., Schulze, F., Reitzenstein, S., Knorr, A.: Single photon delayed feedback: a way to stabilize intrinsic quantum cavity electrodynamics. Phys. Rev. Lett. 110(1), 013601 (2013)ADSCrossRefGoogle Scholar
  6. 6.
    Crisanti, A., Puglisi, A., Villamaina, D.: Nonequilibrium and information: the role of cross correlations. Phys. Rev. E 85(6), 061127 (2012)ADSCrossRefGoogle Scholar
  7. 7.
    De Vries, B., Principe, J.C.: A theory of neural networks with time delays. In: Lippmann, R.P., Moody., J.E., Touretzky, D.S. (eds.) Advances in Neural Information Processing Systems, pp. 162–168. Morgan Kaufmann, San Mateo, CA (1991)Google Scholar
  8. 8.
    Durve, M., Saha, A., Sayeed, A.: Active particle condensation by non-reciprocal and time-delayed interactions. Eur. Phys. J. E 41(4), 49 (2018)CrossRefGoogle Scholar
  9. 9.
    Frank, T.D.: Analytical results for fundamental time-delayed feedback systems subjected to multiplicative noise. Phys. Rev. E 69, 061104 (2004)ADSMathSciNetCrossRefGoogle Scholar
  10. 10.
    Frank, T.D.: Delay Fokker–Planck equations, Novikov’s theorem, and Boltzmann distributions as small delay approximations. Phys. Rev. E 72(1), 011112 (2005)ADSMathSciNetCrossRefGoogle Scholar
  11. 11.
    Frank, T.D.: Delay Fokker–Planck equations, perturbation theory, and data analysis for nonlinear stochastic systems with time delays. Phys. Rev. E 71(3), 031106 (2005)ADSCrossRefGoogle Scholar
  12. 12.
    Frank, T.D., Beek, P.J.: Stationary solutions of linear stochastic delay differential equations: applications to biological systems. Phys. Rev. E 64(2), 021917 (2001)ADSCrossRefGoogle Scholar
  13. 13.
    Frank, T.D., Beek, P.J., Friedrich, R.: Fokker–Planck perspective on stochastic delay systems: exact solutions and data analysis of biological systems. Phys. Rev. E 68(2), 021912 (2003)ADSCrossRefGoogle Scholar
  14. 14.
    Gardiner, C.W.: Handbook of Stochastic Methods, 2nd edn. Springer, Berlin (2002)Google Scholar
  15. 15.
    Gernert, R., Loos, S.A.M., Lichtner, K., Klapp, S.H.L.: Feedback control of colloidal transport. In: Schöll, E., Klapp, S., Hövel, P. (eds.) Control of Self-Organizing Nonlinear Systems, pp. 375–392. Springer, Berlin (2016)Google Scholar
  16. 16.
    Giuggioli, L., McKetterick, T.J., Kenkre, V.M., Chase, M.: Fokker–Planck description for a linear delayed Langevin equation with additive Gaussian noise. J. Phys. A 49(38), 384002 (2016)ADSMathSciNetzbMATHCrossRefGoogle Scholar
  17. 17.
    Guillouzic, S., L’Heureux, I., Longtin, A.: Small delay approximation of stochastic delay differential equations. Phys. Rev. E 59(4), 3970 (1999)ADSzbMATHCrossRefGoogle Scholar
  18. 18.
    Gupta, V., Kadambari, K., et al.: Neuronal model with distributed delay: analysis and simulation study for gamma distribution memory kernel. Biol. Cybern. 104(6), 369–383 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Hall, M.J.W., Rignatto, M.: Ensembles on configuration space. In: Fundamental Theories of Physics. Springer Nature, Switzerland (2016)Google Scholar
  20. 20.
    Kane, D.M., Shore, K.A. (eds.): Unlocking Dynamical Diversity: Optical Feedback Effects on Semiconductor Lasers. Wiley, New York (2005)Google Scholar
  21. 21.
    Kawaguchi, K., Nakayama, Y.: Fluctuation theorem for hidden entropy production. Phys. Rev. E 88(2), 022147 (2013)ADSCrossRefGoogle Scholar
  22. 22.
    Khadka, U., Holubec, V., Yang, H., Cichos, F.: Active particles bound by information flows. Nat. Commun. 9, 3864 (2018)ADSCrossRefGoogle Scholar
  23. 23.
    Kotar, J., Leoni, M., Bassetti, B., Lagomarsino, M.C., Cicuta, P.: Hydrodynamic synchronization of colloidal oscillators. Proc. Natl. Acad. Sci. USA 107(17), 7669–7673 (2010)ADSCrossRefGoogle Scholar
  24. 24.
    Krüeger, M., Maes, C.: The modified Langevin description for probes in a nonlinear medium. J. Phys.: Condens. Matter 29(6), 064004 (2016)ADSGoogle Scholar
  25. 25.
    Kubo, R.: The fluctuation-dissipation theorem. Rep. Prog. Phys. 29(1), 255 (1966)ADSzbMATHCrossRefGoogle Scholar
  26. 26.
    Küchler, U., Mensch, B.: Langevins stochastic differential equation extended by a time-delayed term. Stoch. Stoch. Rep. 40(1–2), 23–42 (1992)MathSciNetzbMATHCrossRefGoogle Scholar
  27. 27.
    Liu, Y., Chen, H., Liu, J., Davis, P., Aida, T.: Synchronization of optical-feedback-induced chaos in semiconductor lasers by optical injection. Phys. Rev. A 63(3), 031802 (2001)ADSCrossRefGoogle Scholar
  28. 28.
    Longtin, A.: Complex Time-Delay Systems: Theory and Applications. Springer, Berlin (2010)zbMATHGoogle Scholar
  29. 29.
    Longtin, A., Milton, J.G., Bos, J.E., Mackey, M.C.: Noise and critical behavior of the pupil light reflex at oscillation onset. Phys. Rev. A 41(12), 6992 (1990)ADSCrossRefGoogle Scholar
  30. 30.
    Loos, S.A.M., Gernert, R., Klapp, S.H.L.: Delay-induced transport in a rocking ratchet under feedback control. Phys. Rev. E 89(5), 052136 (2014)ADSCrossRefGoogle Scholar
  31. 31.
    Loos, S.A.M., Klapp, S.H.L.: Force-linearization closure for non-Markovian Langevin systems with time delay. Phys. Rev. E 96(13), 012106 (2017)ADSCrossRefGoogle Scholar
  32. 32.
    Loos, S.A.M., Klapp, S.H.L.: Heat flow due to time-delayed feedback. Sci. Rep. 9, 2491 (2019)ADSCrossRefGoogle Scholar
  33. 33.
    Maes, C.: On the second fluctuation-dissipation theorem for nonequilibrium baths. J. Stat. Phys. 154(3), 705–722 (2014)ADSMathSciNetzbMATHCrossRefGoogle Scholar
  34. 34.
    Maes, C., Thomas, S.R.: From Langevin to generalized Langevin equations for the nonequilibrium rouse model. Phys. Rev. E 87(2), 022145 (2013)ADSCrossRefGoogle Scholar
  35. 35.
    Masoller, C.: Noise-induced resonance in delayed feedback systems. Phys. Rev. Lett. 88, 034102 (2002)ADSCrossRefGoogle Scholar
  36. 36.
    Mehl, J., Lander, B., Bechinger, C., Blickle, V., Seifert, U.: Role of hidden slow degrees of freedom in the fluctuation theorem. Phys. Rev. Lett. 108(22), 220601 (2012)ADSCrossRefGoogle Scholar
  37. 37.
    Mijalkov, M., McDaniel, A., Wehr, J., Volpe, G.: Engineering sensorial delay to control phototaxis and emergent collective behaviors. Phys. Rev. X 6(1), 011008 (2016)Google Scholar
  38. 38.
    Német, N., Parkins, S.: Enhanced optical squeezing from a degenerate parametric amplifier via time-delayed coherent feedback. Phys. Rev. A 94(2), 023809 (2016)ADSCrossRefGoogle Scholar
  39. 39.
    Niculescu, S.I., Gu, K.: Advances in Time-Delay Systems, vol. 38. Springer, Berlin (2012)zbMATHGoogle Scholar
  40. 40.
    Novikov, E.A.: Functionals and the random-force method in turbulence theory. Sov. Phys. JETP 20(5), 1290–1294 (1965)ADSMathSciNetGoogle Scholar
  41. 41.
    Puglisi, A., Villamaina, D.: Irreversible effects of memory. EPL 88(3), 30004 (2009)ADSCrossRefGoogle Scholar
  42. 42.
    Rateitschak, K., Wolkenhauer, O.: Intracellular delay limits cyclic changes in gene expression. Math. Biosci. 205(2), 163–179 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  43. 43.
    Reimann, P.: Brownian motors noisy transport far from equilibrium. Phys. Rep. 361, 57–265 (2002)ADSMathSciNetzbMATHCrossRefGoogle Scholar
  44. 44.
    René, A., Longtin, A.: Mean, covariance, and effective dimension of stochastic distributed delay dynamics. Chaos 27(11), 114322 (2017)ADSMathSciNetzbMATHCrossRefGoogle Scholar
  45. 45.
    Rosinberg, M.L., Tarjus, G., Munakata, T.: Influence of time delay on information exchanges between coupled linear stochastic systems. Phys. Rev. E 98(3), 032130 (2018)ADSCrossRefGoogle Scholar
  46. 46.
    Rosinberg, M.L., Munakata, T., Tarjus, G.: Stochastic thermodynamics of Langevin systems under time-delayed feedback control: second-law-like inequalities. Phys. Rev. E 91, 042114 (2015)ADSMathSciNetCrossRefGoogle Scholar
  47. 47.
    Schneider, I.: Delayed feedback control of three diffusively coupled Stuart–Landau oscillators: a case study in equivariant Hopf bifurcation. Philos. Trans. R. Soc. A 371(1999), 20120472 (2013)ADSMathSciNetzbMATHCrossRefGoogle Scholar
  48. 48.
    Schneider, I., Bosewitz, M.: Eliminating restrictions of time-delayed feedback control using equivariance. Discret. Contin. Dyn. Syst. A 36(1), 451–467 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
  49. 49.
    Schöll, E., Klapp, S.H.L., Hövel, P. (eds.): Control of Self-organizing Nonlinear Systems. Springer, Berlin (2016)zbMATHGoogle Scholar
  50. 50.
    Schöll, E., Schuster, H.G. (eds.): Handbook of Chaos Control. Wiley, New York (2008)zbMATHGoogle Scholar
  51. 51.
    Seifert, U.: Stochastic thermodynamics, fluctuation theorems and molecular machines. Rep. Prog. Phys. 75(12), 126001 (2012)ADSCrossRefGoogle Scholar
  52. 52.
    Sekimoto, K.: Stochastic Energetics, vol. 799. Springer, Berlin (2010)zbMATHCrossRefGoogle Scholar
  53. 53.
    Siegle, P., Goychuk, I., Hänggi, P.: Markovian embedding of fractional superdiffusion. EPL 93(2), 20002 (2011)ADSCrossRefGoogle Scholar
  54. 54.
    Siegle, P., Goychuk, I., Talkner, P., Hänggi, P.: Markovian embedding of non-Markovian superdiffusion. Phys. Rev. E 81(1), 011136 (2010)ADSCrossRefGoogle Scholar
  55. 55.
    Smith, H.L.: An Introduction to Delay Differential Equations with Applications to the Life Sciences, vol. 57. Springer, New York (2011)zbMATHCrossRefGoogle Scholar
  56. 56.
    Tambue, A., Brown, E.K., Mohammed, S.: A stochastic delay model for pricing debt and equity: numerical techniques and applications. Commun. Nonlinear Sci. Numer. Simul. 20(1), 281–297 (2015)ADSMathSciNetzbMATHCrossRefGoogle Scholar
  57. 57.
    Tsimring, L.S., Pikovsky, A.: Noise-induced dynamics in bistable systems with delay. Phys. Rev. Lett. 87(25), 250602 (2001)ADSCrossRefGoogle Scholar
  58. 58.
    Villamaina, D., Baldassarri, A., Puglisi, A., Vulpiani, A.: The fluctuation-dissipation relation: how does one compare correlation functions and responses? J. Stat. Mech. Theory Exp. 2009(07), P07024 (2009)CrossRefGoogle Scholar
  59. 59.
    Waibel, A., Hanazawa, T., Hinton, G., Shikano, K., Lang, K.J.: Backpropagation: Theory, Architectures and Applications. Lawrence Erlbaum Associates, Mahwah, NJ (1995)Google Scholar
  60. 60.
    Zakharova, A., Loos, S.A.M., Siebert, J., Gjurchinovski, A., Claussen, J.C., Schöll, E.: Controlling chimera patterns in networks: interplay of structure, noise, delay. In: Schöll, E., Klapp, S., Hövel, P. (eds.) Control of Self-Organizing Nonlinear Systems, pp. 3–23. Springer, Berlin (2016)Google Scholar
  61. 61.
    Zheng, Y., Sun, X.: Governing equations for Probability densities of stochastic differential equations with discrete time delays. Discret. Contin. Dyn. Syst. Ser. B 22(9) (2017)Google Scholar
  62. 62.
    Zwanzig, R.: Nonlinear generalized Langevin equations. J. Stat. Phys. 9(3), 215–220 (1973)ADSCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.ITP, Technische Universität BerlinBerlinGermany

Personalised recommendations