Journal of Statistical Physics

, Volume 167, Issue 3–4, pp 892–933 | Cite as

Dimension Reduction for Systems with Slow Relaxation

In Memory of Leo P. Kadanoff
  • Shankar C. Venkataramani
  • Raman C. Venkataramani
  • Juan M. Restrepo


We develop reduced, stochastic models for high dimensional, dissipative dynamical systems that relax very slowly to equilibrium and can encode long term memory. We present a variety of empirical and first principles approaches for model reduction, and build a mathematical framework for analyzing the reduced models. We introduce the notions of universal and asymptotic filters to characterize ‘optimal’ model reductions for sloppy linear models. We illustrate our methods by applying them to the practically important problem of modeling evaporation in oil spills.


Dimension reduction Slow relaxation Sloppy models Mori–Zwanzig projection Multi-scale Aging Weathering Glassy systems Oil spills 



S.V. would like to acknowledge the many, very illuminating discussions with Kevin Lin who was very generous with his time and his ideas. We are grateful to an anonymous referee for pointing out the potential connections between our work and the sloppy models universality class. This viewpoint turns out to be particularly fruitful. This work was funded in part by a Grant from GoMRI. We also received support from NSF-DMS-1109856 and NSF-OCE-1434198.


  1. 1.
    Amir, A., Oreg, Y., Imry, Y.: On relaxations and aging of various glasses. Proc. Natl Acad. Sci. U.S.A. 109, 1850–1855 (2012)ADSCrossRefGoogle Scholar
  2. 2.
    Arnold, H.M., Moroz, I.M., Palmer, T.N.: Stochastic parametrizations and model uncertainty in the Lorenz ’96 system. Philos. Trans. R. Soc. Lond. A 371, 20120510 (2013)CrossRefGoogle Scholar
  3. 3.
    Baladi, V.: Positive Transfer Operators and Decay of Correlations, vol. 16. Advanced Series in Nonlinear Dynamics. World Scientific, Singapore (2000)Google Scholar
  4. 4.
    Berkenbusch, M.K., Claus, I., Dunn, C., Kadanoff, L.P., Nicewicz, M., Venkataramani, S.C.: Discrete charges on a two dimensional conductor. J. Stat. Phys. 116, 1301–1358 (2004)ADSMathSciNetMATHCrossRefGoogle Scholar
  5. 5.
    Berry, T., Harlim, J.: Forecasting turbulent modes with nonparametric diffusion models: learning from Noisy data. Physica D 320, 57–76 (2016)ADSMathSciNetCrossRefGoogle Scholar
  6. 6.
    Bouchaud, J.-P., Cugliandolo, L.F., Kurchan, J., Mezard, M.: Out of equilibrium dynamics in spin-glasses and other glassy systems. In: Spin Glasses and Random Fields, pp. 161–223. World Scientific, Singapore (1998)Google Scholar
  7. 7.
    Bouchaud, J.-P.: Aging in glassy systems: new experiments, simple models, and open questions. In: Cates, M.E., Evans, M. (eds.) Soft and Fragile Matter: Nonequilibrium Dynamics, Metastability and Flow, pp. 285–304. Institute of Physics, Bristol (2000)CrossRefGoogle Scholar
  8. 8.
    Box, G.E.P., Jenkins, G.M., Reinsel, G.C., Ljung, G.M.: Time Series Analysis: Forecasting and Control. Wiley Series in Probability and Statistics. Wiley, Hoboken (2015)Google Scholar
  9. 9.
    Brown, K.S., Sethna, J.P.: Statistical mechanical approaches to models with many poorly known parameters. Phys. Rev. E 68, 021904 (2003)ADSCrossRefGoogle Scholar
  10. 10.
    Budisic, M., Mohr, R., Mezic, I.: Applied Koopmanism. Chaos 22(4), 047510 (2012)ADSMathSciNetMATHCrossRefGoogle Scholar
  11. 11.
    Chekroun, M.D., Kondrashov, D., Ghil, M.: Predicting stochastic systems by noise sampling, and application to the El Niño-southern oscillation. Proc. Natl Acad. Sci. U.S.A. 108, 11766–11771 (2011)ADSCrossRefGoogle Scholar
  12. 12.
    Chorin, A.J., Hald, O.H.: Stochastic Tools in Mathematics and Science, vol. 58. Texts in Applied Mathematics. Springer, New York (2014)Google Scholar
  13. 13.
    Chorin, A.J., Hald, O.H., Kupferman, R.: Optimal prediction and the Mori–Zwanzig representation of irreversible processes. Proc. Natl Acad. Sci. U.S.A. 97, 2968–2973 (2000)ADSMathSciNetMATHCrossRefGoogle Scholar
  14. 14.
    Chorin, A., Hald, O., Kupferman, R.: Optimal prediction with memory. Physica D 166, 239–257 (2002)ADSMathSciNetMATHCrossRefGoogle Scholar
  15. 15.
    Chorin, A.J., Lu, F.: Discrete approach to stochastic parametrization and dimension reduction in nonlinear dynamics. Proc. Natl Acad. Sci. U.S.A. 112, 9804–9809 (2015)ADSCrossRefGoogle Scholar
  16. 16.
    Chorin, A., Stinis, P.: Problem reduction, renormalization, and memory. Commun. Appl. Math. Comput. Sci. 1, 1–27 (2007)MathSciNetMATHCrossRefGoogle Scholar
  17. 17.
    Coifman, R.R., Lafon, S.: Diffusion maps. Appl. Comput. Harmonic Anal. 21, 5–30 (2006)MathSciNetMATHCrossRefGoogle Scholar
  18. 18.
    Comeau, D., Zhao, Z., Giannakis, D., Majda, A.J.: Data-driven prediction strategies for low-frequency patterns of north pacific climate variability. Clim. Dyn. 48(5), 1855–1872 (2015)Google Scholar
  19. 19.
    Crisanti, A., Ritort, F.: Violation of the fluctuation–dissipation theorem in glassy systems: basic notions and the numerical evidence. J. Phys. A 36, R181 (2003)ADSMathSciNetMATHCrossRefGoogle Scholar
  20. 20.
    Darve, E., Solomon, J., Kia, A.: Computing generalized langevin equations and generalized Fokker–Planck equations. Proc. Natl Acad. Sci. U.S.A. 106, 10884–10889 (2009)ADSCrossRefGoogle Scholar
  21. 21.
    Dixon, P.K., Wu, L., Nagel, S.R., Williams, B.D., Carini, J.P.: Scaling in the relaxation of supercooled liquids. Phys. Rev. Lett. 65, 1108–1111 (1990)ADSCrossRefGoogle Scholar
  22. 22.
    Fingas, M.F.: A literature review of the physics and predictive modelling of oil spill evaporation. J. Hazard. Mater. 42, 157–175 (1995)CrossRefGoogle Scholar
  23. 23.
    Fingas, M.: Modeling evaporation using models that are not boundary-layer regulated. J. Hazard Mater. 107, 27–36 (2004)CrossRefGoogle Scholar
  24. 24.
    Fingas, M.: Modeling oil and petroleum evaporation. J. Pet. Sci. Res. 2(3), 104–115 (2013)Google Scholar
  25. 25.
    Flajolet, P., Odlyzko, A.: Singularity analysis of generating functions. SIAM J. Discret. Math. 3, 216–240 (1990)MathSciNetMATHCrossRefGoogle Scholar
  26. 26.
    Giannakis, D., Majda, A.J.: Nonlinear Laplacian spectral analysis for time series with intermittency and low-frequency variability. Proc. Natl Acad. Sci. U.S.A. 109, 2222–2227 (2012)ADSMathSciNetMATHCrossRefGoogle Scholar
  27. 27.
    Givon, D., Kupferman, R., Stuart, A.: Extracting macroscopic dynamics: model problems and algorithms. Nonlinearity 17, R55 (2004)ADSMathSciNetMATHCrossRefGoogle Scholar
  28. 28.
    Givon, D., Kupferman, R., Hald, O.H.: Existence proof for orthogonal dynamics and the Mori–Zwanzig formalism. Isr. J. Math. 145, 221–241 (2005)MathSciNetMATHCrossRefGoogle Scholar
  29. 29.
    Harlim, J., Kang, E.L., Majda, A.J.: Regression models with memory for the linear response of turbulent dynamical systems. Commun. Math. Sci. 11(2), 481–498 (2013)MathSciNetMATHCrossRefGoogle Scholar
  30. 30.
    Hoult, D.P. (ed.): Oil on the Sea: Proceedings of a Symposium on the Scientific and Engineering Aspects of Oil Pollution of the Sea. Springer, New York (1969)Google Scholar
  31. 31.
    Jazwinski, A.H.: Stochastic Processes and Filter Theory. Academic Press, New York (1970)MATHGoogle Scholar
  32. 32.
    Kampen, N.V.: Stochastic Processes in Physics and Chemistry, 3rd edn. North-Holland Personal Library. North Holland, Amsterdam (2007)Google Scholar
  33. 33.
    Kawasaki, K.: Simple derivations of generalized linear and nonlinear Langevin equations. J. Phys. A 6, 1289 (1973)ADSMathSciNetCrossRefGoogle Scholar
  34. 34.
    Kawasaki, K.: Theoretical methods dealing with slow dynamics. J. Phys. 12, 6343 (2000)Google Scholar
  35. 35.
    Kondrashov, D., Chekroun, M., Ghil, M.: Data-driven non-Markovian closure models. Physica D 297, 33–55 (2015)ADSMathSciNetCrossRefGoogle Scholar
  36. 36.
    Kubo, R.: The fluctuation–dissipation theorem. Rep. Prog. Phys. 29, 255 (1966)ADSMATHCrossRefGoogle Scholar
  37. 37.
    Kutner, M., Nachtsheim, C., Neter, J., Li, W.: Applied Linear Statistical Models. McGraw-Hill/Irwin, Chicago (2004)Google Scholar
  38. 38.
    Lin, K., Lu, F.: Stochastic parametrization, filtering, and the Mori–Zwanzig formalism. Preprint (2017)Google Scholar
  39. 39.
    Lu, F., Lin, K.K., Chorin, A.J.: Data-based stochastic model reduction for the Kuramoto–Sivashinsky equation. Physica D 340, 46–57 (2017)MathSciNetCrossRefGoogle Scholar
  40. 40.
    Mackay, D., Matsugu, R.S.: Evaporation rates of liquid hydrocarbon spills on land and water. Can. J. Chem. Eng. 51, 434–439 (1973)CrossRefGoogle Scholar
  41. 41.
    Majda, A.J., Harlim, J.: Physics constrained nonlinear regression models for time series. Nonlinearity 26, 201 (2013)ADSMathSciNetMATHCrossRefGoogle Scholar
  42. 42.
    Matan, K., Williams, R.B., Witten, T.A., Nagel, S.R.: Crumpling a thin sheet. Phys. Rev. Lett. 88, 076101 (2002)ADSCrossRefGoogle Scholar
  43. 43.
    Moghimi, S., Ramírez, J.M., Restrepo, J.M., Venkataramani, S.C.: Mass exchange dynamics of surface and subsurface oil in shallow-water transport. Preprint (2017)Google Scholar
  44. 44.
    Mori, H.: Transport, collective motion, and Brownian motion. Prog. Theor. Phys. 33, 423–455 (1965)ADSMATHCrossRefGoogle Scholar
  45. 45.
    Oded, B., Rubinstein, S.M., Fineberg, J.: Slip-stick and the evolution of frictional strength. Nature 463, 76–9 (2010)ADSCrossRefGoogle Scholar
  46. 46.
    Oppenheim, A.V., Schafer, R.W., Buck, J.R.: Discrete-Time Signal Processing, 2nd edn. Prentice-Hall Signal Processing Series. Prentice Hall, Englewood Cliffs (1999)Google Scholar
  47. 47.
    Ott, E.: Chaos in Dynamical Systems. Cambridge University Press, New York (1993)MATHGoogle Scholar
  48. 48.
    Polya, G., Szegö, G.: Problems and Theorems in Analysis II: Theory of Functions, Zeros, Polynomials, Determinants, Number Theory, Geometry, Classics in Mathematics. Springer, New York (1998)Google Scholar
  49. 49.
    Restrepo, J.M., Venkataramani, S.C., Dawson, C.: Nearshore sticky waters. Ocean Model. 80, 49–58 (2014)ADSCrossRefGoogle Scholar
  50. 50.
    Restrepo, J.M., Ramírez, J.M., Venkataramani, S.C.: An oil fate model for shallow waters. J. Marine Sci. Eng. 3, 1504–1543 (2015)CrossRefGoogle Scholar
  51. 51.
    Spaulding, M.L.: A state-of-the-art review of oil spill trajectory and fate modeling. Oil Chem. Pollut. 4, 39–55 (1988)CrossRefGoogle Scholar
  52. 52.
    Stinis, P.: Renormalized Mori–Zwanzig-reduced models for systems without scale separation. Proc. R. Soc. Lond. Ser. A 471, 20140446 (2015)ADSMathSciNetCrossRefGoogle Scholar
  53. 53.
    Stiver, W., Mackay, D.: Evaporation rate of spills of hydrocarbons and petroleum mixtures. Environ. Sci. Technol. 18(11), 834–840 (1984)ADSCrossRefGoogle Scholar
  54. 54.
    Sutton, O.G.: Wind structure and evaporation in a turbulent atmosphere. Proc. R. Soc. Lond. Ser. A 146, 701–722 (1934)ADSMATHCrossRefGoogle Scholar
  55. 55.
    Takens, E.: Detecting strange attractors in turbulence. In: Rand, D., Young, L.-S. (eds.) Dynamical Systems and Turbulence, pp. 366–381. Springer, Berlin (1981)Google Scholar
  56. 56.
    Transtrum, M.K., Machta, B.B., Sethna, J.P.: Geometry of nonlinear least squares with applications to sloppy models and optimization. Phys. Rev. E 83, 036701 (2011)ADSCrossRefGoogle Scholar
  57. 57.
    Vautard, R., Yiou, P., Ghil, M.: Singular-spectrum analysis: a toolkit for short, Noisy chaotic signals. Physica D 58, 95–126 (1992)ADSCrossRefGoogle Scholar
  58. 58.
    Vautard, R., Ghil, M.: Singular spectrum analysis in nonlinear dynamics, with applications to paleoclimatic time series. Physica D 35, 395–424 (1989)ADSMathSciNetMATHCrossRefGoogle Scholar
  59. 59.
    Venturi, D., Cho, H., Karniadakis, G.: Mori–Zwanzig approach to uncertainty quantification. In: Ghanem, R., Higdon, D., Owhadi, H. (eds.) Handbook of Uncertainty Quantification. Springer, Heidelberg (2016)Google Scholar
  60. 60.
    Venturi, D., Karniadakis, G.: Convolutionless Nakajima–Zwanzig equations for stochastic analysis in nonlinear dynamical systems. Proc. R. Soc. Math. Phys. Eng. Sci. 470, 20130754–20130754 (2014)ADSMathSciNetCrossRefGoogle Scholar
  61. 61.
    Walker, G.: On periodicity in series of related terms. Proc. R. Soc. Lond. Ser. A 131, 518–532 (1931)ADSMATHCrossRefGoogle Scholar
  62. 62.
    Waterfall, J.J., Casey, F.P., Gutenkunst, R.N., Brown, K.S., Myers, C.R., Brouwer, P.W., Elser, V., Sethna, J.P.: Sloppy-model universality class and the Vandermonde matrix. Phys. Rev. Lett. 97, 150601 (2006)ADSCrossRefGoogle Scholar
  63. 63.
    Yule, G.U.: On a method of investigating periodicities in disturbed series, with special reference to Wolfer’s sunspot numbers. Philos. Trans. R. Soc. Lond. Ser. A 226, 267–298 (1927)ADSMATHCrossRefGoogle Scholar
  64. 64.
    Zwanzig, R.: Problems in nonlinear transport theory. In: Garrido, L. (ed.) Systems Far from Equilibrium, vol. 132. Lecture Notes in Physics. Springer, Berlin (1980)Google Scholar
  65. 65.
    Zwanzig, R.: Nonlinear generalized Langevin equations. J. Stat. Phys. 9, 215–220 (1973)ADSCrossRefGoogle Scholar
  66. 66.
    Zwanzig, R.: Nonequilibrium Statistical Mechanics. Oxford University Press, New York (2001)MATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.Department of MathematicsUniversity of ArizonaTucsonUSA
  2. 2.Seagate TechnologyLongmontUSA
  3. 3.Department of MathematicsOregon State UniversityCorvallisUSA

Personalised recommendations