# Dimension Reduction for Systems with Slow Relaxation

- 141 Downloads

## Abstract

We develop reduced, stochastic models for high dimensional, dissipative dynamical systems that relax very slowly to equilibrium and can encode long term memory. We present a variety of empirical and first principles approaches for model reduction, and build a mathematical framework for analyzing the reduced models. We introduce the notions of universal and asymptotic filters to characterize ‘optimal’ model reductions for sloppy linear models. We illustrate our methods by applying them to the practically important problem of modeling evaporation in oil spills.

### Keywords

Dimension reduction Slow relaxation Sloppy models Mori–Zwanzig projection Multi-scale Aging Weathering Glassy systems Oil spills## Notes

### Acknowledgements

S.V. would like to acknowledge the many, very illuminating discussions with Kevin Lin who was very generous with his time and his ideas. We are grateful to an anonymous referee for pointing out the potential connections between our work and the sloppy models universality class. This viewpoint turns out to be particularly fruitful. This work was funded in part by a Grant from GoMRI. We also received support from NSF-DMS-1109856 and NSF-OCE-1434198.

### References

- 1.Amir, A., Oreg, Y., Imry, Y.: On relaxations and aging of various glasses. Proc. Natl Acad. Sci. U.S.A.
**109**, 1850–1855 (2012)ADSCrossRefGoogle Scholar - 2.Arnold, H.M., Moroz, I.M., Palmer, T.N.: Stochastic parametrizations and model uncertainty in the Lorenz ’96 system. Philos. Trans. R. Soc. Lond. A
**371**, 20120510 (2013)CrossRefGoogle Scholar - 3.Baladi, V.: Positive Transfer Operators and Decay of Correlations, vol. 16. Advanced Series in Nonlinear Dynamics. World Scientific, Singapore (2000)Google Scholar
- 4.Berkenbusch, M.K., Claus, I., Dunn, C., Kadanoff, L.P., Nicewicz, M., Venkataramani, S.C.: Discrete charges on a two dimensional conductor. J. Stat. Phys.
**116**, 1301–1358 (2004)ADSMathSciNetMATHCrossRefGoogle Scholar - 5.Berry, T., Harlim, J.: Forecasting turbulent modes with nonparametric diffusion models: learning from Noisy data. Physica D
**320**, 57–76 (2016)ADSMathSciNetCrossRefGoogle Scholar - 6.Bouchaud, J.-P., Cugliandolo, L.F., Kurchan, J., Mezard, M.: Out of equilibrium dynamics in spin-glasses and other glassy systems. In: Spin Glasses and Random Fields, pp. 161–223. World Scientific, Singapore (1998)Google Scholar
- 7.Bouchaud, J.-P.: Aging in glassy systems: new experiments, simple models, and open questions. In: Cates, M.E., Evans, M. (eds.) Soft and Fragile Matter: Nonequilibrium Dynamics, Metastability and Flow, pp. 285–304. Institute of Physics, Bristol (2000)CrossRefGoogle Scholar
- 8.Box, G.E.P., Jenkins, G.M., Reinsel, G.C., Ljung, G.M.: Time Series Analysis: Forecasting and Control. Wiley Series in Probability and Statistics. Wiley, Hoboken (2015)Google Scholar
- 9.Brown, K.S., Sethna, J.P.: Statistical mechanical approaches to models with many poorly known parameters. Phys. Rev. E
**68**, 021904 (2003)ADSCrossRefGoogle Scholar - 10.Budisic, M., Mohr, R., Mezic, I.: Applied Koopmanism. Chaos
**22**(4), 047510 (2012)ADSMathSciNetMATHCrossRefGoogle Scholar - 11.Chekroun, M.D., Kondrashov, D., Ghil, M.: Predicting stochastic systems by noise sampling, and application to the El Niño-southern oscillation. Proc. Natl Acad. Sci. U.S.A.
**108**, 11766–11771 (2011)ADSCrossRefGoogle Scholar - 12.Chorin, A.J., Hald, O.H.: Stochastic Tools in Mathematics and Science, vol. 58. Texts in Applied Mathematics. Springer, New York (2014)Google Scholar
- 13.Chorin, A.J., Hald, O.H., Kupferman, R.: Optimal prediction and the Mori–Zwanzig representation of irreversible processes. Proc. Natl Acad. Sci. U.S.A.
**97**, 2968–2973 (2000)ADSMathSciNetMATHCrossRefGoogle Scholar - 14.Chorin, A., Hald, O., Kupferman, R.: Optimal prediction with memory. Physica D
**166**, 239–257 (2002)ADSMathSciNetMATHCrossRefGoogle Scholar - 15.Chorin, A.J., Lu, F.: Discrete approach to stochastic parametrization and dimension reduction in nonlinear dynamics. Proc. Natl Acad. Sci. U.S.A.
**112**, 9804–9809 (2015)ADSCrossRefGoogle Scholar - 16.Chorin, A., Stinis, P.: Problem reduction, renormalization, and memory. Commun. Appl. Math. Comput. Sci.
**1**, 1–27 (2007)MathSciNetMATHCrossRefGoogle Scholar - 17.Coifman, R.R., Lafon, S.: Diffusion maps. Appl. Comput. Harmonic Anal.
**21**, 5–30 (2006)MathSciNetMATHCrossRefGoogle Scholar - 18.Comeau, D., Zhao, Z., Giannakis, D., Majda, A.J.: Data-driven prediction strategies for low-frequency patterns of north pacific climate variability. Clim. Dyn.
**48**(5), 1855–1872 (2015)Google Scholar - 19.Crisanti, A., Ritort, F.: Violation of the fluctuation–dissipation theorem in glassy systems: basic notions and the numerical evidence. J. Phys. A
**36**, R181 (2003)ADSMathSciNetMATHCrossRefGoogle Scholar - 20.Darve, E., Solomon, J., Kia, A.: Computing generalized langevin equations and generalized Fokker–Planck equations. Proc. Natl Acad. Sci. U.S.A.
**106**, 10884–10889 (2009)ADSCrossRefGoogle Scholar - 21.Dixon, P.K., Wu, L., Nagel, S.R., Williams, B.D., Carini, J.P.: Scaling in the relaxation of supercooled liquids. Phys. Rev. Lett.
**65**, 1108–1111 (1990)ADSCrossRefGoogle Scholar - 22.Fingas, M.F.: A literature review of the physics and predictive modelling of oil spill evaporation. J. Hazard. Mater.
**42**, 157–175 (1995)CrossRefGoogle Scholar - 23.Fingas, M.: Modeling evaporation using models that are not boundary-layer regulated. J. Hazard Mater.
**107**, 27–36 (2004)CrossRefGoogle Scholar - 24.Fingas, M.: Modeling oil and petroleum evaporation. J. Pet. Sci. Res.
**2**(3), 104–115 (2013)Google Scholar - 25.Flajolet, P., Odlyzko, A.: Singularity analysis of generating functions. SIAM J. Discret. Math.
**3**, 216–240 (1990)MathSciNetMATHCrossRefGoogle Scholar - 26.Giannakis, D., Majda, A.J.: Nonlinear Laplacian spectral analysis for time series with intermittency and low-frequency variability. Proc. Natl Acad. Sci. U.S.A.
**109**, 2222–2227 (2012)ADSMathSciNetMATHCrossRefGoogle Scholar - 27.Givon, D., Kupferman, R., Stuart, A.: Extracting macroscopic dynamics: model problems and algorithms. Nonlinearity
**17**, R55 (2004)ADSMathSciNetMATHCrossRefGoogle Scholar - 28.Givon, D., Kupferman, R., Hald, O.H.: Existence proof for orthogonal dynamics and the Mori–Zwanzig formalism. Isr. J. Math.
**145**, 221–241 (2005)MathSciNetMATHCrossRefGoogle Scholar - 29.Harlim, J., Kang, E.L., Majda, A.J.: Regression models with memory for the linear response of turbulent dynamical systems. Commun. Math. Sci.
**11**(2), 481–498 (2013)MathSciNetMATHCrossRefGoogle Scholar - 30.Hoult, D.P. (ed.): Oil on the Sea: Proceedings of a Symposium on the Scientific and Engineering Aspects of Oil Pollution of the Sea. Springer, New York (1969)Google Scholar
- 31.Jazwinski, A.H.: Stochastic Processes and Filter Theory. Academic Press, New York (1970)MATHGoogle Scholar
- 32.Kampen, N.V.: Stochastic Processes in Physics and Chemistry, 3rd edn. North-Holland Personal Library. North Holland, Amsterdam (2007)Google Scholar
- 33.Kawasaki, K.: Simple derivations of generalized linear and nonlinear Langevin equations. J. Phys. A
**6**, 1289 (1973)ADSMathSciNetCrossRefGoogle Scholar - 34.Kawasaki, K.: Theoretical methods dealing with slow dynamics. J. Phys.
**12**, 6343 (2000)Google Scholar - 35.Kondrashov, D., Chekroun, M., Ghil, M.: Data-driven non-Markovian closure models. Physica D
**297**, 33–55 (2015)ADSMathSciNetCrossRefGoogle Scholar - 36.Kubo, R.: The fluctuation–dissipation theorem. Rep. Prog. Phys.
**29**, 255 (1966)ADSMATHCrossRefGoogle Scholar - 37.Kutner, M., Nachtsheim, C., Neter, J., Li, W.: Applied Linear Statistical Models. McGraw-Hill/Irwin, Chicago (2004)Google Scholar
- 38.Lin, K., Lu, F.: Stochastic parametrization, filtering, and the Mori–Zwanzig formalism. Preprint (2017)Google Scholar
- 39.Lu, F., Lin, K.K., Chorin, A.J.: Data-based stochastic model reduction for the Kuramoto–Sivashinsky equation. Physica D
**340**, 46–57 (2017)MathSciNetCrossRefGoogle Scholar - 40.Mackay, D., Matsugu, R.S.: Evaporation rates of liquid hydrocarbon spills on land and water. Can. J. Chem. Eng.
**51**, 434–439 (1973)CrossRefGoogle Scholar - 41.Majda, A.J., Harlim, J.: Physics constrained nonlinear regression models for time series. Nonlinearity
**26**, 201 (2013)ADSMathSciNetMATHCrossRefGoogle Scholar - 42.Matan, K., Williams, R.B., Witten, T.A., Nagel, S.R.: Crumpling a thin sheet. Phys. Rev. Lett.
**88**, 076101 (2002)ADSCrossRefGoogle Scholar - 43.Moghimi, S., Ramírez, J.M., Restrepo, J.M., Venkataramani, S.C.: Mass exchange dynamics of surface and subsurface oil in shallow-water transport. Preprint (2017)Google Scholar
- 44.Mori, H.: Transport, collective motion, and Brownian motion. Prog. Theor. Phys.
**33**, 423–455 (1965)ADSMATHCrossRefGoogle Scholar - 45.Oded, B., Rubinstein, S.M., Fineberg, J.: Slip-stick and the evolution of frictional strength. Nature
**463**, 76–9 (2010)ADSCrossRefGoogle Scholar - 46.Oppenheim, A.V., Schafer, R.W., Buck, J.R.: Discrete-Time Signal Processing, 2nd edn. Prentice-Hall Signal Processing Series. Prentice Hall, Englewood Cliffs (1999)Google Scholar
- 47.Ott, E.: Chaos in Dynamical Systems. Cambridge University Press, New York (1993)MATHGoogle Scholar
- 48.Polya, G., Szegö, G.: Problems and Theorems in Analysis II: Theory of Functions, Zeros, Polynomials, Determinants, Number Theory, Geometry, Classics in Mathematics. Springer, New York (1998)Google Scholar
- 49.Restrepo, J.M., Venkataramani, S.C., Dawson, C.: Nearshore sticky waters. Ocean Model.
**80**, 49–58 (2014)ADSCrossRefGoogle Scholar - 50.Restrepo, J.M., Ramírez, J.M., Venkataramani, S.C.: An oil fate model for shallow waters. J. Marine Sci. Eng.
**3**, 1504–1543 (2015)CrossRefGoogle Scholar - 51.Spaulding, M.L.: A state-of-the-art review of oil spill trajectory and fate modeling. Oil Chem. Pollut.
**4**, 39–55 (1988)CrossRefGoogle Scholar - 52.Stinis, P.: Renormalized Mori–Zwanzig-reduced models for systems without scale separation. Proc. R. Soc. Lond. Ser. A
**471**, 20140446 (2015)ADSMathSciNetCrossRefGoogle Scholar - 53.Stiver, W., Mackay, D.: Evaporation rate of spills of hydrocarbons and petroleum mixtures. Environ. Sci. Technol.
**18**(11), 834–840 (1984)ADSCrossRefGoogle Scholar - 54.Sutton, O.G.: Wind structure and evaporation in a turbulent atmosphere. Proc. R. Soc. Lond. Ser. A
**146**, 701–722 (1934)ADSMATHCrossRefGoogle Scholar - 55.Takens, E.: Detecting strange attractors in turbulence. In: Rand, D., Young, L.-S. (eds.) Dynamical Systems and Turbulence, pp. 366–381. Springer, Berlin (1981)Google Scholar
- 56.Transtrum, M.K., Machta, B.B., Sethna, J.P.: Geometry of nonlinear least squares with applications to sloppy models and optimization. Phys. Rev. E
**83**, 036701 (2011)ADSCrossRefGoogle Scholar - 57.Vautard, R., Yiou, P., Ghil, M.: Singular-spectrum analysis: a toolkit for short, Noisy chaotic signals. Physica D
**58**, 95–126 (1992)ADSCrossRefGoogle Scholar - 58.Vautard, R., Ghil, M.: Singular spectrum analysis in nonlinear dynamics, with applications to paleoclimatic time series. Physica D
**35**, 395–424 (1989)ADSMathSciNetMATHCrossRefGoogle Scholar - 59.Venturi, D., Cho, H., Karniadakis, G.: Mori–Zwanzig approach to uncertainty quantification. In: Ghanem, R., Higdon, D., Owhadi, H. (eds.) Handbook of Uncertainty Quantification. Springer, Heidelberg (2016)Google Scholar
- 60.Venturi, D., Karniadakis, G.: Convolutionless Nakajima–Zwanzig equations for stochastic analysis in nonlinear dynamical systems. Proc. R. Soc. Math. Phys. Eng. Sci.
**470**, 20130754–20130754 (2014)ADSMathSciNetCrossRefGoogle Scholar - 61.Walker, G.: On periodicity in series of related terms. Proc. R. Soc. Lond. Ser. A
**131**, 518–532 (1931)ADSMATHCrossRefGoogle Scholar - 62.Waterfall, J.J., Casey, F.P., Gutenkunst, R.N., Brown, K.S., Myers, C.R., Brouwer, P.W., Elser, V., Sethna, J.P.: Sloppy-model universality class and the Vandermonde matrix. Phys. Rev. Lett.
**97**, 150601 (2006)ADSCrossRefGoogle Scholar - 63.Yule, G.U.: On a method of investigating periodicities in disturbed series, with special reference to Wolfer’s sunspot numbers. Philos. Trans. R. Soc. Lond. Ser. A
**226**, 267–298 (1927)ADSMATHCrossRefGoogle Scholar - 64.Zwanzig, R.: Problems in nonlinear transport theory. In: Garrido, L. (ed.) Systems Far from Equilibrium, vol. 132. Lecture Notes in Physics. Springer, Berlin (1980)Google Scholar
- 65.Zwanzig, R.: Nonlinear generalized Langevin equations. J. Stat. Phys.
**9**, 215–220 (1973)ADSCrossRefGoogle Scholar - 66.Zwanzig, R.: Nonequilibrium Statistical Mechanics. Oxford University Press, New York (2001)MATHGoogle Scholar