Nonlinear Dynamics

, Volume 41, Issue 1–3, pp 47–67 | Cite as

Dimensionality Reduction Using Secant-Based Projection Methods: The Induced Dynamics in Projected Systems

  • D. S. Broomhead
  • M. J. Kirby


In previous papers we have developed an approach to the data reduction problem which is based on a well-known, constructive proof of Whitney’s embedding theorem [Broomhead, D. S. and Kirby, M., SIAM Journal of Applied Mathematics 60(6), 2000, 2114–2142; Broomhead, D. S. and Kirby, M., Neural Computation 13, 2001, 2595–2616]. This approach involves picking projections of the high-dimensional system which are optimised in the sense that they are easy to invert. This is done by considering the effect of the projections on the set of unit secants constructed from the data. In the present paper we discuss the implications of this idea in the case that the high-dimensional data is generated by a dynamical system. We ask if the existence of an easily invertible projection leads to practical methods for the construction of an equivalent, low-dimensional dynamical system. The paper consists of a review of the secant-based projection method and simple methods for finding good representations of the (nonlinear) inverse of the projections. We then discuss two variants of a way to find the dynamical system induced by a projection which lead to quite distinct numerical approximations. One of these is developed further as we describe various ways in which knowledge of the full dynamical system can be incorporated into the approximate projected system. The ideas of the paper are illustrated in some more or less simple examples, which range from a simple system of nonlinear ODEs which have an attracting limit cycle, to low-dimensional solutions of the Kuramoto–Sivashinsky equation which need many Galerkin modes for their description.

Key Words

reduced dynamical systems secant projections 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Kohonen, T., Self-Organization and Associative Memory, Springer, Berlin, 1984.Google Scholar
  2. 2.
    Linsker, R., ‘Self-organization in a perceptual network’, Computer 21, 1988, 105–117.CrossRefGoogle Scholar
  3. 3.
    Kelso, J. A. Scott, Dynamic Patterns : The Self-Organization of Brain and Behavior, MIT Press, Boston, MA, 1995.Google Scholar
  4. 4.
    Peitgen, H.-O., ‘Hartmut Jürgens, and Dietmar Saupe’, Chaos and Fractals: New Frontiers of Science, Springer, New York, 1992.Google Scholar
  5. 5.
    Constantin, P., Foias, C., Nicolaenko, B., and Temam, R., Integral Manifolds and Inertial Manifolds for Dissipative Partial Differential Equations, Springer, Berlin, Heidelberg, New York, 1989.Google Scholar
  6. 6.
    Jauberteau, F., Rosier, C., and Temam, R., ‘The nonlinear Galerkin method in computational fluid dynamics’, Applied Numerical Mathematics 6, 1989/1990, 361–370.CrossRefGoogle Scholar
  7. 7.
    Jolly, M. S., Kevrekidis, I. G., and Titi, E. S., ‘Approximate inertial manifolds for the Kuramoto-Sivashinsky equation: Analysis and computations’, Physica D 44, 1990, 38–60.Google Scholar
  8. 8.
    Marion, M. and Temam, R., ‘Nonlinear Galerkin methods’, SIAM Journal of Numerical Analysis 26, 1990, 1139–1157.CrossRefGoogle Scholar
  9. 9.
    Foias, C., Sell, G. R., and Temam, R., ‘Inertial manifolds for nonlinear evolutionary equations’, Journal of Differential Equations 73, 1988, 309–353.CrossRefGoogle Scholar
  10. 10.
    Sirovich, L., Knight, B. W., and Rodriguez, J. D., ‘Optimal low-dimensional dynamical approximations’, Quarterly of Applied Mathematics XLVIII, 1990, 535.Google Scholar
  11. 11.
    Landau, L. D. and Lifshitz, E. M., Fluid Mechanics, Pergamon Press, New York, 1959.Google Scholar
  12. 12.
    Sellers, W. D., ‘A statistical-dynamic approach to numerical weather prediction’, Science Report No. 2, Statistical Forecasting Project 2, MIT, Cambridge, MA, 1957.Google Scholar
  13. 13.
    Sirovich, L., ‘Turbulence and the dynamics of coherent structures, Part I: Coherent structures’, Quarterly of Applied Mathematics XLV(3), 1987, 561–571.Google Scholar
  14. 14.
    Sirovich, L., ‘Turbulence and the dynamics of coherent structures, Part II: Symmetries and transformations’, Quarterly of Applied Mathematics XLV(3), 1987, 573–582.Google Scholar
  15. 15.
    Sirovich, L., ‘Turbulence and the dynamics of coherent structures, Part III: Dynamics and scaling’, Quarterly of Applied Mathematics XLV(3), 1987, 583–590.Google Scholar
  16. 16.
    Aubry, N., Holmes, P., Lumley, J. L., and Stone, E., ‘Models for coherent structures in the wall layer’, Journal of Fluid Mechanics 192, 1988, 115–172.Google Scholar
  17. 17.
    Rodriguez, J. D. and Sirovich, L., ‘Low-dimensional dynamics for the complex Ginzburg–Landau equation’, Physica D 43, 1990, 77.Google Scholar
  18. 18.
    Berkooz, G., Philip, H., and Lumley, J. L., ‘The proper orthogonal decomposition in the analysis of turbulent flow’, Annual Review on Fluid Mechanics 25, 1993, 539–575.CrossRefGoogle Scholar
  19. 19.
    Karhunen, K., ‘Über lineare Methoden in der Wahrscheinlichkeitsrechnung’, Annales Academiae Scientiarum Fennicae 37, 1946.Google Scholar
  20. 20.
    Loève, M., Probability Theory, von Nostrand, Princeton, NJ, 1955.Google Scholar
  21. 21.
    Lumley, J. L., Stochastic Tools in Turbulence, Academic Press, New York, 1970.Google Scholar
  22. 22.
    Lorenz, E., ‘Empirical orthogonal eigenfunctions and statistical weather prediction’, Science Report No. 1, Statistical Forecasting Project 1, MIT, Cambridge, MA, 1956.Google Scholar
  23. 23.
    Kirby, M., Geometric Data Analysis: An Empirical Approach to Dimensionality Reduction and the Study of Patterns, Wiley, New York, 2001.Google Scholar
  24. 24.
    Kirby, M. and Armbruster, D. ‘Reconstructing phase-space from PDE simulations’, Zeitschrift für angewandte Mathematik und Physik 43, 1992, 999–1022.CrossRefGoogle Scholar
  25. 25.
    Kirby, M. and Miranda, R., ‘Nonlinear reduction of high-dimensional dynamical systems via neural networks’, Physics Review Letters 72(12), 1994, 1822–1825.CrossRefGoogle Scholar
  26. 26.
    Kirby, M. and Miranda, R., ‘The remodeling of chaotic dynamical systems’, in Intelligent Engineering Through Artificial Neural Networks, S. H. Dagli, B. R. Fernandez, J. Ghosh, and R. T. Soundar Kumara (eds.), ASME, New York, 1994, Number 4, pp. 831–836.Google Scholar
  27. 27.
    Kirby, M. and Miranda, R., ‘Empirical dynamical system reduction I: Global nonlinear transformations’, in Semi-analytic Methods for the Navier–Stokes Equations (Montreal, 1995), K. Coughlin (ed.), Vol. 20 of {CRM Proceedings Lecture Notes}, American Mathematical Society, Providence, RI, 1999, pp. 41–64.Google Scholar
  28. 28.
    Broomhead, D. S., Indik, R., Newell, A. C., and Rand, D. A., ‘Local adaptive Galerkin bases for large-dimensional dynamical systems’, Nonlinearity 4, 1991, 159–197.CrossRefGoogle Scholar
  29. 29.
    Hundley, D., Kirby, M., and Miranda, R., ‘Empirical dynamical system reduction II: Neural charts’, in Semi-Analytic Methods for the Navier–Stokes Equations (Montreal, 1995), K. Coughlin (ed.), Vol. 20 of {CRM Proceedings Lecture Notes}, American Mathematical Society, Providence, RI, 1999, pp. 65–83.Google Scholar
  30. 30.
    Hirsch, M. W., Differential Topology, Graduate Texts in Mathematics 33. Springer, Berlin, Heidelberg, New York, 1976.Google Scholar
  31. 31.
    Guillemin, V. and Pollack, A., Differential Topology, Prentice Hall, Englewood Cliffs, NJ, 1974.Google Scholar
  32. 32.
    Trefethen, L. N. and Bau III, D., Numerical Linear Algebra, SIAM, Philadelphia, PA, 1997.Google Scholar
  33. 33.
    Broomhead, D. S. and Kirby, M., ‘A new approach for dimensionality reduction: Theory and algorithms’, SIAM Journal of Applied Mathematics 60(6), 2000, 2114–2142.CrossRefMathSciNetGoogle Scholar
  34. 34.
    Broomhead, D. S. and Kirby, M., ‘The Whitney reduction network: A method for computing autoassociative graphs’, Neural Computation 13, 2001, 2595–2616.CrossRefPubMedGoogle Scholar
  35. 35.
    Edelman, A., Arias, T. A., and Smith, S. T., ‘The geometry of algorithms with orthogonality constraints’, SIAM Journal on Matrix Analysis and Applications 20, 1998, 303–353.CrossRefGoogle Scholar
  36. 36.
    Broomhead, D. S. and David, L., ‘Multivariable functional interpolation and adaptive networks’, Complex Systems 2, 1988, 321–355.MathSciNetGoogle Scholar
  37. 37.
    Krischer, K., Rico-Martinez, R., Kevrikidis, I. G., Rotermund, H. H., Ertl, G., and Hudson, J. L., ‘Model identification of a spatiotemporally varying catalytic reaction’, AIChE Journal 39(1), 1993, 89–98.CrossRefGoogle Scholar
  38. 38.
    Hyman, J. M., Nicolaenko, B., and Zaleski, S., ‘Order and complexity in the Kuramoto–Sivashinsky model of weakly turbulent interfaces’, Physica D 23, 1986, 265–292.Google Scholar
  39. 39.
    Hyman, J. M. and Nicolaenko, B., ‘The Kuramoto–Sivashinsky equation: A bridge between PDE’s and dynamical systems’, Technical Report LA-UR 85-1556, Los Alamos National Laboratory, 1985.Google Scholar
  40. 40.
    Kevrikidis, I. G., Nicolaenko, B., and Scovel, C., ‘Back in the saddle again: A computer assisted study of the Kuramoto–Sivashinsky equation’, SIAM Journal of Applied Mathematics 50, 1990, 760–790.CrossRefGoogle Scholar

Copyright information

© Springer Science + Business Media, Inc. 2005

Authors and Affiliations

  1. 1.Department of MathematicsUMISTManchesterU.K.
  2. 2.Department of MathematicsColorado State UniversityFort CollinsU.S.A.

Personalised recommendations