Skip to main content
SpringerLink
Log in

Dimensionality Reduction Using Secant-Based Projection Methods: The Induced Dynamics in Projected Systems

  • Published:
Nonlinear Dynamics Aims and scope Submit manuscript

Abstract

In previous papers we have developed an approach to the data reduction problem which is based on a well-known, constructive proof of Whitney’s embedding theorem [Broomhead, D. S. and Kirby, M., SIAM Journal of Applied Mathematics 60(6), 2000, 2114–2142; Broomhead, D. S. and Kirby, M., Neural Computation 13, 2001, 2595–2616]. This approach involves picking projections of the high-dimensional system which are optimised in the sense that they are easy to invert. This is done by considering the effect of the projections on the set of unit secants constructed from the data. In the present paper we discuss the implications of this idea in the case that the high-dimensional data is generated by a dynamical system. We ask if the existence of an easily invertible projection leads to practical methods for the construction of an equivalent, low-dimensional dynamical system. The paper consists of a review of the secant-based projection method and simple methods for finding good representations of the (nonlinear) inverse of the projections. We then discuss two variants of a way to find the dynamical system induced by a projection which lead to quite distinct numerical approximations. One of these is developed further as we describe various ways in which knowledge of the full dynamical system can be incorporated into the approximate projected system. The ideas of the paper are illustrated in some more or less simple examples, which range from a simple system of nonlinear ODEs which have an attracting limit cycle, to low-dimensional solutions of the Kuramoto–Sivashinsky equation which need many Galerkin modes for their description.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Kohonen, T., Self-Organization and Associative Memory, Springer, Berlin, 1984.

    Google Scholar 

  2. Linsker, R., ‘Self-organization in a perceptual network’, Computer 21, 1988, 105–117.

    Article  Google Scholar 

  3. Kelso, J. A. Scott, Dynamic Patterns : The Self-Organization of Brain and Behavior, MIT Press, Boston, MA, 1995.

    Google Scholar 

  4. Peitgen, H.-O., ‘Hartmut Jürgens, and Dietmar Saupe’, Chaos and Fractals: New Frontiers of Science, Springer, New York, 1992.

    Google Scholar 

  5. Constantin, P., Foias, C., Nicolaenko, B., and Temam, R., Integral Manifolds and Inertial Manifolds for Dissipative Partial Differential Equations, Springer, Berlin, Heidelberg, New York, 1989.

    Google Scholar 

  6. Jauberteau, F., Rosier, C., and Temam, R., ‘The nonlinear Galerkin method in computational fluid dynamics’, Applied Numerical Mathematics 6, 1989/1990, 361–370.

    Article  Google Scholar 

  7. Jolly, M. S., Kevrekidis, I. G., and Titi, E. S., ‘Approximate inertial manifolds for the Kuramoto-Sivashinsky equation: Analysis and computations’, Physica D 44, 1990, 38–60.

    Google Scholar 

  8. Marion, M. and Temam, R., ‘Nonlinear Galerkin methods’, SIAM Journal of Numerical Analysis 26, 1990, 1139–1157.

    Article  Google Scholar 

  9. Foias, C., Sell, G. R., and Temam, R., ‘Inertial manifolds for nonlinear evolutionary equations’, Journal of Differential Equations 73, 1988, 309–353.

    Article  Google Scholar 

  10. Sirovich, L., Knight, B. W., and Rodriguez, J. D., ‘Optimal low-dimensional dynamical approximations’, Quarterly of Applied Mathematics XLVIII, 1990, 535.

    Google Scholar 

  11. Landau, L. D. and Lifshitz, E. M., Fluid Mechanics, Pergamon Press, New York, 1959.

    Google Scholar 

  12. Sellers, W. D., ‘A statistical-dynamic approach to numerical weather prediction’, Science Report No. 2, Statistical Forecasting Project 2, MIT, Cambridge, MA, 1957.

  13. Sirovich, L., ‘Turbulence and the dynamics of coherent structures, Part I: Coherent structures’, Quarterly of Applied Mathematics XLV(3), 1987, 561–571.

    Google Scholar 

  14. Sirovich, L., ‘Turbulence and the dynamics of coherent structures, Part II: Symmetries and transformations’, Quarterly of Applied Mathematics XLV(3), 1987, 573–582.

    Google Scholar 

  15. Sirovich, L., ‘Turbulence and the dynamics of coherent structures, Part III: Dynamics and scaling’, Quarterly of Applied Mathematics XLV(3), 1987, 583–590.

    Google Scholar 

  16. Aubry, N., Holmes, P., Lumley, J. L., and Stone, E., ‘Models for coherent structures in the wall layer’, Journal of Fluid Mechanics 192, 1988, 115–172.

    Google Scholar 

  17. Rodriguez, J. D. and Sirovich, L., ‘Low-dimensional dynamics for the complex Ginzburg–Landau equation’, Physica D 43, 1990, 77.

    Google Scholar 

  18. Berkooz, G., Philip, H., and Lumley, J. L., ‘The proper orthogonal decomposition in the analysis of turbulent flow’, Annual Review on Fluid Mechanics 25, 1993, 539–575.

    Article  Google Scholar 

  19. Karhunen, K., ‘Über lineare Methoden in der Wahrscheinlichkeitsrechnung’, Annales Academiae Scientiarum Fennicae 37, 1946.

  20. Loève, M., Probability Theory, von Nostrand, Princeton, NJ, 1955.

    Google Scholar 

  21. Lumley, J. L., Stochastic Tools in Turbulence, Academic Press, New York, 1970.

    Google Scholar 

  22. Lorenz, E., ‘Empirical orthogonal eigenfunctions and statistical weather prediction’, Science Report No. 1, Statistical Forecasting Project 1, MIT, Cambridge, MA, 1956.

  23. Kirby, M., Geometric Data Analysis: An Empirical Approach to Dimensionality Reduction and the Study of Patterns, Wiley, New York, 2001.

    Google Scholar 

  24. Kirby, M. and Armbruster, D. ‘Reconstructing phase-space from PDE simulations’, Zeitschrift für angewandte Mathematik und Physik 43, 1992, 999–1022.

    Article  Google Scholar 

  25. Kirby, M. and Miranda, R., ‘Nonlinear reduction of high-dimensional dynamical systems via neural networks’, Physics Review Letters 72(12), 1994, 1822–1825.

    Article  Google Scholar 

  26. Kirby, M. and Miranda, R., ‘The remodeling of chaotic dynamical systems’, in Intelligent Engineering Through Artificial Neural Networks, S. H. Dagli, B. R. Fernandez, J. Ghosh, and R. T. Soundar Kumara (eds.), ASME, New York, 1994, Number 4, pp. 831–836.

  27. Kirby, M. and Miranda, R., ‘Empirical dynamical system reduction I: Global nonlinear transformations’, in Semi-analytic Methods for the Navier–Stokes Equations (Montreal, 1995), K. Coughlin (ed.), Vol. 20 of {CRM Proceedings Lecture Notes}, American Mathematical Society, Providence, RI, 1999, pp. 41–64.

  28. Broomhead, D. S., Indik, R., Newell, A. C., and Rand, D. A., ‘Local adaptive Galerkin bases for large-dimensional dynamical systems’, Nonlinearity 4, 1991, 159–197.

    Article  Google Scholar 

  29. Hundley, D., Kirby, M., and Miranda, R., ‘Empirical dynamical system reduction II: Neural charts’, in Semi-Analytic Methods for the Navier–Stokes Equations (Montreal, 1995), K. Coughlin (ed.), Vol. 20 of {CRM Proceedings Lecture Notes}, American Mathematical Society, Providence, RI, 1999, pp. 65–83.

  30. Hirsch, M. W., Differential Topology, Graduate Texts in Mathematics 33. Springer, Berlin, Heidelberg, New York, 1976.

  31. Guillemin, V. and Pollack, A., Differential Topology, Prentice Hall, Englewood Cliffs, NJ, 1974.

    Google Scholar 

  32. Trefethen, L. N. and Bau III, D., Numerical Linear Algebra, SIAM, Philadelphia, PA, 1997.

    Google Scholar 

  33. Broomhead, D. S. and Kirby, M., ‘A new approach for dimensionality reduction: Theory and algorithms’, SIAM Journal of Applied Mathematics 60(6), 2000, 2114–2142.

    Article  MathSciNet  Google Scholar 

  34. Broomhead, D. S. and Kirby, M., ‘The Whitney reduction network: A method for computing autoassociative graphs’, Neural Computation 13, 2001, 2595–2616.

    Article  CAS  PubMed  Google Scholar 

  35. Edelman, A., Arias, T. A., and Smith, S. T., ‘The geometry of algorithms with orthogonality constraints’, SIAM Journal on Matrix Analysis and Applications 20, 1998, 303–353.

    Article  Google Scholar 

  36. Broomhead, D. S. and David, L., ‘Multivariable functional interpolation and adaptive networks’, Complex Systems 2, 1988, 321–355.

    MathSciNet  Google Scholar 

  37. Krischer, K., Rico-Martinez, R., Kevrikidis, I. G., Rotermund, H. H., Ertl, G., and Hudson, J. L., ‘Model identification of a spatiotemporally varying catalytic reaction’, AIChE Journal 39(1), 1993, 89–98.

    Article  CAS  Google Scholar 

  38. Hyman, J. M., Nicolaenko, B., and Zaleski, S., ‘Order and complexity in the Kuramoto–Sivashinsky model of weakly turbulent interfaces’, Physica D 23, 1986, 265–292.

    Google Scholar 

  39. Hyman, J. M. and Nicolaenko, B., ‘The Kuramoto–Sivashinsky equation: A bridge between PDE’s and dynamical systems’, Technical Report LA-UR 85-1556, Los Alamos National Laboratory, 1985.

  40. Kevrikidis, I. G., Nicolaenko, B., and Scovel, C., ‘Back in the saddle again: A computer assisted study of the Kuramoto–Sivashinsky equation’, SIAM Journal of Applied Mathematics 50, 1990, 760–790.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, UMIST, P.O. Box 88, Manchester, M60 1QD, U.K.

    D. S. Broomhead

  2. Department of Mathematics, Colorado State University, Fort Collins, CO, 80523, U.S.A.

    M. J. Kirby

Authors
  1. D. S. Broomhead
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. J. Kirby
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to D. S. Broomhead.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Broomhead, D.S., Kirby, M.J. Dimensionality Reduction Using Secant-Based Projection Methods: The Induced Dynamics in Projected Systems. Nonlinear Dyn 41, 47–67 (2005). https://doi.org/10.1007/s11071-005-2792-1

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11071-005-2792-1

Key Words

Search

Navigation