A Survey of Continuous-Time Computation Theory

  • Pekka Orponen


Motivated partly by the resurgence of neural computation research, and partly by advances in device technology, there has been a recent increase of interest in analog, continuous-time computation. However, while special-case algorithms and devices are being developed, relatively little work exists on the general theory of continuous- time models of computation. In this paper, we survey the existing models and results in this area, and point to some of the open research questions.


Cellular Automaton Turing Machine Neural Computation Differential Analyzer Hopfield Network 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    L. M. Adleman, Molecular computation of solutions to combinatorial problems. Science 266 (11 Nov. 1994 ), 1021 – 1024.CrossRefGoogle Scholar
  2. [2]
    J. A. Anderson and E. Rosenfeld (eds.), Neurocomputing: Foundations of Research. The MIT Press, Cambridge, MA, 1988.Google Scholar
  3. [3]
    E. Asarin, O. Maler, On some relations between dynamical systems and transition systems. Proc. 21st Internat. Colloq. on Automata, Languages, and Programming, 59-72. Lecture Notes in Computer Science 820, Springer-Verlag, Berlin, 1994.Google Scholar
  4. [4]
    C. H. Bennett, Logical reversibility of computation. IBM J. Res. Develop. 17 (1973), 525 – 532.MathSciNetMATHCrossRefGoogle Scholar
  5. [5]
    C. H. Bennett, Time/space trade-offs for reversible computation. SIAM J. Comput. 18 (1989), 766–776.MathSciNetMATHCrossRefGoogle Scholar
  6. [6]
    E. Bernstein, U. Vazirani, Quantum complexity theory. Proc. 25th ACM Symp. on Theory of Computation, 11–20. ACM Press, New York, NY, 1993.Google Scholar
  7. [7]
    L. Blum, M. Shub, S. Smale, On a theory of computation over the real numbers: NP-completeness, recursive functions and universal machines. Bulletin of the Amer. Math. Soc. 21 (1989), 1–46.MathSciNetMATHCrossRefGoogle Scholar
  8. [8]
    M. Branicky, Analog computation with continuous ODE’ss. Proc. Workshop on Physics and Computation 1994, 265–274. IEEE Computer Society Press, Los Alamitos, CA, 1994.Google Scholar
  9. [9]
    M. Branicky, Universal computation and other capabilities of hybrid and continuous dynamical systems. Theoret. Comput. Sci. 138 (1995), 67–100.MathSciNetMATHCrossRefGoogle Scholar
  10. [10]
    R. W. Brockett, Smooth dynamical systems which realize arithmetical and logical operations. Three Decades of Mathematical System Theory (H. Nijmeijer, J. M. Schumacher, eds.) Lecture Notes in Control and Information Sciences 135, Springer-Verlag, Berlin, 1989.Google Scholar
  11. [11]
    R. W. Brockett, Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems. Linear Algebra and Its Applications 146 (1991), 79–91.MathSciNetMATHCrossRefGoogle Scholar
  12. [12]
    V. Bush, The differential analyzer, a new machine for solving differential equations. J. Franklin Inst. 212 (1931), 447–488.CrossRefGoogle Scholar
  13. [13]
    M. Casey, The dynamics of discrete-time computation, with application to recurrent neural networks and finite state machine extraction. Neural Computation 8 (1996), 1135–1178.CrossRefGoogle Scholar
  14. [14]
    A. Cichocki, R. Unbehauen, Neural Networks for Optimization and Signal Processing. Wiley/Teubner, Stuttgart, 1993.MATHGoogle Scholar
  15. [15]
    Hopfield, J. J. and Tank, D. W. Neural computation of decisions in optimization problems. Biological Cybernetics 52 (1985), 141–152.MathSciNetMATHGoogle Scholar
  16. [16]
    D. G. Feitelson, Optical Computing: A Survey for Computer Scientists. The MIT Press, Cambridge, MA, 1988.Google Scholar
  17. [17]
    M. Garzon, Models of Massive Parallelism: Analysis of Cellular Automata and Neural Networks. Springer-Verlag, Berlin, 1995.MATHGoogle Scholar
  18. [18]
    A. Hausner, Analog and Analog/Hybrid Computer Programming. Prentice-Hall, Englewood Cliffs, NJ, 1971.MATHGoogle Scholar
  19. [19]
    M. W. Hirsch, S. Smale, Differential Equations, Dynamical Systems, and Linear Algebra. Academic Press, San Diego, CA, 1974.MATHGoogle Scholar
  20. [20]
    J. J. Hopfield, Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Nat. Acad. Sci. USA 81 (1984), 3088–3092. Reprinted in [2], pp. 579–583.Google Scholar
  21. [21]
    J. J. Hopfield, D. W. Tank, Neural computation of decisions in optimization problems. Biological Cybernetics 52 (1985), 141–152.MathSciNetMATHGoogle Scholar
  22. [22]
    A. S. Jackson, Analog Computation. McGraw-Hill, New York, NY, 1960.MATHGoogle Scholar
  23. [23]
    C. L. Johnson, Analog Computer Techniques, 2nd Ed. McGraw-Hill, New York, NY, 1963.Google Scholar
  24. [24]
    P. Koiran, Dynamics of discrete time, continuous state Hopfield networks. Neural Computation 6 (1994), 459–468.CrossRefGoogle Scholar
  25. [25]
    P. Koiran, M. Cosnard, M. Garzon, Computability with low- dimensional dynamical systems. Theoret. Comput. Sci. 132 (1994), 113–128.MathSciNetMATHCrossRefGoogle Scholar
  26. [26]
    G. A. Korn, T. M. Korn, Electronic Analog and Hybrid Computers, 3rd Ed. McGraw-Hill, New York, NY, 1964.Google Scholar
  27. [27]
    L. Lipshitz, L. Rubel, A differentially algebraic replacement theorem, and analog computability. Proc. Amer. Math. Soc. 99 (1987), 367–372.MathSciNetMATHCrossRefGoogle Scholar
  28. [28]
    R. J. Lipton, DNA solution of hard computational problems. Science 268 (28 Apr. 1995 ), 542–545.Google Scholar
  29. [29]
    W. Maass, P. Orponen, On the effect of analog noise in discrete- time analog computation. Proc. Neural Information Processing Systems 1996, to appear.Google Scholar
  30. [30]
    C. Mead, Analog VLSI and Neural Systems. Addison-Wesley, Reading, MA, 1989.MATHGoogle Scholar
  31. [31]
    C. Moore, Unpredictability and undecidability in physical systems. Phys. Review Letters 64 (1990), 2354–2357.MATHCrossRefGoogle Scholar
  32. [32]
    C. Moore, Generalized shifts: unpredictability and undecidability in dynamical systems. Nonlinearity 4 (1991), 199–230.MathSciNetMATHCrossRefGoogle Scholar
  33. [33]
    S. Omohundro, Modelling cellular automata with partial differential equations. Physica 10D (1984), 128–134.MathSciNetGoogle Scholar
  34. [34]
    P. Orponen, On the Computational Power of Continuous Time Neural Networks. Project NeuroCOLT Report NC-TR-95-051, Royal Holloway College, Univ. of London, Dept. of Computer Science, 1995. 18 pp.Google Scholar
  35. [35]
    P. Orponen, The computational power of discrete Hopfield nets with hidden units. Neural Computation 8 (1996), 403–415.CrossRefGoogle Scholar
  36. [36]
    P. Orponen, M. Matamala, Universal computation by finite two- dimensional coupled map lattices. Proc. Workshop on Physics and Computation 1996, to appear.Google Scholar
  37. [37]
    J. Palis, Jr., W. de Melo, Geometric Theory of Dynamical Systems: An Introduction. Springer-Verlag, New York, NY, 1982.Google Scholar
  38. [38]
    M. B. Pour-El, Abstract computability and its relation to the general purpose analog computer (some connections between logic, differential equations and analog computers). Trans. Amer. Math. Soc. 199 (1974), 1–28.MathSciNetMATHCrossRefGoogle Scholar
  39. [39]
    M. B. Pour-El, J. I. Richards, Computability in Analysis and Physics. Springer-Verlag, Berlin, 1989.MATHGoogle Scholar
  40. [40]
    P. Pudlák, Complexity theory and genetics. Proc. 9th Ann. IEEE Conf. on Structure in Complexity Theory, 383–395. IEEE Computer Society Press, Los Alamitos, CA, 1994.Google Scholar
  41. [41]
    J. H. Reif, J. D. Tygar, A. Yoshida, The computability and complexity of optical beam tracing. Proc. 31st Ann. IEEE Symp. on Foundations of Computer Science, 106–114. IEEE Computer Society Press, Los Alamitos, CA, 1990.Google Scholar
  42. [42]
    D. Rooß, K. Wagner, On the Power of Bio-Computers. Technical Report, Universität Würzburg, Inst, für Informatik, Feb. 1995.Google Scholar
  43. [43]
    L. A. Rubel, Some mathematical limitations of the general-purpose analog computer. Adv. in Appl. Math. 9 (1988), 22–34.MathSciNetMATHCrossRefGoogle Scholar
  44. [44]
    L. A. Rubel, The extended analog computer. Adv. in Appl. Math. 14 (1993), 39–50.MathSciNetMATHCrossRefGoogle Scholar
  45. [45]
    E. Sánchez-Sinencio, C. Lau, Artificial Neural Networks: Paradigms, Applications, and Hardware Implementations. IEEE Press, New York, 1992.Google Scholar
  46. [46]
    C. E. Shannon, Mathematical theory of the differential analyzer. J. Math. Phys. MIT 20 (1941), 337–354. Reprinted in [48], 496–513.Google Scholar
  47. [47]
    C. E. Shannon, The theory and design of linear differential equation machines. Report to the National Defense Research Council, January 1942. Reprinted in [48], pp. 514–559.Google Scholar
  48. [48]
    C. E. Shannon, Collected Papers (N. J. A. Sloane, A. Wyner, eds.). IEEE Press, Piscataway, NJ, 1993.Google Scholar
  49. [49]
    P. Shor, Algorithms for quantum computation: discrete logarithms and factoring. Proc. 35th Ann. IEEE Symp. on Foundations of Computer Science, 124–134. IEEE Computer Society Press, Los Alamitos, CA, 1994.Google Scholar
  50. [50]
    H. T. Siegelmann, S. Fishman, Analog computing and dynamical systems. Manuscript, April 1996. 34 pp.Google Scholar
  51. [51]
    H. T. Siegelmann, E. D. Sontag, On the computational power of neural nets. J. Comput. System Sciences 50 (1995), 132–150.MathSciNetMATHCrossRefGoogle Scholar
  52. [52]
    A. Vergis, K. Steiglitz, B. Dickinson, The complexity of analog computation. Math, and Computers in Simulation 28 (1986), 91–113.MATHCrossRefGoogle Scholar

Copyright information

© Kluwer Academic Publishers 1997

Authors and Affiliations

  • Pekka Orponen
    • 1
  1. 1.Department of MathematicsUniversity of JyväskyläJyväskyläFinland

Personalised recommendations