A Recurrent Neural Network for Time-series Modelling

  • Abhay B. Bulsari
  • Henrik Saxén


This paper describes an architecture for discrete time feedback neural networks. Some analytical results for networks with linear nodal activation functions are derived, while simulations demonstrate the performance of recurrent networks with nonlinear (sigmoidal) activation functions. The need for models with a capacity to consider correlated noise sequences is pointed out, and it is shown that the recurrent networks can perform state estimation and entertain models with coloured noise.


Finite Impulse Response Recurrent Neural Network Feedforward Network Infinite Impulse Response Recurrent Network 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Pineda, F.J., (1988) “Dynamics and Architecture for Neural Computation”, Journal of Complexity 4, 216–245.MathSciNetMATHCrossRefGoogle Scholar
  2. [2]
    Lapedes, A. and R. Farber, (1987) “How Neural Nets Work”, Neural Information Processing Systems (ed. D.Z. Anderson) 442–456, American Institute of Physics, New York.Google Scholar
  3. [3]
    Sejnowski, T.J. and C.R. Rosenberg, (1987) “Parallel Network that Learn to Pronounce English Text”, Complex Systems 1, 145–168.MATHGoogle Scholar
  4. [4]
    Jordan, M.I., (1986) “Attractor dynamics and parallelism in a connectionist sequential machine”, Proceedings of the Eight Annual Conference of the Cognitive Science Society, Amherst, p. 531–546. Hillsdale: Erlbaum.Google Scholar
  5. [5]
    Elman, J.L,(1990) “Finding Structure in Time”, Cognitive Science 14, 179–211.Google Scholar
  6. [6]
    Cleeremans, A., D. Servan-Schreiber and J.L. McClelland, (1989) “Finite State Automata and Simple Recurrent Networks”, Neural Computation 1, 372–381.CrossRefGoogle Scholar
  7. [7]
    Croft, E.A. and J.P. Huissoon, (1992) “Neural Network Controller for an Autonomous Guided Vehicle”, Proceedings of 11th IASTED International Conference: Modelling, Identification and Control (Ed. M.H. Hamza), p. 46-49, Acta Press, Zurich, 1992.Google Scholar
  8. [8]
    Pham, D.T. and X. Liu, (1992) “Dynamic System Modelling Using Partially Recurrent Neural Networks”, Journal of System Engineering 2, 90–97.Google Scholar
  9. [9]
    Mozer, M.C., (1989) “A Focused Back-Propagation Algorithm for Temporal Pattern Recognition”, Complex Systems 3, 349–381.MathSciNetMATHGoogle Scholar
  10. [10]
    Stornetta, W.S, T. Hogg and B.A. Huberman, (1987) “A Dynamic Approach to Temporal Pattern Recognition”, Neural Information Processing Systems (ed. D.Z. Anderson) 750–759, American Institute of Physics, New York.Google Scholar
  11. [11]
    Back, A.D. and A.C. Tsoi, (1991) “FIR and IIR Synapses, a new neural network architecture for time series modelling”, Neural Computation 3, 375–385.CrossRefGoogle Scholar
  12. [12]
    Rumelhart, D.E., G.E. Hinton and R.J. Williams, (1986) “Learning Internal Representations by Error Propagation”, In Parallel Distributed Processing, Vol. 1, MIT Press.Google Scholar
  13. [13]
    Marquardt, D.W., (1963) “An algorithm for least-squares estimation of nonlinear parameters”, J. SIAM 11, 431–441.MathSciNetMATHGoogle Scholar
  14. [14]
    Bulsari, A.B. and H. Saxén, (1991) “System identification of a biochemical process using feedforward neural networks”, Neurocomputing 3, 125–133.MATHCrossRefGoogle Scholar
  15. [15]
    Box, G.E.P. and G.M. Jenkins, (1976) Time Series Analysis. Forecasting and Control, Holden-Day, Oakland, CA.Google Scholar
  16. [16]
    Aoki, M., (1987) State Space Modeling of Time Series, Springer-Verlag, New York.MATHGoogle Scholar
  17. [17]
    Bennett, R.J., (1979) Spatial Time Series, Pion Limited, London, England.Google Scholar
  18. [18]
    Bulsari A.B. and H. Saxén, (1991) “A class of partially recurrent neural networks in discrete time”, Technical report 91–17, Heat Eng. Lab., Åbo Akademi, Åbo, Finland, 1991.Google Scholar
  19. [19]
    Pandit, S.M. and S.M. Wu., (1983) Time Series Analysis with Applications, John Wiley and Sons, New York.MATHGoogle Scholar
  20. [20]
    Pearlmutter, B., (1989) “Learning state space trajectories in recurrent neural networks”, Neural Computation 1, 263–269.CrossRefGoogle Scholar
  21. [21]
    Pearlmutter, B., (1990) “Dynamic Recurrent Neural Networks”, Technical report CMU-CS-90-196, Carnegie Mellon University, Pittsburgh, USA.Google Scholar
  22. [22]
    Bulsari, A.B. and H. Saxén, (1992) “A Partially Recurrent Connectionist Model”, Proc. of the 10th European Conf. on Artificial Intelligence (ECAI’92), (Ed. B. Neumann), Vienna, Austria, 1992, p. 198-202.Google Scholar
  23. [23]
    Williams, R.J. and D. Zipser, (1989) “Experimental Analysis of the Real-time Recurrent Learning Algorithm”, Connection Science 1, 87–111.CrossRefGoogle Scholar
  24. [24]
    Billings, S.A., H.B. Jamaluddin and S. Chen, (1992) “Properties of neural networks with applications to modelling non-linear dynamical systems”, Int. J. Control 55 193–224.MathSciNetMATHCrossRefGoogle Scholar
  25. [25]
    Bulsari, A.B. and H. Saxén, (1992) “A Recurrent Neural Network Model”, in Artificial Neural Networks, 2 (Eds. I. Aleksander and J. Taylor) (Proceedings of the International Conference on Artificial Neural Networks, Brighton, UK, September 1992), Vol. 2, p. 1091–1094.Google Scholar

Copyright information

© Springer-Verlag/Wien 1993

Authors and Affiliations

  • Abhay B. Bulsari
    • 1
  • Henrik Saxén
    • 1
  1. 1.Heat Engineering Laboratory Department of Chemical EngineeringÅbo AkademiÅboFinland

Personalised recommendations