Abstract
It is an open question how spontaneous or “existing” activity plays a constructive role in information processing of the central nervous systems in the brain. In this study, using the FORCE learning framework, we investigate the problem of multiple temporal pattern generations by a single recurrent neural network (RNN) pushed by appropriate combinations of input pulses. We show that weak chaos meaning that the maximal Lyapunov exponent is small but strictly positive (this is not the so-called “edge of chaos”) is required as a generic property of the RNN for giving optimal performances. Furthermore, such a network exhibits intermittent switching processes among several quasi-stable patterns when there is no external inputs, which reminds us a prominent behavior of the high-dimensional nonlinear dynamical systems called chaotic itinerancy. We also characterize this itinerant dynamics in terms of the fluctuations of the finite-time Lyapunov exponents and the eigenvalue spectra of the recurrent weight matrices after learning.
Supported by MEXT KAKENHI Grant Numbers 16H0167 and 18H05136, and by JSPS KAKENHI Grant Number 19H041183.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Hebb, D.: The Organization of Behavior: A Neuropsychological Theory. Wiley, Hoboken (1949)
Jäger, H.: The echo state approach to analysing and training recurrent neural networks. GMD Technical report 148, 13, German National Research Center for Information Technology, Bonn, Germany (2001)
Kaneko, K., Tsuda, I.: Chaotic itinerancy. Chaos 13, 926 (2003). https://doi.org/10.1063/1.1607783
Kenet, T., Bibitchkov, D., Tsodyks, M., Grinvald, A., Arieli, A.: Spontaneously emerging cortical representations of visual attributes. Nature 425, 954 (2003). https://doi.org/10.1038/nature02078
Luczak, A., Barthó, P., Harris, K.D.: Spontaneous events outline the realm of possible sensory responses in neocortical population. Neuron 62, 413 (2009). https://doi.org/10.1016/j.neuron.2009.03.014
Maas, W., Natschläger, T., Markram, H.: Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2351 (2002). https://doi.org/10.1162/089976602760407955
Suetani, H.: Weak sensitivity to initial conditions for generating temporal patterns in recurrent neural networks: a reservoir computing approach. In: Sanayei, A., Rössler, O., Zelinka, I. (eds.) Interdisciplinary Symposium on Complex Systems (ISCS), vol. 13, pp. 47–55. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-10759-2_6
Sussillo, D., Abbott, L.: Generating coherent patterns of activity from chaotic neural networks. Neuron 63, 544 (2009). https://doi.org/10.1016/j.neuron.2009.07.018
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Suetani, H. (2019). Multiple Pattern Generations and Chaotic Itinerant Dynamics in Reservoir Computing. In: Tetko, I., Kůrková, V., Karpov, P., Theis, F. (eds) Artificial Neural Networks and Machine Learning – ICANN 2019: Workshop and Special Sessions. ICANN 2019. Lecture Notes in Computer Science(), vol 11731. Springer, Cham. https://doi.org/10.1007/978-3-030-30493-5_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-30493-5_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30492-8
Online ISBN: 978-3-030-30493-5
eBook Packages: Computer ScienceComputer Science (R0)