The Role of Structure and Complexity on Reservoir Computing Quality
We explore the effect of structure and connection complexity on the dynamical behaviour of Reservoir Computers (RC). At present, considerable effort is taken to design and hand-craft physical reservoir computers. Both structure and physical complexity are often pivotal to task performance, however, assessing their overall importance is challenging. Using a recently proposed framework, we evaluate and compare the dynamical freedom (referring to quality) of neural network structures, as an analogy for physical systems. The results quantify how structure affects the range of behaviours exhibited by these networks. It highlights that high quality reached by more complex structures is often also achievable in simpler structures with greater network size. Alternatively, quality is often improved in smaller networks by adding greater connection complexity. This work demonstrates the benefits of using abstract behaviour representation, rather than evaluation through benchmark tasks, to assess the quality of computing substrates, as the latter typically has biases, and often provides little insight into the complete computing quality of physical systems.
KeywordsReservoir computing Unconventional computing Echo state networks Structure Complexity
This work is part of the SpInspired project, funded by EPSRC grant EP/R032823/1. Jack Dewhirst is funded by an EPSRC DTP PhD studentship.
- 6.Dale, M.: Neuroevolution of hierarchical reservoir computers. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 410–417. ACM (2018)Google Scholar
- 8.Dale, M., Miller, J.F., Stepney, S., Trefzer, M.A.: Reservoir computing in materio: an evaluation of configuration through evolution. In: 2016 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1–8, December 2016Google Scholar
- 9.Dale, M., Miller, J.F., Stepney, S., Trefzer, M.A.: Reservoir computing in materio: a computational framework for in materio computing. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 2178–2185, May 2017Google Scholar
- 10.Dale, M., Miller, J.F., Stepney, S., Trefzer, M.A.: A substrate-independent framework to characterise reservoir computers. arXiv preprint arXiv:1810.07135 (2018)
- 13.Jaeger, H.: The “echo state” approach to analysing and training recurrent neural networks-with an erratum note. German National Research Center for Information Technology GMD Technical Report 148:34, Bonn, Germany (2001)Google Scholar
- 14.Jaeger, H.: Short term memory in echo state networks. GMD-Forschungszentrum Informationstechnik (2001)Google Scholar
- 17.Lehman, J., Stanley, K.O.: Exploiting open-endedness to solve problems through the search for novelty. In: ALIFE, pp. 329–336 (2008)Google Scholar
- 21.Paquot, Y., et al.: Optoelectronic reservoir computing. Scientific Reports, 2 (2012)Google Scholar
- 26.Schrauwen, B., Verstraeten, D., Van Campenhout, J.: An overview of reservoir computing: theory, applications and implementations. In: Proceedings of the 15th European Symposium on Artificial Neural Networks. Citeseer (2007)Google Scholar
- 28.Tanaka, G., et al.: Recent advances in physical reservoir computing: a review. arXiv preprint arXiv:1808.04962 (2018)