Abstract
We explore the effect of structure and connection complexity on the dynamical behaviour of Reservoir Computers (RC). At present, considerable effort is taken to design and hand-craft physical reservoir computers. Both structure and physical complexity are often pivotal to task performance, however, assessing their overall importance is challenging. Using a recently proposed framework, we evaluate and compare the dynamical freedom (referring to quality) of neural network structures, as an analogy for physical systems. The results quantify how structure affects the range of behaviours exhibited by these networks. It highlights that high quality reached by more complex structures is often also achievable in simpler structures with greater network size. Alternatively, quality is often improved in smaller networks by adding greater connection complexity. This work demonstrates the benefits of using abstract behaviour representation, rather than evaluation through benchmark tasks, to assess the quality of computing substrates, as the latter typically has biases, and often provides little insight into the complete computing quality of physical systems.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bala, A., Ismail, I., Ibrahim, R., Sait, S.M.: Applications of metaheuristics in reservoir computing techniques: a review. IEEE Access 6, 58012–58029 (2018)
Adamatzky, A.: Game of Life Cellular Automata, vol. 1. Springer, London (2010). https://doi.org/10.1007/978-1-84996-217-9
Appeltant, L., et al.: Information processing using a single dynamical node as complex system. Nature Commun. 2, 468 (2011)
Büsing, L., Schrauwen, B., Legenstein, R.: Connectivity, dynamics, and memory in reservoir computing with binary and analog neurons. Neural Comput. 22(5), 1272–1311 (2010)
Crutchfield, J.P.: The calculi of emergence. Physica D 75(1–3), 11–54 (1994)
Dale, M.: Neuroevolution of hierarchical reservoir computers. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 410–417. ACM (2018)
Dale, M., Miller, J.F., Stepney, S., Trefzer, M.A.: Evolving carbon nanotube reservoir computers. In: Amos, M., Condon, A. (eds.) UCNC 2016. LNCS, vol. 9726, pp. 49–61. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41312-9_5
Dale, M., Miller, J.F., Stepney, S., Trefzer, M.A.: Reservoir computing in materio: an evaluation of configuration through evolution. In: 2016 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1–8, December 2016
Dale, M., Miller, J.F., Stepney, S., Trefzer, M.A.: Reservoir computing in materio: a computational framework for in materio computing. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 2178–2185, May 2017
Dale, M., Miller, J.F., Stepney, S., Trefzer, M.A.: A substrate-independent framework to characterise reservoir computers. arXiv preprint arXiv:1810.07135 (2018)
Gallicchio, C., Micheli, A., Pedrelli, L.: Deep reservoir computing: a critical experimental analysis. Neurocomputing 268, 87–99 (2017)
Goudarzi, A., Lakin, M.R., Stefanovic, D.: DNA reservoir computing: a novel molecular computing approach. In: Soloveichik, D., Yurke, B. (eds.) DNA 2013. LNCS, vol. 8141, pp. 76–89. Springer, Cham (2013). https://doi.org/10.1007/978-3-319-01928-4_6
Jaeger, H.: The “echo state” approach to analysing and training recurrent neural networks-with an erratum note. German National Research Center for Information Technology GMD Technical Report 148:34, Bonn, Germany (2001)
Jaeger, H.: Short term memory in echo state networks. GMD-Forschungszentrum Informationstechnik (2001)
Lavis, D.A.: Equilibrium statistical mechanics of lattice models. Springer, Dordrecht (2015). https://doi.org/10.1007/978-94-017-9430-5
Legenstein, R., Maass, W.: Edge of chaos and prediction of computational performance for neural circuit models. Neural Networks 20(3), 323–334 (2007)
Lehman, J., Stanley, K.O.: Exploiting open-endedness to solve problems through the search for novelty. In: ALIFE, pp. 329–336 (2008)
Lloyd, S.: Ultimate physical limits to computation. Nature 406(6799), 1047 (2000)
Lukoševičius, M.: A practical guide to applying echo state networks. In: Montavon, G., Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 7700, pp. 659–686. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-35289-8_36
Lukoševičius, M., Jaeger, H.: Reservoir computing approaches to recurrent neural network training. Comput. Sci. Rev. 3(3), 127–149 (2009)
Paquot, Y., et al.: Optoelectronic reservoir computing. Scientific Reports, 2 (2012)
Pearson, J.E.: Complex patterns in a simple system. Science 261(5118), 189–192 (1993)
Rodan, A., Tiňo, P.: Simple deterministically constructed recurrent neural networks. In: Fyfe, C., Tino, P., Charles, D., Garcia-Osorio, C., Yin, H. (eds.) IDEAL 2010. LNCS, vol. 6283, pp. 267–274. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15381-5_33
Rodan, A., Tino, P.: Minimum complexity echo state network. IEEE Trans. Neural Networks 22(1), 131–144 (2011)
Rodan, A., Tiňo, P.: Simple deterministically constructed cycle reservoirs with regular jumps. Neural Comput. 24(7), 1822–1852 (2012)
Schrauwen, B., Verstraeten, D., Van Campenhout, J.: An overview of reservoir computing: theory, applications and implementations. In: Proceedings of the 15th European Symposium on Artificial Neural Networks. Citeseer (2007)
Stepney, S.: The neglected pillar of material computation. Physica D 237(9), 1157–1164 (2008)
Tanaka, G., et al.: Recent advances in physical reservoir computing: a review. arXiv preprint arXiv:1808.04962 (2018)
Verstraeten, D., Schrauwen, B., D’Haene, M., Stroobandt, D.: An experimental unification of reservoir computing methods. Neural Networks 20(3), 391–403 (2007)
Xue, Y., Yang, L., Haykin, S.: Decoupled echo state networks with lateral inhibition. Neural Networks 20(3), 365–376 (2007)
Acknowledgments
This work is part of the SpInspired project, funded by EPSRC grant EP/R032823/1. Jack Dewhirst is funded by an EPSRC DTP PhD studentship.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Dale, M., Dewhirst, J., O’Keefe, S., Sebald, A., Stepney, S., Trefzer, M.A. (2019). The Role of Structure and Complexity on Reservoir Computing Quality. In: McQuillan, I., Seki, S. (eds) Unconventional Computation and Natural Computation. UCNC 2019. Lecture Notes in Computer Science(), vol 11493. Springer, Cham. https://doi.org/10.1007/978-3-030-19311-9_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-19311-9_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-19310-2
Online ISBN: 978-3-030-19311-9
eBook Packages: Computer ScienceComputer Science (R0)