On-line Training of ESN and IP Tuning Effect

  • Petia Koprinkova-Hristova
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8681)


In the present paper we investigate influence of IP tuning of Echo state network (ESN) reservoir on the overall behavior of the on-line trained adaptive critic network. The experiments were done using Adaptive Critic Design (ACD) scheme with on-line trainable ESN critic for real time control of a mobile laboratory robot. Comparison of behavior of ESN critics trained with and without IP tuning showed that IP algorithm improved critic behavior significantly. It was observed that IP tuning prevents uncontrolled increase of reservoir output weights during on-line training.


Echo state network RNN stability IP training of reservoir on-line training Recursive Least Squares 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Barabanov, N., Prokhorov, D.: Stability analysis of discrete-time recurrent neural networks. IEEE Trans. on Neural Networks 13(2), 292–303 (2002)Google Scholar
  2. 2.
    Jaeger, H.: Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the echo state network approach. GMD Report 159, German National Research Center for Information Technology (2002)Google Scholar
  3. 3.
    Jaeger, H.: Adaptive nonlinear system identification with echo state networks. In: Advances in Neural Information Processing Systems 15 (NIPS 2002), pp. 593–600. MIT Press, Cambridge (2003)Google Scholar
  4. 4.
    Haddad, W.M., Chellaboina, V.S., Nersesov, S.G.: Thermodynamics: A Dynamical System Approach. Princeton University Press (2005)Google Scholar
  5. 5.
    Koprinkova-Hristova, P., Oubbati, M., Palm, G.: Heuristic dynamic programming using echo state network as online trainable adaptive critic. International Journal of Adaptive Control and Signal Processing 27(10), 902–914 (2013)zbMATHMathSciNetGoogle Scholar
  6. 6.
    Koprinkova-Hristova, P., Palm, G.: ESN Intrinsic Plasticity versus Reservoir Stability. In: Honkela, T. (ed.) ICANN 2011, Part I. LNCS, vol. 6791, pp. 69–76. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  7. 7.
    Prokhorov, D.V.: Adaptive critic designs and their applications. Ph.D. dissertation. Department of Electrical Engineering, Texas Tech. Univ. (1997)Google Scholar
  8. 8.
    Lukosevicius, M., Jaeger, H.: Reservoir computing approaches to recurrent neural network training. Computer Science Review 3, 127–149 (2009)CrossRefGoogle Scholar
  9. 9.
    Ozturk, M., Xu, D., Principe, J.: Analysis and design of Echo state networks. Neural Computation 19, 111–138 (2007)CrossRefzbMATHGoogle Scholar
  10. 10.
    Schrauwen, B., Wandermann, M., Verstraeten, D., Steil, J.J., Stroobandt, D.: Improving reservoirs using intrinsic plasticity. Neurocomputing 71, 1159–1171 (2008)CrossRefGoogle Scholar
  11. 11.
    Steil, J.J.: Online reservoir adaptation by intrinsic plasticity for back-propagation-decoleration and echo state learning. Neural Networks 20, 353–364 (2007)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Petia Koprinkova-Hristova
    • 1
  1. 1.IICT, Bulgarian Academy of SciencesSofiaBulgaria

Personalised recommendations