Optimal Training Sequences for Locally Recurrent Neural Networks

  • Krzysztof Patan
  • Maciej Patan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5768)


The problem of determining an optimal training schedule for a locally recurrent neural network is discussed. Specifically, the proper choice of the most informative measurement data guaranteeing the reliable prediction of the neural network response is considered. Based on a scalar measure of the performance defined on the Fisher information matrix related to the network parameters, the problem was formulated in terms of optimal experimental design. Then, its solution can be readily achieved via the adaptation of effective numerical algorithms based on the convex optimization theory. Finally, some illustrative experiments are provided to verify the presented approach.


Input Sequence Recurrent Neural Network Fisher Information Matrix Optimum Experimental Design Experimental Design Theory 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Gupta, M.M., Jin, L., Homma, N.: Static and Dynamic Neural Networks. From Fundamentals to Advanced Theory. John Wiley & Sons, New Jersey (2003)CrossRefGoogle Scholar
  2. 2.
    Korbicz, J., Kościelny, J., Kowalczuk, Z., Cholewa, W.: Fault Diagnosis. Models, Artificial Intelligence, Applications. Springer, Heidelberg (2004)zbMATHGoogle Scholar
  3. 3.
    van de Wal, M., de Jager, B.: A review of methods for input/output selection. Automatica 37, 487–510 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Patan, K.: Artificial Neural Networks for the Modelling and Fault Diagnosis of Technical Processes. LNCIS. Springer, Berlin (2008)Google Scholar
  5. 5.
    Fukumizu, K.: Statistical active learning in multilayer perceptrons. IEEE Transactions on Neural Networks 11, 17–26 (2000)CrossRefGoogle Scholar
  6. 6.
    Witczak, M.: Toward the training of feed-forward neural networks with the D-optimum input sequence. IEEE Transactions on Neural Networks 17, 357–373 (2006)CrossRefGoogle Scholar
  7. 7.
    Patan, K., Patan, M.: Selection of training sequences for locally recurrent neural network training. In: Malinowski, K., Rutkowski, L. (eds.) Recent Advances in Control and Automation, pp. 252–262. Academic Publishing House, EXIT, Warsaw (2008)Google Scholar
  8. 8.
    Fedorov, V.V., Hackl, P.: Model-Oriented Design of Experiments. Lecture Notes in Statistics. Springer, New York (1997)CrossRefzbMATHGoogle Scholar
  9. 9.
    Tsoi, A.C., Back, A.D.: Locally recurrent globally feedforward networks: A critical review of architectures. IEEE Transactions on Neural Networks 5, 229–239 (1994)CrossRefGoogle Scholar
  10. 10.
    Marcu, T., Mirea, L., Frank, P.M.: Development of dynamical neural networks with application to observer based fault detection and isolation. International Journal of Applied Mathematics and Computer Science 9(3), 547–570 (1999)zbMATHGoogle Scholar
  11. 11.
    Patan, K.: Stability analysis and the stabilization of a class of discrete-time dynamic neural networks. IEEE Transactions on Neural Networks 18, 660–673 (2007)CrossRefGoogle Scholar
  12. 12.
    Atkinson, A.C., Donev, A.N.: Optimum Experimental Designs. Clarendon Press, Oxford (1992)zbMATHGoogle Scholar
  13. 13.
    Walter, E., Pronzato, L.: Identification of Parametric Models from Experimental Data. Springer, London (1997)zbMATHGoogle Scholar
  14. 14.
    Patan, M.: Optimal Observation Strategies for Parameter Estimation of Distributed Systems. Lecture Notes in Control and Computer Science, vol. 5. Zielona Góra University Press, Zielona Góra (2004)zbMATHGoogle Scholar
  15. 15.
    Uciński, D.: Optimal selection of measurement locations for parameter estimation in distributed processes. International Journal of Applied Mathematics and Computer Science 10(2), 357–379 (2000)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Rafajłowicz, E.: Optimum choice of moving sensor trajectories for distributed parameter system identification. International Journal of Control 43(5), 1441–1451 (1986)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Uciński, D.: Optimal Measurement Methods for Distributed Parameter System Identification. CRC Press, Boca Raton (2005)zbMATHGoogle Scholar
  18. 18.
    Kiefer, J., Wolfowitz, J.: Optimum designs in regression problems. The Annals of Mathematical Statistics 30, 271–294 (1959)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Pázman, A.: Foundations of Optimum Experimental Design. Mathematics and Its Applications. D. Reidel Publishing Company, Dordrecht (1986)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Krzysztof Patan
    • 1
  • Maciej Patan
    • 1
  1. 1.Institute of Control and Computation EngineeringUniversity of Zielona GóraPoland

Personalised recommendations