The RBF Neural Network in Approximate Dynamic Programming

  • Branko Šter
  • Andrej Dobnikar


A radial basis function (RBF) neural network was applied to an optimal control problem. The role of an approximation architecture in the task of dynamic programming is emphasised. While it has been proved that dynamic programming works well for moderate discrete spaces, research is continuing on how to apply dynamic programming techniques to large discrete and continuous spaces. For continuous spaces there does not yet exist a universal approach, but it seems that a RBF network is able to solve the problem with a negligible amount of manual experimentation.


Radial Basis Function Optimal Control Problem Reinforcement Learn Hide Neuron Radial Basis Function Neural Network 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Watkins, J.C.H., and Dayan, P.: Technical Note: Q-learning. Machine Learning Vol. 8, pp. 279–292 (1992).MATHGoogle Scholar
  2. [2]
    Barto, A.G., Sutton, R.S., and Anderson, C.W.: Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Transactions on Systems, Man, and Cybernetics Vol. 13(5), pp. 834–846 (1983).CrossRefGoogle Scholar
  3. [3]
    Sutton, R.S.: Learning to predict by the methods of temporal differences. Machine Learning Vol. 3(1), pp. 9–44 (1988).Google Scholar
  4. [4]
    Barto, A.G.: Reinforcement Learning and Adaptive Critic Methods. In Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches. D. A. White and D. A. Sofge (Eds.). New York: Van Nostrand Reinhold, pp. 469–491 (1992).Google Scholar
  5. [5]
    Kaelbling, L.P., Littman, M.L., and Moore, A.W.: Reinforcement Learning: A Survey, Journal of Artificial Intelligence Research Vol. 4 (1996).Google Scholar
  6. [6]
    Bellman, R.: Dynamic Programming. Princeton University Press, Princeton, NJ (1957).MATHGoogle Scholar
  7. [7]
    Bertsekas, Dimitri P., and Tsitsiklis, John N.: Neuro-Dynamic Programming. Athena Scientific, Belmont, Massachusetts (1996).MATHGoogle Scholar
  8. [8]
    Moody J. and Darken C.J.: Fast Learning in Networks of Locally Tuned Processing Units. Neural Computation 1 pp. 281–294 (1989).Google Scholar
  9. [9]
    Haykin, S.: Neural Networks. Macmillan College Publishing Company (1994).Google Scholar
  10. [10]
    Boyan, J.A., and Moore, A.W.: Generalization in Reinforcement Learning: Safely Approximating the Value Function, in G. Tesauro, D.S. Touretzky and T.K. Leen, eds., Advances in Neural Information Processing Systems 7, MIT Press, Cambridge MA (1995).Google Scholar
  11. [11]
    Bradtke, S.: Incremental Dynamic Programming for On-line Adaptive Optimal Control. PhD Thesis. University of Massachusetts, Amherst (1994).Google Scholar
  12. [12]
    Urbancic, T.: PhD Thesis, University of Ljubljana (1994).Google Scholar
  13. [13]
    Tsitsiklis, J.N., and Van Roy, B.: Feature-Based Methods for Large Scale Dynamic Programming, Machine Learning Vol. 22, pp. 59–94 (1996).MATHGoogle Scholar

Copyright information

© Springer-Verlag Wien 1999

Authors and Affiliations

  • Branko Šter
    • 1
  • Andrej Dobnikar
    • 1
  1. 1.Faculty of Computer and Information ScienceUniversity of LjubljanaSlovenia

Personalised recommendations