Abstract
Model-based reinforcement learning methods are known to be highly efficient with respect to the number of trials required for learning optimal policies. In this article a novel fuzzy model-based reinforcement learning approach, fuzzy prioritized sweeping (F-PS), is presented. The approach is capable of learning strategies for Markov decision problems with continuous state and action spaces. The output of the algorithm are Takagi-Sugeno fuzzy systems approximating the Q-functions corresponding to the given control problems. From these Q-functions optimal control strategies can be easily derived. The effectiveness of the F-PS approach is shown by applying it to the task of selecting optimal framework signal plans in urban traffic networks. It is shown that the method outperforms existing model-based approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Appl, M. (2000). Model-Based Reinforcement Learning in Continuous Environments. Ph.D. thesis. Technical University of Munich, Department of Computer Science. http://www.martinappl.de
Appl, M. and W. Brauer (2000). Indirect reinforcement learning with adaptive state space partitions. Proceedings of the Third European Symposium on Intelligent Techniques.
Bertsekas, D. P. and J.N. Tsitsiklis (1996). Neuro-Dynamic Programming. Athena Scientific.
Bingham, E. (1998). Neurofuzzy traffic signal control. Master’s thesis, Helsinki University of Technology.
Davies, S. (1997). Multidimensional triangulation and interpolation for reinforcement learning. In M.C. Mozer, M.I. Jordan, and T. Petsche (Eds.), Advances in Neural Information Processing Systems, Volume 9, pp. 1005–1011. The MIT Press.
Horiuchi, T., A. Fujino, O. Katai, and T. Sawaragi (1996). Fuzzy interpolation-based Q-learning with continuous states and actions. Proceedings of the Fifth IEEE International Conference on Fuzzy Systems, 594–600.
Moore, A.W. and C.G. Atkeson (1993). Memory-based reinforcement learning: Converging with less data and less time. Robot Learning, 79–103.
Sugeno, M. (1985). An introductory survey of fuzzy control. Information Sciences 36, 59–83.
Sutton, R.S. and A.G. Barto (1998). Reinforcement Learning — An Introduction. MIT Press/Bradford Books, Cambridge, MA.
Takagi, T. and M. Sugeno (1985). Fuzzy identification of systems and its application to modeling and control. In IEEE Transactions on Systems, Man and Cybernetics, Volume 15, pp. 116–132.
Thorpe, T. (1997). Vehicle Traffic Light Control Using SARSA. Ph.D. thesis. Department of Computer Science, Colorado State University.
Watkins, CJ.C.H. (1989). Learning from Delayed Rewards. Ph.D. thesis, Cambridge University.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2002 Springer Science+Business Media New York
About this chapter
Cite this chapter
Appl, M., Brauer, W. (2002). Fuzzy Model-Based Reinforcement Learning. In: Zimmermann, HJ., Tselentis, G., van Someren, M., Dounias, G. (eds) Advances in Computational Intelligence and Learning. International Series in Intelligent Technologies, vol 18. Springer, Dordrecht. https://doi.org/10.1007/978-94-010-0324-7_15
Download citation
DOI: https://doi.org/10.1007/978-94-010-0324-7_15
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-010-3872-0
Online ISBN: 978-94-010-0324-7
eBook Packages: Springer Book Archive