Abstract
One problem in robotic soccer (and in robotics in general) is to adapt skills and the overall behavior to a changing environment and to hardware improvements. We applied hierarchical reinforcement learning in an SMDP framework learning on all levels simultaneously. As our experiments show, learning simultaneously on the skill level and on the skill selection level is advantageous since it allows for a smooth adaption to a changing environment. Furthermore, the skills we trained turn also out to be quite competitive when run on the real robotic players of the players of our CS Freiburg team.
This work has been partially supported by Deutsche Forschungsgemeinschaft (DFG) and by SICK AG.
Chapter PDF
References
Albus, J.S.: A theory of cerebellar function. In: Mathematical Biosciences, vol. 10, pp. 25–61
Barto, A., Sutton, R.S.: Reinforcement Learning – An Introduction. MIT Press, Cambridge (1998)
Bradtke, S.J., Duff, M.O.: Reinforcement learning methods for continuous time Markov decision problems. In: Schlender, B., Frielinghaus, W. (eds.) GI-Fachtagung 1974. LNCS, vol. 7, pp. 393–400. Springer, Heidelberg (1995)
Crites, R.H., Barto, A.G.: Improving elevator performance using reinforcement learning. In: Touretzky, D.S., Mozer, M.C., Hasselmo, M.E. (eds.) Advances in Neural Information Processing Systems, vol. 8, pp. 1017–1023. The MIT Press, Cambridge (1996)
Dietterich, T.G.: The MAXQ method for hierarchical reinforcement learning. In: Fifteenth International Conference on Machine Learning. Morgan Kaufmann, San Francisco (1998)
Kitano, H., Tambe, M., Stone, P., Veloso, M., Coradeschi, S., Osawa, E., Matsubara, H., Noda, I., Asada, M.: The RoboCup synthetic agent challenge. In: International Joint Conference on Artificial Intelligence, IJCAI 1997 (1997)
Parr, R.: Hierarchical Control and Learning for Markov decision processes. Ph.d. thesis, University of California at Berkeley (1998)
Riedmiller, M., Merke, A.: Karlsruhe Brainstormers – a reinforcement learning approach to robotic soccer ii. In: Veloso et al. [15] (to appear)
Rummery, G., Niranjan, M.: On-line q-learning using connectionist systems. Technical Report CUED/F-INFENG/TR, Cambridge University Engineering Department (1996)
Stone, P., Riley, P., Veloso, M.: The CMUnited-99 champion simulator team. In: Veloso, M., Pagello, E., Kitano, H. (eds.) RoboCup-99: Robot Soccer World Cup III, Berlin, Heidelberg. Springer, Heidelberg (2000)
Stone, P., Sutton, R.S.: Scaling reinforcement learning toward RoboCup soccer. In: Proceedings of the 18th International Conference on Machine Learning (2001)
Stone, P., Veloso, M.: Layered learning. In: Lopez de Mantaras, R., Plaza, E. (eds.) Eleventh European Conference on Machine Learning (ECML 2000). Springer, Heidelberg (2000)
Sutton, R., Precup, D., Singh, S.: Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence, vol. 112, pp. 181–211 (1999)
Takahashi, Y., Asada, M.: Vision-guided behavior acquisition of a mobile robot by multi-layered reinforcement learning. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 1, pp. 395–402 (2000)
Veloso, M., Balch, T., Stone, P. (eds.): International RoboCup Symposium 2001 (2002) (to appear)
Watkins, C.J.C.H.: Learning with Delayed Rewards. Ph.d. thesis, Cambridge University (1989)
Weigel, T., Kleiner, A., Diesch, F., Dietl, M., Gutmann, J.S., Nebel, B., Stiegeler, P., Szerbakowski, B.: CS Freiburg 2001. In: Veloso et al. [15] (2001) (to appear)
Weigel, T., Auerbach, W., Dietl, M., Dümler, B., Gutmann, J.-S., Marko, K., Müller, K., Nebel, B., Szerbakowski, B., Thiel, M.: CS Freiburg: Doing the right thing in a group. In: Stone, P., Kraetzschmar, G., Balch, T. (eds.) RoboCup-2000: Robot Soccer World Cup IV, pp. 52–63. Springer, Heidelberg (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kleiner, A., Dietl, M., Nebel, B. (2003). Towards a Life-Long Learning Soccer Agent. In: Kaminka, G.A., Lima, P.U., Rojas, R. (eds) RoboCup 2002: Robot Soccer World Cup VI. RoboCup 2002. Lecture Notes in Computer Science(), vol 2752. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45135-8_10
Download citation
DOI: https://doi.org/10.1007/978-3-540-45135-8_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40666-2
Online ISBN: 978-3-540-45135-8
eBook Packages: Springer Book Archive