A Kernel-Based Sarsa(\(\lambda \)) Algorithm with Clustering-Based Sample Sparsification
In the past several decades, as a significant class of solutions to the large scale or continuous space control problems, kernel-based reinforcement learning (KBRL) methods have been a research hotspot. While the existing sample sparsification methods of KBRL exist the problems of low time efficiency and poor effect. For this problem, we propose a new sample sparsification method, clustering-based novelty criterion (CNC), which combines a clustering algorithm with a distance-based novelty criterion. Besides, we propose a clustering-based selective kernel Sarsa(\(\lambda \)) (CSKS(\(\lambda \))) on the basis of CNC, which applies Sarsa(\(\lambda \)) to learning parameters of the selective kernel-based value function based on local validity. Finally, we illustrate that our CSKS(\(\lambda \)) surpasses other state-of-the-art algorithms by Acrobot experiment.
KeywordsReinforcement learning Kernel method Sample sparsification Clustering Sarsa(\(\lambda \))
This work was funded by National Science Foundation of China (61303108, 61373094, 61472262), Natural Science Foundation of Jiangsu (BK2012616), High School Natural Foundation of Jiangsu (13KJB520020), Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University (93K172014K04), Suzhou Industrial application of basic research program part (SYG201422), Provincial Key Laboratory for Computer Information Processing Technology, Soochow University (KJS1524).
- 5.Fan-Cheng, M., Ya-Ping, D.: Reinforcement learning adaptive control for upper limb rehabilitation robot based on fuzzy neural network. In: Control Conference (CCC), pp. 5157–5161 (2012)Google Scholar
- 8.van Seijen, H., Sutton, R.S.: True online TD(\(\lambda \)). In: Proceedings of the 31st International Conference on Machine Learning (ICML), pp. 692–700 (2014)Google Scholar
- 10.Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar