Abstract
Hybrid systems consisting of model-based and model-free systems will be engaged in the behavior/dialog control systems of future robots/agents to satisfy several user’s requirements and simultaneously cope with diverse and unexpected situations. We have constructed a modular neural network model based on reinforcement learning for model-free learning. For an effective hybrid system, the model-free learning system should be aware of the current targets. This can be achieved by automatically acquiring a list of important sequential events. We propose a basic mechanism that can automatically acquire the list of sequential events with confidence measures reflecting current situations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Takeuchi, J., Shouno, O., Tsujino, H.: Modular neural networks for reinforcement learning with temporal intrinsic rewards. In: Proceedings of 2007 International Joint Conference on Neural Networks, IJCNN (2007) (CD-ROM)
Takeuchi, J., Shouno, O., Tsujino, H.: Modular neural networks for model-free behavioral learning. In: Proceedings of the 18th International Conference on Artificial Neural Networks (ICANN), vol. I, pp. 730–739 (2008)
Jaeger, H.: The ‘echo state’ approach to analysing and training recurrent neural networks. Gmd report 148, German National Research Center for Information Technology (2001)
Barto, A.G., Singh, S., Chentanez, N.: Intrinsically motivated learning of hierarchical collection of skills. In: Proceedings of the 3rd International Conference on Developmental Learning, ICDL (2004)
Singh, S., Barto, A.G., Chentanez, N.: Intrinsically motivated reinforcement learning. In: Advances in Neural Information Processing Systems 17, pp. 1281–1288. MIT Press, Cambridge (2005)
Sutton, R.S., Precup, D., Singh, S.P.: Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence 112(1-2), 181–211 (1999)
Sutton, R.S., Tanner, B.: Temporal-difference networks. In: Advances in Neural Information Processing Systems 17 (NIPS 2004), pp. 1377–1384 (2005)
Makino, T., Takagi, T.: On-line discovery of temporal-difference networks. In: Proceedings of the twenty-fifth international conference on machine learning, ICML (2008)
Doya, K., Samejima, K., Katagiri, K.i., Kawato, M.: Multiple model-based reinforcement learning. Neural Computation 14, 1347–1369 (2002)
Nishida, S., Ishii, K., Furukawa, T.: An online adaptation control system using mnSOM. In: King, I., Wang, J., Chan, L.-W., Wang, D. (eds.) ICONIP 2006. LNCS, vol. 4232, pp. 935–942. Springer, Heidelberg (2006)
Kohonen, T.: Self-Organizing Maps. Springer, Heidelberg (1995)
Berglund, E., Sitte, J.: The parameterless self-organizing map algorithm. IEEE transactions on neural networks 17(2), 305–316 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Takeuchi, J., Shouno, O., Tsujino, H. (2009). Self-Referential Event Lists for Self-Organizing Modular Reinforcement Learning. In: Köppen, M., Kasabov, N., Coghill, G. (eds) Advances in Neuro-Information Processing. ICONIP 2008. Lecture Notes in Computer Science, vol 5507. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03040-6_28
Download citation
DOI: https://doi.org/10.1007/978-3-642-03040-6_28
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03039-0
Online ISBN: 978-3-642-03040-6
eBook Packages: Computer ScienceComputer Science (R0)