Abstract
Learning is essential for an autonomous agent to adapt to an environment. One method that can be used is learning through trial and error. However, it is impractical because of the long learning time required when the agent learns in a complex environment. Therefore, some guidelines are necessary to expedite the learning process in the environment. Imitation can be used by agents as a guideline for learning. Sakato, Ozeki and Oka (2012) proposed a computational model of imitation and autonomous behavior. In the model, an agent can reduce its learning time through imitation. In this paper, we extend the model to continuous spaces, and add a function for selecting a target action for imitation from observed actions to the model. By these extension and adaptation, the model comes to adapt to more complex environment. Even in continuous spaces, the experimental results indicate that the model can adapt to an environment faster than a baseline model that learns only through trial and error.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Wiewiora, E., Cottrell, G., Elkan, C.: Principled methods for advising reinforcement learning agents. In: Proceedings of the Twentieth International Conference on Machine Learning (ICML 2003), pp. 792–799 (2003)
Ng, A., Harada, D., Russell, S.: Policy invariance under reward transformations: Theory and application to reward shaping. In: Proceedings of the Sixteenth International Conference on Machine Learning, pp. 278–287 (1999)
Price, B., Boutilier, C.: Implicit imitation in multiagent reinforcement learning. In: Proceedings of the Sixteenth International Conference on Machine Learning, pp. 325–334 (1999)
Billard, A., Epars, Y., Calinon, S., Schaal, S., Cheng, G.: Discovering optimal imitation strategies. Robotics and Autonomous Systems 47(2-3), 69–77 (2004)
Alissandrakis, A., Nehaniv, C., Dautenhahn, K.: Imitation with alice: Learning to imitate corresponding actions across dissimilar embodiments. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans 32(4), 482–496 (2002)
Nehaniv, C., Dautenhahn, K.: Of hummingbirds and helicopters: An algebraic framework for interdisciplinary studies of imitation and its applications. In: Interdisciplinary Approaches to Robot Learning, pp. 136–161 (1999)
Sakato, T., Ozeki, M., Oka, N.: A computatonal model of imitation and autonomous behavior. In: 2012 13th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel Distributed Computing (SNPD), pp. 13–18 (August 2012)
Tabuchi, K., Taniguchi, T., Sawaragi, T.: Efficient acquisition of behaviors by harmonizing reinforcement learning with imitation learning. In: The 20th Annual Conference of the Japanese Society for Artificial Intelligence (2006) (in Japanese)
van Hasselt, H.: Reinforcement learning in continuous state and action spaces. In: Reinforcement Learning: State of the Art, pp. 207–252. Springer (2012)
Kuniyoshi, Y.: Adaptive and emergent imitation as the fundamental of humanoid intelligence. Journal of the Robotics Society of Japan 25(5), 671–677 (2007) (in Japanese)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer International Publishing Switzerland
About this paper
Cite this paper
Sakato, T., Ozeki, M., Oka, N. (2013). A Computational Model of Imitation and Autonomous Behavior in Continuous Spaces. In: Lee, R. (eds) Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing. Studies in Computational Intelligence, vol 492. Springer, Heidelberg. https://doi.org/10.1007/978-3-319-00738-0_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-00738-0_4
Publisher Name: Springer, Heidelberg
Print ISBN: 978-3-319-00737-3
Online ISBN: 978-3-319-00738-0
eBook Packages: EngineeringEngineering (R0)