Abstract
Imitation Learning is considered both as a method to acquire complex human and agent behaviors, and as a way to provide seeds for further learning. However, it is not clear what is a building block in imitation learning and what is the interface of blocks; therefore, it is difficult to apply imitation learning in a constructive way. This paper addresses agents’ intentions as the building block that abstracts local situations of the agent and proposes a hierarchical hidden Markov model (HMM) in order to tackle this issue. The key of the proposed model is introduction of gate probabilities that restrict transition among agents’ intentions according to others’ intentions. Using these probabilities, the framework can control transitions flexibly among basic behaviors in a cooperative behavior. A learning method for the framework can be derived based on Baum-Welch’s algorithm, which enables learning by observation of mentors’ demonstration. Imitation learning by the proposed method can generalize behaviors from even one demonstration, because the mentors’ behaviors are expressed as a distributed representation of a flow of likelihood in HMM.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Ghahramani, Z., Jordan, M.I.: Factorial hidden markov models. Machine Learning 29, 245–275 (1997)
Intille, S.S., Bobick, A.F.: Recognizing planned, multiperson action. Computer Vision and Image Understanding: CVIU 81(3), 414–445 (2001)
Ivanov, Y.A., Bobick, A.F.: Recognition of multi-agent interaction in video surveillance. In: International Conference on Computer Vision, September 1999, vol. 1, pp. 169–176. IEEE, Los Alamitos (1999)
Jordan, M.I., Ghahramani, Z., Jaakkola, T., Saul, L.K.: An introduction to variational methods for graphical models. Machine Learning 37(2), 183–233 (1999)
Jordan, M.I., Ghahramani, Z., Saul, L.K.: Hidden markov decision trees. In: Mozer, M.C., Jordan, M.I., Petsche, T. (eds.) Advances in Neural Information Processing Systems, vol. 9, p. 501. The MIT Press, Cambridge (1997)
Kuniyoshi, Y., Inoue, H.: Qualitative recognition of ongoing human action sequences. In: Proc. IJCAI 1993, pp. 1600–1609 (1993)
Miyamoto, H., Kawato, M.: A tennis serve and upswing learning robot based on bi-directional theory. Neural Networks 11, 1331–1344 (1998)
Oliver, N.M., Rosario, B., Pentland, A.: A bayesian computer vision system for modeling human interactions. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(8), 831–843 (2000)
Schaal, S.: Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences 3(6), 233–242 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Noda, I. (2004). Hidden Markov Modeling of Team-Play Synchronization. In: Polani, D., Browning, B., Bonarini, A., Yoshida, K. (eds) RoboCup 2003: Robot Soccer World Cup VII. RoboCup 2003. Lecture Notes in Computer Science(), vol 3020. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-25940-4_9
Download citation
DOI: https://doi.org/10.1007/978-3-540-25940-4_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22443-3
Online ISBN: 978-3-540-25940-4
eBook Packages: Springer Book Archive