Abstract
Several models have been proposed for describing grouping behavior such as bird flocking, terrestrial animal herding, and fish schooling. In these models, a fixed rule has been imposed on each individual a priori for its interactions in a reductive and rigid manner. We have proposed a new framework for self-organized grouping of agents by reinforcement learning. It is important to introduce a learning scheme for developing collective behavior in artificial autonomous distributed systems. This scheme can be expanded to cases in which predators are present. In this study we integrate grouping and anti-predator behaviors into our proposed scheme. The behavior of agents is demonstrated and evaluated in detail through computer simulations, and their grouping and anti-predator behaviors developed as a result of learning are shown to be diverse and robust by changing some parameters of the scheme.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Shaw, E.: Schooling Fishes. American Scientist 66, 166–175 (1978)
Partridge, B.L.: The structure and function of fish schools. Scientific American 246, 90–99 (1982)
Aoki, I.: A Simulation Study on the Schooling Mechanism in Fish. Bulletin of the Japanese Society of Scientific Fisheries 48(8), 1081–1088 (1982)
Reynolds, C.W.: Flocks, Herds, and Schools: A Distributed Behavioral Model. Computer Graphics 21(4), 25–34 (1987)
Huth, A., Wissel, C.: The Simulation of the Movement of Fish Schools. Journal of Theoretical Biology 156, 365–385 (1992)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement Learning: A Survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Glorennec, P.Y.: Reinforcement Learning: an Overview. In: Proceedings of ESIT 2000 - European Symposium on Intelligent Techniques, Aachen, Germany, pp. 17–35 (2000)
Watkins, C.: Learning from delayed rewards, PhD Thesis, University of Cambridge, England (1989)
Watkins, C., Dayan, P.: Q-learning, Machine Learning, vol. 8, pp. 279–292 (1992)
Buşoniu, L., De Schutter, B., Babuška, R.: Learning and coordination in dynamic multiagent systems, Technical report 05-019, Delft Center for Systems and Control, Delft University of Technology, Delft, The Netherlands (2005)
Dietterich, T.G., Becker, S., Ghahramani, Z.: Advances in Neural Information Processing Systems 14. MIT Press, Cambridge (2002)
Mahadevan, S.: Reinforcement Learning Repository, University of Massachusetts, http://www-anw.cs.umass.edu/rlr/
Thrun, S.B.: Efficient Exploratio. In: Reinforcement Learning, Technical report CMU-CS-92-102, Carnegie Mellon University, Pittsburgh, PA (1992)
Thrun, S.B.: The role of exploration in learning control. In: White, A.D., Sofge, D.A. (eds.) Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, Van Nostrand Reinhold, New York (1992)
Pitcher, T.J., Wyche, C.J.: Predator avoidance behaviour of sand-eel schools: why schools seldom split. In: Noakes, D.L.G., Lindquist, B.G., Helfman, G.S., Ward, J.A. (eds.) Predators and Prey in Fishes, pp. 193–204. Junk, The Hague (1983)
Huth, A., Wissel, C.: The Simulation of the fish schools in comparison with experimental data. Ecological Modeling 75/76, 135–145 (1994)
Niwa, H.-S.: Self-organizing dynamic model of fish schooling. Journal of theoretical Biology 171, 123–136 (1994)
Shimoyama, N., Sugawara, K., Mizuguchi, T., Hayakawa, Y., Sano, M.: Collective Motion in a System of Motile Elements. Physical Review Letters 76, 3870–3873 (1996)
Vabo, R., Nottestad, L.: An individual based model of fish school reactions: predicting antipredator behaviour as observed in nature. Fisheries Oceanography 6, 155–171 (1997)
Inada, Y., Kawachi, K.: Order and Flexibility in the Motion of Fish Schools. Journal of theoretical Biology 214, 371–387 (2002)
Oboshi, T., Kato, S., Mutoh, A., Itoh, H.: A Simulation Study on the Form of Fish Schooling for Escape from Predator. Forma 18, 119–131 (2003)
Tomimasu, M., Morihiro, K., Nishimura, H., Isokawa, T., Matsui, N.: A Reinforcement Learning Scheme of Adaptive Flocking Behavior. In: Proc. of the 10th Int. Symp. on Artificial Life and Robotics (AROB), GS1-4, Oita, Japan (2005)
Morihiro, K., Isokawa, T., Nishimura, H., Tomimasu, M., Kamiura, N., Matsui, N.: Reinforcement Learning Scheme for Flocking Behavior Emergence. Journal of Advanced Computational Intelligence and Intelligent Informatics(JACIII) 11(2), 155–161 (2007)
Morihiro, K., Nishimura, H., Isokawa, T., Matsui, N.: Learning Grouping and Anti-Predator Behaviors for Multi-Agent Systems. In: Lovrek, I., Howlett, R.J., Jain, L.C. (eds.) KES 2008, Part II. LNCS (LNAI), vol. 5178, pp. 426–433. Springer, Heidelberg (2008)
Morihiro, K., Isokawa, T., Nishimura, H., Matsui, N.: Emergence of Flocking Behavior Based on Reinforcement Learning. In: Gabrys, B., Howlett, R.J., Jain, L.C. (eds.) KES 2006. LNCS (LNAI), vol. 4253, pp. 699–706. Springer, Heidelberg (2006)
Morihiro, K., Nishimura, H., Isokawa, T., Matsui, N.: Reinforcement Learning Scheme for Grouping and Anti-predator Behavior. In: Apolloni, B., Howlett, R.J., Jain, L. (eds.) KES 2007, Part III. LNCS (LNAI), vol. 4694, pp. 115–122. Springer, Heidelberg (2007)
Morihiro, K., Nishimura, H., Isokawa, T., Matsui, N.: Emergence of Grouping and Anti-Predator Behavior by Reinforcement Learning Scheme (submitted)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Morihiro, K., Nishimura, H., Isokawa, T., Matsui, N. (2010). Grouping and Anti-predator Behaviors for Multi-agent Systems Based on Reinforcement Learning Scheme. In: Srinivasan, D., Jain, L.C. (eds) Innovations in Multi-Agent Systems and Applications - 1. Studies in Computational Intelligence, vol 310. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-14435-6_6
Download citation
DOI: https://doi.org/10.1007/978-3-642-14435-6_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-14434-9
Online ISBN: 978-3-642-14435-6
eBook Packages: EngineeringEngineering (R0)