Agents and Multi-agent Coordination

  • Pratyusha RakshitEmail author
  • Amit Konar
Part of the Cognitive Intelligence and Robotics book series (CIR)


An intelligent agent is an entity that performs its task in a given environment by exploiting the knowledge acquired from its interaction with the environment during problem-solving process. Over the past two decades, multi-agent systems have emerged as a new methodology to address the issue of organizing a large-scale system by assembling and coordinating individual agents to achieve a goal jointly. Remarkable features of multi-agent systems resulting in their immense real-world applications include low implementation cost, adaptability with dynamicity of environment, enhanced flexibility, great robustness, and ease of maintenance. A multi-agent system is primarily characterized by goal-oriented coordination among its agents, both in cooperative and in competitive circumstances. This chapter introduces the basic concepts of cooperative and competitive multi-agent coordination. It begins with formal definitions of agency and elaborately discusses the perceptual and learning capability of an agent based on its architecture. Gradually, the chapter unveils the emergence of multi-agent coordination due to handshaking of distributed artificial intelligence and machine intelligence. The chapter then highlights the significance of planning and learning in multi-agent coordination to solve real-world problems. The chapter next demonstrates the scope of evolutionary optimization algorithms to maximize coordination efficiency in multi-agent robotics by optimal utilization of system resources. The chapter ends with a discussion on enhancing the performance of traditional evolutionary optimization algorithms to handle measurement noise in real-world multi-robot coordination problems.


Agent Agent perception Agent architecture Multi-agent coordination Multi-agent planning Multi-agent learning Coordination in presence of noise 


  1. 1.
    N. Kasabov, R. Kozma, Introduction: hybrid intelligent adaptive systems. Int. J. Intell. Syst. 13(6), 453–454 (1998)CrossRefGoogle Scholar
  2. 2.
    D. Romero, P. Bernus, O. Noran, J. Stahre, Å. Fast-Berglund, in The Operator 4.0: Human Cyber-Physical Systems & Adaptive Automation Towards Human-Automation Symbiosis Work Systems. IFIP International Conference on Advances in Production Management Systems (Springer, Cham, 2016), pp. 677–686Google Scholar
  3. 3.
    G. Weiss, Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence (MIT press, 1999)Google Scholar
  4. 4.
    C. Peters, G. Castellano, M. Rehm, E. André, A. Raouzaiou, K. Rapantzikos, K. Karpouzis, G. Volpe, A. Camurri, A. Vasalou, Fundamentals of agent perception and attention modelling, in Emotion-Oriented Systems (Springer, Berlin, Heidelberg, 2011), pp. 293–319Google Scholar
  5. 5.
    M.A. Arbib, Brains, Machines, and Mathematics (Springer Science & Business Media, 2012)Google Scholar
  6. 6.
    S.J. Russell, P. Norvig, Artificial Intelligence: A Modern Approach (Prentice Hall, Englewood Cliffs, NJ, 1995)zbMATHGoogle Scholar
  7. 7.
    M.R. Genesereth, N.J. Nilsson, Logical Foundations of Artificial Intelligence (Morgan Kaufmann, 1987)Google Scholar
  8. 8.
    K. Konolige, A Deduction Model of Belief (Morgan Kaufmann, 1986)Google Scholar
  9. 9.
    R. Brooks, A robust layered control system for a mobile robot. IEEE J. Robot. Autom. 2(1), 14–23 (1986)CrossRefGoogle Scholar
  10. 10.
    J. Ferber, Reactive distributed artificial intelligence: principles and applications, in Foundations of Distributed Artificial Intelligence (1996), pp. 287–314Google Scholar
  11. 11.
    M.W. Matlin, Cognition (Wiley, Hoboken, NJ, 2005)Google Scholar
  12. 12.
    M.E. Bratman, D.J. Israel, M.E. Pollack, Plans and resource-bounded practical reasoning. Comput. Intell. 4(3), 349–355 (1988)CrossRefGoogle Scholar
  13. 13.
    M. Bratman, Intention, Plans, and Practical Reason (Harvard University Press, 1987)Google Scholar
  14. 14.
    N.J. Nilsson, Artificial Intelligence: A New Synthesis (Elsevier, 1998)Google Scholar
  15. 15.
    T. Dean, J. Allen, Y. Aloimonos, Artificial Intelligence: Theory and Practice (Benjamin-Cummings Publishing Co. Inc., 1995)Google Scholar
  16. 16.
    J. Pearl, Heuristic search theory: survey of recent results. IJCAI 1, 554–562 (1981)Google Scholar
  17. 17.
    S.M. LaValle, Planning Algorithms (Cambridge University Press, 2006)Google Scholar
  18. 18.
    A. Konar, Artificial Intelligence and Soft Computing: Behavioral and Cognitive Modeling of The Human Brain (CRC Press, 1999)Google Scholar
  19. 19.
    E.W. Dijkstra, A note on two problems in connexion with graphs. Numer. Math. 1, 269–271 (1959)MathSciNetCrossRefGoogle Scholar
  20. 20.
    A. Stentz, Optimal and efficient path planning for partially-known environments, in Proceedings of IEEE International Conference on Robotics and Automation, 1994, pp. 3310–3317Google Scholar
  21. 21.
    O. Takahashi, R.J. Schilling, Motion planning in a plane using generalized Voronoi diagrams. IEEE Trans. Robot. Autom. 5(2), 143–150 (1989)CrossRefGoogle Scholar
  22. 22.
    S. Kambhampati, L. Davis, Multiresolution path planning for mobile robots. IEEE J. Robot. Autom. 2(3), 135–145 (1986)CrossRefGoogle Scholar
  23. 23.
    P. Bhattacharjee, P. Rakshit, I. Goswami, A. Konar, A.K. Nagar, Multi-robot path-planning using artificial bee colony optimization algorithm, in IEEE World Congress on Nature and Biologically Inspired Computing, 2011, pp. 219–224Google Scholar
  24. 24.
    P. Rakshit, A. Konar, P. Bhowmik, I. Goswami, S. Das, L.C. Jain, A.K. Nagar, Realization of an adaptive memetic algorithm using differential evolution and Q-learning: a case study in multirobot path planning. IEEE Trans. Syst. Man Cybern. Syst. 43(4), 814–831 (2013)CrossRefGoogle Scholar
  25. 25.
    M. Arbib, The Handbook of Brain Theory and Neural Networks (MIT Press, 2003)Google Scholar
  26. 26.
    B. Kosko, Fuzzy cognitive maps. Int. J. Man Mach. Stud. 24(1), 65–75 (1986)CrossRefGoogle Scholar
  27. 27.
    R. Axelrod, Structure of Decision: The Cognitive Maps of Political Elites (Princeton University Press, 2015)Google Scholar
  28. 28.
    D.O. Hebb, The Organization of Behavior: A Neuropsychological Theory (Psychology Press, 2005)Google Scholar
  29. 29.
    A. Konar, S. Pal, Modeling cognition with fuzzy neural nets. Fuzzy Theory Syst. 1341–1391 (1999)Google Scholar
  30. 30.
    D.H. Schunk, Learning Theories (Printice Hall Inc., New Jersey, 1996)Google Scholar
  31. 31.
    E. Guigon, B. Dorizzi, Y. Burnod, W. Schultz, Neural correlates of learning in the prefrontal cortex of the monkey: a predictive model. Cereb. Cortex 5(2), 135–147 (1995)CrossRefGoogle Scholar
  32. 32.
    M. Wooldridge, An Introduction to Multiagent Systems (Wiley, 2009)Google Scholar
  33. 33.
    J. Ferber, Multi-agent Systems: An Introduction to Distributed Artificial Intelligence (Addison-Wesley, 1999)Google Scholar
  34. 34.
    J. Liu, J. Wu, Multiagent Robotic Systems (CRC Press, 2001)Google Scholar
  35. 35.
    S. Kraus, Negotiation and cooperation in multi-agent environments. Artif. Intell. 94(1–2), 79–97 (1997)CrossRefGoogle Scholar
  36. 36.
    R. D’Andrea, G.E. Dullerud, Distributed control design for spatially interconnected systems. IEEE Trans. Autom. Control 48(9), 1478–1495 (2003)MathSciNetCrossRefGoogle Scholar
  37. 37.
    E.H. Durfee, V.R. Lesser, D.D. Corkill, Coherent cooperation among communicating problem solvers. IEEE Trans. Comput. 100(11), 1275–1291 (1987)CrossRefGoogle Scholar
  38. 38.
    M.P. Wellman, A market-oriented programming environment and its application to distributed multicommodity flow problems. JAIR 1, 1–23 (1993)CrossRefGoogle Scholar
  39. 39.
    R.G. Smith, The contract net protocol: high-level communication and control in a distributed problem solver. IEEE Trans. Comput. 12, 1104–1113 (1980)CrossRefGoogle Scholar
  40. 40.
    M. Georgeff, Communication and interaction in multi-agent planning. Read. Distrib. Artif. Intell. 313, 125–129 (1988)Google Scholar
  41. 41.
    T. Ishida, L. Gasser, M. Yokoo, Organization self-design of distributed production systems. IEEE Trans. Knowl. Data Eng. 4(2), 123–134 (1992)CrossRefGoogle Scholar
  42. 42.
    W. Ren, R.W. Beard, E.M. Atkins, A survey of consensus problems in multi-agent coordination, in Proceedings of IEEE American Control Conference, 2005, pp. 1859–1864Google Scholar
  43. 43.
    Y. Cao, W. Yu, W. Ren, G. Chen, An overview of recent progress in the study of distributed multi-agent coordination. IEEE Trans. Industr. Inf. 9(1), 427–438 (2013)CrossRefGoogle Scholar
  44. 44.
    T. Sandholm, Q. Huai, Nomad: mobile agent system for an internet-based auction house. IEEE Internet Comput. 4(2), 80–86 (2000)CrossRefGoogle Scholar
  45. 45.
    R.C. Arkin, Cooperation and Communication in Multi-agent Reactive Robotic Systems (Georgia Institute of Technology, 1994)Google Scholar
  46. 46.
    M. De Weerdt, B. Clement, Introduction to planning in multiagent systems. Multiagent Grid Syst. 5(4), 345–355 (2009)CrossRefGoogle Scholar
  47. 47.
    M.P. Wellman, W.E. Walsh, P.R. Wurman, J.K. MacKie-Mason, Auction protocols for decentralized scheduling. Games Econ. Behav. 35(1), 271–303 (2001)MathSciNetCrossRefGoogle Scholar
  48. 48.
    W.E. Walsh, M.P. Wellman, A market protocol for decentralized task allocation, in Proceedings of IEEE International Conference on Multi Agent Systems, 1998, pp. 325–332Google Scholar
  49. 49.
    E.D. Sacerdoti, The Nonlinear Nature of Plans, No. SRI-TN-101 (Stanford Research Inst Menlo Park CA, 1975)Google Scholar
  50. 50.
    K. Erol, J. Hendler, D.S. Nau, HTN planning: complexity and expressivity, in AAAI, vol. 94, 1994, pp. 1123–1128Google Scholar
  51. 51.
    M.J. Katz, J.S. Rosenschein, Plans for multiple agents. Distrib. Artif. Intell. 2, 197–228 (1989)CrossRefGoogle Scholar
  52. 52.
    E.P.D. Pednault, Formulating multiagent, dynamic-world problems in the classical planning framework, in Reasoning about Actions & Plans (1987), pp. 47–82Google Scholar
  53. 53.
    E.H. Durfee, C.L. Ortiz Jr., M.J. Wolverton, A survey of research in distributed, continual planning. AI Mag. 20(4), 13 (1999)Google Scholar
  54. 54.
    Y. Qi, Approaches to Multi-agent PlanningGoogle Scholar
  55. 55.
    I. Tsamardinos, M.E. Pollack, J.F. Horty, Merging plans with quantitative temporal constraints, temporally extended actions, and conditional branches, in AIPS, 2000, pp. 264–272Google Scholar
  56. 56.
    E.H. Durfee, V.R. Lesser, Planning coordinated actions in dynamic domains, in Proceedings of the DARPA Knowledge Based Planning Workshop, 1987, pp. 18.1–18.10Google Scholar
  57. 57.
    B.J. Grosz, S. Kraus, “The Evolution of Shared Plans”, in Foundations of Rational Agency (Springer, Netherlands, 1999), pp. 227–262CrossRefGoogle Scholar
  58. 58.
    D.A. Braun, P.A. Ortega, D.M. Wolpert, Nash equilibria in multi-agent motor interactions. PLoS Comput. Biol. 5(8), e1000468 (2009)MathSciNetCrossRefGoogle Scholar
  59. 59.
    S. Sen, M. Sekaran, J. Hale, Learning to coordinate without sharing information, in AAAI, 1994, pp. 426–431Google Scholar
  60. 60.
    S. Sen, S. Airiau, Emergence of norms through social learning. IJCAI 1507, 1512 (2007)Google Scholar
  61. 61.
    L. Panait, S. Luke, Cooperative multi-agent learning: the state of the art. Auton. Agent. Multi-agent Syst. 11(3), 387–434 (2005)CrossRefGoogle Scholar
  62. 62.
    R.J. Aumann, Game Theory (Palgrave Macmillan, London, 1989), pp. 1–53Google Scholar
  63. 63.
    E. Maskin, Nash equilibrium and welfare optimality. Rev. Econ. Stud. 66(1), 23–38 (1999)MathSciNetCrossRefGoogle Scholar
  64. 64.
    J. Hu, M.P. Wellman, Nash Q-learning for general-sum stochastic games. J. Mach. Learn. Res. 4, 1039–1069 (2003)MathSciNetzbMATHGoogle Scholar
  65. 65.
    A. Yamashita, M. Fukuchi, J. Ota, T. Arai, H. Asama, Motion planning for cooperative transportation of a large object by multiple mobile robots in a 3D environment, in Proceedings of IEEE International Conference on Robotics and Automation, 2000, pp. 3144–3151Google Scholar
  66. 66.
    S. Premvuti, S. Yuta, Consideration on the cooperation of multiple autonomous mobile robots, in Proceedings of IEEE International Workshop on Intelligent Robots and Systems, 1990, pp. 59–63Google Scholar
  67. 67.
    L.E. Parker, C. Touzet, Multi-robot learning in a cooperative observation task, in Distributed Autonomous Robotic Systems (Springer, 2000), pp. 391–401Google Scholar
  68. 68.
    S. Mahadevan, J. Connell, Automatic programming of behavior-based robots using reinforcement learning. Artif. Intell. 55(2–3), 311–365 (1992)CrossRefGoogle Scholar
  69. 69.
    M.J. Mataric, Interaction and Intelligent Behavior, No. AI-TR-1495 (Massachusetts Inst of Tech Cambridge Artificial Intelligence Lab, 1994)Google Scholar
  70. 70.
    S. Marsella, J. Adibi, Y. Al-Onaizan, G.A. Kaminka, I. Muslea, M. Tambe, On being a teammate: experiences acquired in the design of RoboCup teams, in Proceedings of Autonomous Agents, 1999, pp. 221–227Google Scholar
  71. 71.
    W. Burgard, M. Moors, D. Fox, R. Simmons, S. Thrun, Collaborative multi-robot exploration, in Proceedings of IEEE International Conference on Robotics and Automation, 2000, pp. 476–481Google Scholar
  72. 72.
    D. Fox, W. Burgard, H. Kruppa, S. Thrun, Collaborative multi-robot localization, in Mustererkennung (Springer, Berlin, Heidelberg, 1999), pp. 15–26Google Scholar
  73. 73.
    K. Inoue, J. Ota, T. Hirano, D. Kurabayashi, T. Arai, Iterative transportation by cooperative mobile robots in unknown environment, in Distributed Autonomous Robotic Systems, vol. 3 (Springer, Berlin, Heidelberg, 1998), pp. 3–12CrossRefGoogle Scholar
  74. 74.
    S. Nolfi, Evolutionary robotics: exploiting the full power of self-organization. Connect. Sci. 10, 167–184 (1998)CrossRefGoogle Scholar
  75. 75.
    E. Bonabeau, M. Dorigo, G. Theraulaz, Swarm Intelligence: From Natural to Artificial Systems (Oxford University Press, 1999)Google Scholar
  76. 76.
    T. Balch, Learning roles: behavioral diversity in robot teams, in College of Computing Technical Report GIT-CC-97–12 (Georgia Institute of Technology, Atlanta, Georgia 73, 1997)Google Scholar
  77. 77.
    T. Bäck, Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms (Oxford university press, 1996)Google Scholar
  78. 78.
    J. Brownlee, Clever Algorithms: Nature-Inspired Programming Recipes (2011)Google Scholar
  79. 79.
    J. Chakraborty, A. Konar, A distributed multi-robot path-planning using particle swarm optimization, in 2nd National Conference on Recent Trends in Information Systems, 2008, pp. 216–221Google Scholar
  80. 80.
    J. Chakraborty, A. Konar, U.K. Chakraborty, L.C. Jain, Distributed co-operative multi robot path-planning using differential evolution, in IEEE Congress on Evolutionary Computation, 2009, pp. 718–725Google Scholar
  81. 81.
    S.X. Yang, Y. Hu, M. Q.H. Meng, A knowledge based GA for path planning of multiple mobile robots in dynamic environments, in IEEE Conference on Robotics, Automation and Mechatronics, 2006, pp. 1–6Google Scholar
  82. 82.
    R.R. Sahoo, P. Rakshit, MdT Haider, S. Swarnalipi, B.K. Balabantaray, S. Mohapatra, Navigational path-planning of multi-robot using honey bee mating optimization algorithm (HBMO). Int. J. Comput. Appl. 27(11), 1–8 (2011)Google Scholar
  83. 83.
    A.K. Sadhu, P. Rakshit, A. Konar, A modified imperialist competitive algorithm for multi-robot stick-carrying application. Robot. Auton. Syst. 76, 15–35 (2016)CrossRefGoogle Scholar
  84. 84.
    P. Rakshit, A.K. Sadhu, P. Bhattacharjee, A. Konar, R. Janarthanan, Multi-robot box-pushing using non-dominated sorting bee colony optimization algorithm, in Proceedings of Swarm, Evolutionary and Memetic Computing Conference, vol. 7076, Dec 2011, pp. 601–609Google Scholar
  85. 85.
    J. Chakraborty, A. Konar, A. Nagar, S. Das, Rotation and translation selective pareto optimal solution to the box-pushing problem by mobile robots using NSGA-II, in Proceedings of IEEE Congress on Evolutionary Computation, 2009, pp. 2120–2126Google Scholar
  86. 86.
    S. Markon, D.V. Arnold, T. Back, T. Beielstein, H.G. Beyer, Thresholding-a selection operator for noisy ES, in Proceedings of IEEE Congress on Evolutionary Computation, vol. 1, 2001, pp. 465–472Google Scholar
  87. 87.
    H. Eskandari, C.D. Geiger, R. Bird, Handling uncertainty in evolutionary multiobjective optimization: SPGA, in Proceedings of IEEE Congress on Evolutionary Computation, 2007, pp. 4130–4137Google Scholar
  88. 88.
    E.J. Hughes, Constraint handling with uncertain and noisy multi-objective evolution, in Proceedings of IEEE Congress on Evolutionary Computation, vol. 2, 2001, pp. 963–970Google Scholar
  89. 89.
    D. Buche, P. Stoll, R. Dornberger, P. Koumoutsakos, Multiobjective evolutionary algorithm for the optimization of noisy combustion processes. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 32(4), 460–473 (2002)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.Department of Electronics and Telecommunication EngineeringJadavpur UniversityKolkataIndia
  2. 2.Department of Electronics and Telecommunication EngineeringJadavpur UniversityKolkataIndia

Personalised recommendations