Evaluating the Coordination of Agents in Multi-agent Reinforcement Learning

  • Sean L. BartonEmail author
  • Erin Zaroukian
  • Derrik E. Asher
  • Nicholas R. Waytowich
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 903)


The present study provides an in-depth analysis of inter-agent coordination through a complete exploration of agent behavioral dimensions. We evaluate the behavioral dimensions in a multi-agent predator-prey pursuit task where predator agent coordination necessarily exists due to a shared goal. We explore two conditions, one that is void of explicit coordination (fixed-strategy), and one that has the potential for explicit coordination (learning agents). This comprehensive evaluation of multi-agent behavioral dimensions provides theoretical evidence for true inter-agent coordination by a learning algorithm and the behavioral dimensions that agents coordinate in a cooperative task.


Coordination Multi-agent Reinforcement learning Predator-prey pursuit Teaming 



This research was sponsored by the Army Research Laboratory and was accomplished under Cooperative Agreement Number W911NF-18-2-0058. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.


  1. 1.
    Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)CrossRefGoogle Scholar
  2. 2.
    Matignon, L., Laurent, G.J., Le Fort-Piat, N.: Independent reinforcement learners in cooperative markov games: a survey regarding coordination problems. Knowl. Eng. Rev. 27, 1–31 (2012)CrossRefGoogle Scholar
  3. 3.
    Lowe, R., Wu, Y., Tamar, A., Harb, J., Pieter Abbeel, O., Mordatch, I.: Multi-agent actor-critic for mixed cooperative-competitive environments. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems 30, pp. 6382–6393. Curran Associates, Inc. (2017)Google Scholar
  4. 4.
    Sugihara, G., May, R., Ye, H., Hsieh, C.-h., Deyle, E., Fogarty, M., Munch, S.: Detecting causality in complex ecosystems. Science, 1227079 (2012)Google Scholar
  5. 5.
    Barton, S.L., Waytowich, N.R., Zaroukian, E., Asher, D.E.: Measuring collaborative emergent behavior in multi-agent reinforcement learning. In: 1st International Conference on Human Systems Engineering and Design. IHSED; SpringerGoogle Scholar
  6. 6.
    Brockman, G., et al.: OpenAI Gym. arXiv:1606.01540 [cs] (2016)
  7. 7.
    Clark, A.T., et al.: Spatial convergent cross mapping to detect causal relationships from short time series. Ecology. 96, 1174–1181 (2015)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Sean L. Barton
    • 1
    Email author
  • Erin Zaroukian
    • 1
  • Derrik E. Asher
    • 1
  • Nicholas R. Waytowich
    • 2
  1. 1.Computational & Information Sciences DirectorateU.S. Army Research LaboratoryAdelphiUSA
  2. 2.Human Research & Engineering DirectorateU.S. Army Research LaboratoryAdelphiUSA

Personalised recommendations