Coordination of Communication in Robot Teams by Reinforcement Learning

  • Darío Maravall
  • Javier de Lope
  • Raúl Domínguez
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6686)


In Multi-agent systems, the study of language and communication is an active field of research. In this paper we present the application of Reinforcement Learning (RL) to the self-emergence of a common lexicon in robot teams. By modeling the vocabulary or lexicon of each agent as an association matrix or look-up table that maps the meanings (i.e. the objects encountered by the robots or the states of the environment itself) into symbols or signals we check whether it is possible for the robot team to converge in an autonomous, decentralized way to a common lexicon by means of RL, so that the communication efficiency of the entire robot team is optimal. We have conducted several experiments aimed at testing whether it is possible to converge with RL to an optimal Saussurean Communication System. We have organized our experiments alongside two main lines: first, we have investigated the effect of the team size centered on teams of moderated size in the order of 5 and 10 individuals, typical of multi-robot systems. Second, and foremost, we have also investigated the effect of the lexicon size on the convergence results. To analyze the convergence of the robot team we have defined the team’s consensus when all the robots (i.e. 100% of the population) share the same association matrix or lexicon. As a general conclusion we have shown that RL allows the convergence to lexicon consensus in a population of autonomous agents.


Multi-agent systems Multi-robot systems Dynamics of artificial languages Computational semiotics Reinforcement learning Self-collective coordination Language games Signaling games 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Dorigo, M., Stützle, T.: Ant Colony Optimization. The MIT Press, Cambridge (2004)zbMATHGoogle Scholar
  2. 2.
    Lenaerts, T., Jansen, B., Tuyls, K., De Vylder, B.: The evolutionary language game: An orthogonal approach. J. Theor. Biol. 235, 566–582 (2005)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Lewis, D.K.: Convention. Harvard University Press, Cambridge (1969)Google Scholar
  4. 4.
    Loula, A., Gudwin, R., El-Hani, C.N., Queiroz, J.: Emergence of self-organized symbol-based communication in artificial cretures. Cognitive Systems Research 11(2), 131–147 (2010)CrossRefGoogle Scholar
  5. 5.
    Maravall, D., de Lope, J., Domínguez, R.: Self-emergence of lexicon consensus in a population of autonomous agents by means of evolutionary strategies. In: Corchado, E., Graña Romay, M., Manhaes Savio, A. (eds.) HAIS 2010. LNCS, vol. 6077, pp. 77–84. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  6. 6.
    Maravall, D., de Lope, J., Domínguez, R.: Self-emergence of a common lexicon by evolution in teams of autonomous agents. Neurocomputing (in press)Google Scholar
  7. 7.
    Nowak, M.: The evolutionary language game. J. Theor. Biol. 200, 147–162 (1999)CrossRefGoogle Scholar
  8. 8.
    Peirce, C.S.: Selected Writings. Dover, New York (1966)Google Scholar
  9. 9.
    de Saussure, F.: Cours de Linguistic Général. Payot, Paris (1916); Ibidem Course on General Linguistics. English Edition. McGraw-Hill, New York (1969)Google Scholar
  10. 10.
    Steels, L., Kaplan, F.: Bootstrapping grounded word semantics. In: Briscoe, T. (ed.) Linguistic Evolution Through Language Acquisition, pp. 53–73. Cambridge University Press, Cambridge (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Darío Maravall
    • 1
  • Javier de Lope
    • 1
    • 2
  • Raúl Domínguez
    • 1
  1. 1.Cognitive Robotics Group, Dept. of Artificial IntelligenceUniversidad Politécnica de MadridSpain
  2. 2.Dept. Applied Intelligent SystemsUniversidad Politécnica de MadridSpain

Personalised recommendations