Impact of Neighboring Agent’s Characteristics with Q-Learning in Network Multi-agent System

  • Harjot KaurEmail author
  • Ginni Devi
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 955)


In a Network Multiagent System (NMAS), agent learning is one of the most investigated attribute and vital issues in terms of enhancing the global productivity of the system. The work in this paper implements a new approach that uses neighboring agent characteristics to improve the learning performance of an agent in NMAS considering social networks. The main contributions of this work are: First, it presents the basic principles of multi-agent learning while considering various learning issues. Second, it reviews the main research developments in the field of multi-agent learning. Third, it introduces a network multi-agent learning framework by considering the neighboring agent characteristics, i.e., the relative degree of neighboring nodes and past interaction histories of neighboring agents in a social network. Furthermore, the proposed framework is experimentally assessed by implementing various social networks (scale-free and small-world) in the form of a NMAS.


Q-learning Relative degree of neighboring nodes Multi-agent systems Social networks Multi-agent learning 


  1. 1.
    Claus, C., Boutilier, C.: The Dynamics of reinforcement learning in cooperative multiagent systems. In: Proceedings of the Fifteenth National Conference on Artificial Intelligence, pp. 746–752 (1998)Google Scholar
  2. 2.
    Franchi, E.: A multi-agent implementation of social networks. In: Proceedings of the 11th {WOA} 2010 Workshop, Dagli Oggetti Agli Agenti, Rimini, Italy (2010)Google Scholar
  3. 3.
    Franchi, E., Poggi, A.: Multi-Agent systems and social networks. Handbook of Research on Business Social Networking: Organizational, Managerial, and Technological Dimensions. IGI Global, Hershey, Pennsylvania, pp. 84–97 (2011)Google Scholar
  4. 4.
    Hao, J., Leung, H.: The dynamics of reinforcement social learning in cooperative multiagent systems. In: Proceedings of IJCAI 2013, Beijing, pp. 184–190 (2013)Google Scholar
  5. 5.
    Hao, J., Leung, H.: Achieving socially optimal outcomes in multiagent systems with reinforcement social learning. ACM Trans. Auton. Adapt. Syst. 8(3), 1–23 (2013)CrossRefGoogle Scholar
  6. 6.
    Hao, J., Huang, D., Cai, Y., Leung, H.: Networked reinforcement social learning towards coordination in cooperative multiagent systems. In: 26th IEEE International Conference on Tools with Artificial Intelligence (2014)Google Scholar
  7. 7.
    Hao, J., Leung, H.: Reinforcement social learning of coordination in cooperative multiagent systems (extended abstract). In: Proceedings of AAMAS 2013, Saint Paul, pp. 1321–1322 (2013)Google Scholar
  8. 8.
    Hao, J., Huang, D., Cai, Y., Leung, H.: The dynamics of reinforcement social learning in networked cooperative multiagent systems. In: Proceedings of AAAI 1958, pp. 111–122 (2017)Google Scholar
  9. 9.
    Hao, J.Y., Leung, H. F.: Learning to achieve social rationality using tag mechanism in repeated interactions. In: Proceedings of ICTAI 2011, pp. 148–155 (2011)Google Scholar
  10. 10.
    Jiang, Y., Zhou, Y., Wang, W.: Task allocation for undependable multiagent systems in social networks. IEEE Trans. Parallel Distrib. Syst. 24(8), 1671–1681 (2013)CrossRefGoogle Scholar
  11. 11.
    Jiang, Y., Jiang, J.C.: Understanding social networks from a multiagent perspective. IEEE Trans. Parallel Distrib. Syst. 25(10), 2743–2759 (2013)CrossRefGoogle Scholar
  12. 12.
    Kapetanakis, S., Kudenko, D.: Reinforcement learning of coordination in cooperative multiagent systems. In: Proceedings of the AAAI 2002, pp. 326–331 (2002)Google Scholar
  13. 13.
    Lauer, M., Riedmiller, M.: An algorithm for distributed reinforcement learning in cooperative multi-agent systems. In: Proceedings of ICML 2000, pp. 535–542 (2012)Google Scholar
  14. 14.
    Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., Mordatch, I.: Multi-Agent actor-critic for mixed cooperative-competitive environments. In: Proceedings of 2017 Conference on Neural Information Processing Systems (2017)Google Scholar
  15. 15.
    Matignon, L., Laurent, G.J., Le For-Piat, N.: Independent reinforcement learners in cooperative Markov games: a survey regarding coordination problems. Knowl. Eng. Rev. 27, 1–31 (2012)CrossRefGoogle Scholar
  16. 16.
    Matlock, M., Sen, S.: Effective tag mechanisms for evolving coordination. In: Proceedings of AAMAS 2007, pp. 1340–1347 (2007)Google Scholar
  17. 17.
    Matthew, E.G., Marie des, J.: Social network structures and their impact on multi-agent system dynamics. In: Proceedings of FLAIRS 2005, pp. 32–37 (2005)Google Scholar
  18. 18.
    Multi-Agent system. Wikipedia, Wikipedia Foundation. Accessed 5 Jan 2018
  19. 19.
    Glavic, M.: Agents and Multi-Agent Systems: A Short Introduction for Power Engineers.
  20. 20.
    Silva, F., Glatt, R., Costa, A.H.R.: Simultaneously learning and advising in multi-agent reinforcement learning. In: Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems (AAMAS 2017). International Foundation for Autonomous Agents and Multiagent Systems, Richland, pp. 1100–1108 (2017)Google Scholar
  21. 21.
    Tuyls, K., Weiss, G.: Multiagent learning: basics, challenges, and prospects. AI Mag. 33(3), 41–50 (2012)CrossRefGoogle Scholar
  22. 22.
    Wooldridge, M.: An Introduction to Multiagent Systems. John Wiley, Hoboken (2002)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringGuru Nanak Dev University Regional CampusGurdaspurIndia

Personalised recommendations