Skip to main content
SpringerLink
Log in

Social or Individual Learning? An Aggregated Solution for Coordination in Multiagent Systems

  • Published:
Journal of Systems Science and Systems Engineering Aims and scope Submit manuscript

Abstract

There are mainly two different ways of learning for animals and humans: trying on yourself through interactions or imitating/copying others through communication/observation. How these two learning strategies differ and what roles they are playing in achieving coordination among individuals are two challenging problems for researchers from various disciplines. In multiagent systems, most existing work simply focuses on individual learning for achieving coordination among agents. The social learning perspective has been largely neglected. Against this background, this article contributes by proposing an integrated solution to decision making between social learning and individual learning in multiagent systems. Two integration modes have been proposed that enable agents to choose in between these two learning strategies, either in a fixed or in an adaptive manner. Experimental evaluations have shown that these two kinds of leaning strategies have different roles in maintaining efficient coordination among agents. These differences can reveal some significant insights into the manipulation and control of agent behaviors in multiagent systems, and also shed light on understanding the social factors in shaping coordinated behaviors in humans and animals.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Acemoglu D. and Ozdaglar A. (2011). Opinion dynamics and learning in social networks. Dynamic Games and Applications, 1(1): 3–49.

    Article  MathSciNet  MATH  Google Scholar 

  2. Barabási A. and Albert R. (2002). Statistical mechanics of complex networks. Rev. Modern Phys., 74: 47–97.

    Article  MathSciNet  MATH  Google Scholar 

  3. Castellano C., Fortunato S., and Loreto V. (2009). Statistical physics of social dynamics. Reviews of modern physics, 81(2): 591.

    Article  Google Scholar 

  4. Hegselmann R., Krause U. et al. (2002). Opinion dynamics and bounded confidence models, analysis, and simulation. Journal of artificial societies and social simulation, 5(3): 1–33.

    Google Scholar 

  5. Hethcote H. W. (2000). The mathematics of infectious diseases. SIAM review, 42(4): 599–653.

    Article  MathSciNet  MATH  Google Scholar 

  6. Hofmann L., Chakraborty N., and Sycara K. (2011). The evolution of cooperation in self-interested agent societies: a critical study. In Proc. of 10th AAMAS, pp. 685–692, Toronto, Canada, May 10-14, 2011.

    Google Scholar 

  7. Laland K. N. (2004). Social learning strategies. Learning & behavior, 32(1): 4–14.

    Article  Google Scholar 

  8. Mihaylov M., Tuyls K., and Nowé A. (2014). A decentralized approach for convention emergence in multi-agent systems. Autonomous Agents and Multi-Agent Systems, 28(5): 749–778.

    Article  Google Scholar 

  9. Mukherjee P., Sen S., and Airiau S. (2008). Norm emergence under constrained interactions in diverse societies. In Proc. of 7th AAMAS, pp. 779–786, Estoril, Portugal, May 12-16, 2008.

    Google Scholar 

  10. Rendell L., Fogarty L., Hoppitt W. J., Morgan T. J., Webster M. M., and Laland K. N. (2011). Cognitive culture: theoretical and empirical insights into social learning strategies. Trends in cognitive sciences, 15(2): 68–76.

    Article  Google Scholar 

  11. Savarimuthu B., Arulanandam R., and Purvis M. (2011). Aspects of active norm learning and the effect of lying on norm emergence in agent societies. Agents in Principle, Agents in Practice, LNAI, 7047: 36–50.

    Article  Google Scholar 

  12. Sen O. and Sen S. (2010). Effects of social network topology and options on norm emergence. Coordination, Organizations, Institutions and Norms in Agent Systems V, LNAI, 6069: 211–222.

    Article  Google Scholar 

  13. Sen S. and Airiau S. (2007). Emergence of norms through social learning. In Proc. of 20th IJCAI, pp. 1507–1512, Hyderabad, India, January 6-12, 2007.

    Google Scholar 

  14. Shibusawa R. and Sugawara T. (2014). Norm emergence via influential weight propagation in complex networks. In Network Intelligence Conference (ENIC), 2014 European. IEEE, pp. 30–37, Wroclaw, Poland, September 29-30, 2014.

    Google Scholar 

  15. Shoham Y. and Tennenholtz M. (1997). On the emergence of social conventions: modeling, analysis, and simulations. Artif. Intel., 94(1-2): 139–166.

    Article  MATH  Google Scholar 

  16. Szabo G. and Fáth G. (2007). Evolutionary games on graphs. Phys. Rep., 446(4-6): 97–216.

    Article  MathSciNet  Google Scholar 

  17. Verhagen H. (2001). Simulation of the learning of norms. Social Science Computer Review, 19(3): 296–306.

    Article  Google Scholar 

  18. Villatoro D., Sabater-Mir J., and Sen S. (2011). Social instruments for robust convention emergence. In Proc. of 22nd IJCAI, pp. 420–425, Barcelona, Catalonia, Spain, July 16-22, 2011, AAAI Press.

    Google Scholar 

  19. Villatoro D., Sen S., and Sabater-Mir J. (2009). Topology and memory effect on convention emergence. In Proc. of WIIAT’09, pp. 233–240, Milan, Italy, September 15-18, 2009.

    Google Scholar 

  20. Watts D. and Strogatz S. (1998). Collective dynamics of smallworld networks. Nature, 393: 440–442.

    Article  MATH  Google Scholar 

  21. Watts D. J. and Dodds P. S. (2007). Influentials, networks, and public opinion formation. Journal of consumer research, 34(4): 441–458.

    Article  Google Scholar 

  22. Young H. (1996). The economics of convention. J. Econ. Pers., 10(2): 105–122.

    Article  Google Scholar 

  23. Yu C., Lv H., Sen S., Hao J., Ren F., and Liu R. (2016). An adaptive learning framework for efficient emergence of social norms: (extended abstract). In International Conference on Autonomous Agents Multiagent Systems, pp. 1307–1308, Singapore, May 9-13, 2016.

    Google Scholar 

  24. Yu C., Liu R., and Wang Y. (2016). Social learning in networked agent societies. In IEEE International Conference on Agents, pp. 57–62, Matsue, Japan, September 28-30, 2016.

    Google Scholar 

  25. Yu C., Lv H., Ren F., Bao H., and Hao J. (2015). Hierarchical learning for emergence of social norms in networked multiagent systems. In Australasian Joint Conference on Artificial Intelligence. pp. 630–643, Canberrra, Australia, Nov. 30-Dec. 4, 2015, Springer.

    Google Scholar 

  26. Yu C., Lv H., Sen S., Ren F., and Tan G. (2016). Adaptive learning for efficient emergence of social norms in networked multiagent systems. In Pacific Rim International Conference on Trends in Artificial Intelligence, pp. 805–818, Phuket, Thailand, August 22-26, 2016.

    Google Scholar 

  27. Yu C., Tan G., Lv H., Wang Z., Meng J., Hao J., and Ren F. (2016). Modelling adaptive learning behaviours for consensus formation in human societies. Scientific Reports, 6: 27626.

    Article  Google Scholar 

  28. Yu C., Wang Z., Lv H., Bao H., and Li Y. (2017). Collective Learning and Information Diffusion for Efficient Emergence of Social Norms. Springer, Singapore.

    Book  Google Scholar 

  29. Yu C., Y. Chen, Lv H., Ren J., Ge H., and Sun L. (2017). Neural learning for the emergence of social norms in multiagent systems. In IEEE International Conference on Agents, pp. 40–45, Beijing, China, July 6-9, 2017.

    Google Scholar 

  30. Yu C., Zhang M., and Ren F. (2014). Collective learning for the emergence of social norms in networked multiagent systems. IEEE Transactions on Cybernetics, 44(12): 2342–2355.

    Article  Google Scholar 

  31. Yu C., Zhang M., Ren F., and Luo X. (2013). Emergence of social norms through collective learning in networked agent societies. In Proc. of AAMAS2013, pp. 475–482, Saint Paul, USA, May 6-10, 2013.

    Google Scholar 

  32. Yu C., Zhang M., and Ren F. (2013). Emotional multiagent reinforcement learning in social dilemmas. In International Conference on Principles and Practice of Multi-Agent Systems, pp. 372–387, Dunedin, New Zealand, December 1-6, 2013.

    Google Scholar 

  33. Yu C., Zhang M., Ren F., and Tan G. (2015). Emotional multiagent reinforcement learning in spatial social dilemmas. IEEE Transactions on Neural Networks & Learning Systems, 26(12): 3083–3096

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

This work is supported by the NationalNatural Science Foundation of China under Grant 61771089, 61502072, 61572104 and 61403059, Fundamental Research Funds for the Central Universities of China under Grant DUT16RC(4)17, and Post-Doctoral Science Foundation of China under Grants 2014M561229 and 2015T80251.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Dalian University of Technology, Dalian, 116024, China

    Bingcai Chen, Chao Yu, Qishuai Diao, Rui Liu & Yuliang Wang

  2. School of Computer Science and Technology, Xinjiang Normal University, Urumqi, 830054, China

    Bingcai Chen

Authors
  1. Bingcai Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chao Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qishuai Diao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rui Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuliang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chao Yu.

Additional information

Bingcai Chen received his MS degree and Ph.D. degree in Information and Communication Engineering from Harbin Institute of Technology (HIT), Harbin, China, in 2003 and 2007, respectively. He has been a visiting scholar in University of British Columbia, Canada, in 2015.

He is now an Associate Professor in the school of computer science and technology at Dalian University of Technology, Dalian, China, and in the School of Computer Science and Technology, Xinjiang Normal University, Urumqi, China. His current research interests include machine learning, computer vision, etc.

He received National Science Foundation Career Award of China in 2009. He is severing as a reviewer for project proposals to National science foundation of China, Ministry of Education of China. He is also serving as a reviewer for some refereed Journals including IEEE Transaction on circuit and systems for video technology, Journal of Electronics, Journal of Communication, et al.

Chao Yu received his Ph.D. degree in computer science from the University of Wollongong, Australia, in 2014. Now, he is an Associated Professor in the School of Computer Science and Technology at the Dalian University of Technology, Dalian, China. His research interests include multi-agent systems and reinforcement learning.

Qishuai Diao is an undergraduate student in the School of Computer Science and Technology at the Dalian University of Technology, Dalian, China. His research interests include multi-agent systems and complex networks.

Rui Liu is an undergraduate student in the School of Computer Science and Technology at the Dalian University of Technology, Dalian, China. His research interests include multi-agent learning and complex networks.

Yuliang Wang is an undergraduate student in the School of Computer Science and Technology at the Dalian University of Technology, Dalian, China. His research interests include multi-agent systems and multiagent learning.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, B., Yu, C., Diao, Q. et al. Social or Individual Learning? An Aggregated Solution for Coordination in Multiagent Systems. J. Syst. Sci. Syst. Eng. 27, 180–200 (2018). https://doi.org/10.1007/s11518-018-5363-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11518-018-5363-y

Keywords

Search

Navigation