Abstract
Multi agent systems are a well-defined solution for implementing dynamic complex environments. One of the open issues of these systems is credit assignment problem. The main concern of credit assignment problem is to properly distributing feedback of overall performance, and brings about learning in each individual agent. In this paper a genetic framework for solving Multi-agent credit assignment problem is proposed. Our framework, Negotiation Based Credit Assignment, NBCA, applies negotiation for both enriching agents’ knowledge as well as organizing populations by a mode analyzer. The proposed architecture includes a mentor agent which responsible for credit assignment without any context related information leading to a general solution. Furthermore, the mentor agent does not receive any information regarding correctness of a particular agent’s behavior. Carry and non-Carry cases have been considered for evaluating this method. In addition, the effects of noise as a source of uncertainty on NBCA performance are examined. Our finding indicated that the proposed method is superior to previous credit assignment approaches. This is due to the argumentation and negotiation features of multi agent systems that are used to accomplish team learning and credit assignment respectively. The analysis of obtained results which are theoretically discussed, demonstrate that, in comparison with KEBCA (OR-type), our approach performs better than KEBCA after 5000 trials in 0% noisy environment. However, it performs worse than KEBCA in 10% and 30% noisy environment.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alonso, E., D’inverno, M., Kudenko, D., Luck, M., Noble, J.: Learning in multi-agent systems. The Knowledge Engineering Review 16(3), 277–284 (2001)
Harati, A., Ahmadabadi, M.: Experimental analysis of knowledge based multiagent credit assignment. In: Rajapakse, J.C., Wang, L. (eds.) Neural Information Processing: Research and Development. STUDFUZZ, vol. 152, pp. 437–459. Springer, Heidelberg (2004)
Schneider, J.G., Wong, W.-K., Moore, A.W., Riedmiller, M.A.: Distributed value functions. In: Proceedings of the Sixteenth International Conference on Machine Learning, ICML, pp. 371–378. Morgan Kaufmann Publishers Inc., San Francisco (1999)
Holland, J.H.: Properties of the bucket brigade. In: Proceedings of the 1st International Conference on Genetic Algorithms, pp. 1–7. L. Erlbaum Associates Inc., Hillsdale (1985)
Harati, A., Ahmadabadi, M.: A new approach to credit assignment in a team of cooperative q-learning agents. In: 2002 IEEE International Conference on Systems, Man and Cybernetics, vol. 4, p. 6 (October 2002)
Beer, M., D’inverno, M., Luck, M., Jennings, N., Preist, C., Schroeder, M.: Negotiation in multi-agent systems. Knowl. Eng. Rev. 14(3), 285–289 (1999)
Goldberg, D.E.: Genetic Algorithms in Search, Optimization and Machine Learning, 1st edn. Addison-Wesley Longman Publishing Co., Inc., Boston (1989)
Srivastava, R.P., Goldberg, D.E.: Verification of the theory of genetic and evolutionary continuation (2001)
Cantú-Paz, E., Goldberg, D.: Are multiple runs of genetic algorithms better than one? In: Cantú-Paz, E., et al. (eds.) GECCO 2003. LNCS, vol. 2723, pp. 801–812. Springer, Heidelberg (2003)
Harati, A., Ahmadabadi, M., Araabi, B.: Knowledge-based multiagent credit assignment: A study on task type and critic information. IEEE Systems Journal 1(1), 55–67 (2007)
Mao, W., Gratch, J.: The social credit assignment problem. In: Rist, T., Aylett, R.S., Ballin, D., Rickel, J. (eds.) IVA 2003. LNCS (LNAI), vol. 2792, pp. 39–47. Springer, Heidelberg (2003)
Jones, J., Goel, A.: Hierarchical judgement composition: Revisiting the structural credit assignment problem. In: Proceedings of the AAAI Workshop on Challenges in Game AI, San Jose, CA, USA, pp. 67–71 (2004)
Agogino, A.K., Tumer, K.: Unifying temporal and structural credit assignment problems. In: Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004, vol. 2, pp. 980–987. IEEE Computer Society, Washington, DC (2004)
Arai, S., Sycara, K., Payne, T.R.: Experience-based reinforcement learning to acquire effective behavior in a multi-agent domain. In: Mizoguchi, R., Slaney, J.K. (eds.) PRICAI 2000. LNCS, vol. 1886, pp. 125–135. Springer, Heidelberg (2000)
Rahaie, Z., Beigy, H.: Toward a solution to multi-agent credit assignment problem. In: International Conference of Soft Computing and Pattern Recognition, SOCPAR 2009, pp. 563–568 (December 2009)
Arai, S., Miyazaki, K., Kobayashi, S.: Multi-agent reinforcement learning for crane control problem: designing rewards for conflict resolution. In: Proceedings of the The Fourth International Symposium on Autonomous Decentralized Systems. Integration of Heterogeneous Systems, pp. 310–317 (1999)
León, A., Morales, E.F., Altamirano, L., Ruiz, J.R.: Teaching a robot to perform task through imitation and on-line feedback. In: San Martin, C., Kim, S.-W. (eds.) CIARP 2011. LNCS, vol. 7042, pp. 549–556. Springer, Heidelberg (2011)
Knox, W.B., Stone, P., Breazeal, C.: Training a robot via human feedback: A case study. In: Herrmann, G., Pearson, M.J., Lenz, A., Bremner, P., Spiers, A., Leonards, U. (eds.) ICSR 2013. LNCS, vol. 8239, pp. 460–470. Springer, Heidelberg (2013)
Li, K., Fialho, A., Kwong, S., Zhang, Q.: Adaptive operator selection with bandits for a multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on Evolutionary Computation 18(1), 114–130 (2014)
Soria Alcaraz, J.A., Ochoa, G., Carpio, M., Puga, H.: Evolvability metrics in adaptive operator selection. In: Proceedings of the 2014 Conference on Genetic and Evolutionary Computation, GECCO 2014, pp. 1327–1334. ACM, New York (2014)
A genetic algorithm based robust learning credit assignment cerebellar model articulation controller. Applied Soft Computing 4(4), 357–367 (2004)
Jin, Y., Branke, J.: Evolutionary optimization in uncertain environments-a survey. IEEE Transactions on Evolutionary Computation 9(3), 303–317 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Pashaei, K., Taghiyareh, F., Badie, K. (2014). A Negotiation-Based Genetic Framework for Multi-Agent Credit Assignment. In: Müller, J.P., Weyrich, M., Bazzan, A.L.C. (eds) Multiagent System Technologies. MATES 2014. Lecture Notes in Computer Science(), vol 8732. Springer, Cham. https://doi.org/10.1007/978-3-319-11584-9_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-11584-9_6
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11583-2
Online ISBN: 978-3-319-11584-9
eBook Packages: Computer ScienceComputer Science (R0)