Abstract
In systems composed by a high number of highly coupled components, aligning the optimum of the system with the optimum of those individual components can be conflicting, especially in situations in which resources are scarce. In order to deal with this, many authors have proposed forms of biasing the optimization process. However, mostly, this works for cooperative scenarios. When resources are scarce, the components compete for them, thus those solutions are not necessarily appropriate. In this paper a new approach is proposed, in which there is a synergy between: (i) a global optimization process in which the system authority employs metaheuristics, and (ii) reinforcement learning processes that run at each component or agent. Both the agents and the system authority exchange solutions that are incorporated by the other party. The contributions are twofold: we propose a general scheme for such synergy and show its benefits in scenarios related to congestion games.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
A learning episode for the RL coincides with a generation for the metaheuristic.
References
Bazzan, A.L.C., Chira, C.: Hybrid evolutionary and reinforcement learning approach to accelerate traffic assignment (extended abstract). In: Bordini, R., Elkind, E., Weiss, G., Yolum, P. (eds.) IFAAMAS Proceedings of the 14th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2015), pp. 1723–1724, May 2015. http://www.aamas2015.com/en/AAMAS_2015_USB/aamas/p1723.pdf
Chira, C., Bazzan, A.L.C., Rossetti, R.J.F.: Multi-objective evolutionary traffic assignment. In: Proceedings of the 18th IEEE Annual Conference on Intelligent Transport Systems (ITSC 2015), pp. 1177–1182. IEEE, September 2015. https://doi.org/10.1109/ITSC.2015.194
Colby, M., Tumer, K.: Shaping fitness functions for coevolving cooperative multiagent systems. In: Proceedings of the Eleventh International Joint Conference on Autonomous Agents and Multiagent Systems, Valencia, Spain, pp. 425–432, June 2012
D’Acierno, L., Montella, B., De Lucia, F.: A stochastic traffic assignment algorithm based on ant colony optimisation. In: Dorigo, M., Gambardella, L.M., Birattari, M., Martinoli, A., Poli, R., Stützle, T. (eds.) ANTS 2006. LNCS, vol. 4150, pp. 25–36. Springer, Heidelberg (2006). https://doi.org/10.1007/11839088_3
Hines, G., Larson, K.: Learning when to take advice: a statistical test for achieving a correlated equilibrium. In: McAllester, D.A., Myllymki, P. (eds.) UAI, pp. 274–281. AUAI Press (2008)
LeBlanc, L.J., Morlok, E.K., Pierskalla, W.P.: An efficient approach to solving the road network equilibrium traffic assignment problem. Transp. Res. 9(5), 309–318 (1975)
de Lima Jr., F., Neto, A., de Melo, J.: Hybrid Metaheuristics Using Reinforcement Learning Applied to Salesman Traveling Problem. INTECH Open Access Publisher (2010)
Stefanello, F., Buriol, L., Hirsch, M., Pardalos, P., Querido, T., Resende, M., Ritt, M.: On the minimization of traffic congestion in road networks with tolls. Ann. Oper. Res. 249, 1–21 (2015)
Wardrop, J.G.: Some theoretical aspects of road traffic research. Proc. Inst. Civ. Eng. Part II 1(36), 325–362 (1952)
Watkins, C.J.C.H., Dayan, P.: Q-learning. Mach. Learn. 8(3), 279–292 (1992)
Wolpert, D.H., Sill, J., Tumer, K.: Reinforcement learning in distributed domains: beyond team games. In: International Joint Conferences on Artificial Intelligence (IJCAI), pp. 819–824 (2001)
Yen, J.Y.: Finding the k shortest loopless paths in a network. Manag. Sci. 17(11), 712–716 (1971). https://doi.org/10.1287/mnsc.17.11.712
Zhang, C., Abdallah, S., Lesser, V.: Integrating organizational control into multi-agent learning. In: ao Sichman, J.S., Decker, K.S., Sierra, C., Castelfranchi, C. (eds.) Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), Budapest, Hungary, pp. 757–764 (2009). http://mas.cs.umass.edu/paper/465
Acknowledgments
Ana Bazzan is partially supported by CNPq (grant 311558/2013-5).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Bazzan, A.L.C. (2018). Accelerating the Computation of Solutions in Resource Allocation Problems Using an Evolutionary Approach and Multiagent Reinforcement Learning. In: Sim, K., Kaufmann, P. (eds) Applications of Evolutionary Computation. EvoApplications 2018. Lecture Notes in Computer Science(), vol 10784. Springer, Cham. https://doi.org/10.1007/978-3-319-77538-8_14
Download citation
DOI: https://doi.org/10.1007/978-3-319-77538-8_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-77537-1
Online ISBN: 978-3-319-77538-8
eBook Packages: Computer ScienceComputer Science (R0)