Skip to main content

Advertisement

SpringerLink
Log in

On nash equilibria in normal-form games with vectorial payoffs

  • Published:
Autonomous Agents and Multi-Agent Systems Aims and scope Submit manuscript

Abstract

We provide an in-depth study of Nash equilibria in multi-objective normal-form games (MONFGs), i.e., normal-form games with vectorial payoffs. Taking a utility-based approach, we assume that each player’s utility can be modelled with a utility function that maps a vector to a scalar utility. In the case of a mixed strategy, it is meaningful to apply such a scalarisation both before calculating the expectation of the payoff vector as well as after. This distinction leads to two optimisation criteria. With the first criterion, players aim to optimise the expected value of their utility function applied to the payoff vectors obtained in the game. With the second criterion, players aim to optimise the utility of expected payoff vectors given a joint strategy. Under this latter criterion, it was shown that Nash equilibria need not exist. Our first contribution is to provide a sufficient condition under which Nash equilibria are guaranteed to exist. Secondly, we show that when Nash equilibria do exist under both criteria, no equilibrium needs to be shared between the two criteria, and even the number of equilibria can differ. Thirdly, we contribute a study of pure strategy Nash equilibria under both criteria. We show that when assuming quasiconvex utility functions for players, the sets of pure strategy Nash equilibria under both optimisation criteria are equivalent. This result is further extended to games in which players adhere to different optimisation criteria. Finally, given these theoretical results, we construct an algorithm to compute all pure strategy Nash equilibria in MONFGs where players have a quasiconvex utility function.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Bergstresser, K., & Yu, P. L. (1977). Domination structures and multicriteria problems in n-person games. Theory and Decision, 8(1), 5–48. https://doi.org/10.1007/BF00133085.

    Article  MathSciNet  MATH  Google Scholar 

  2. Blackwell, D. (1954). An analog of the minimax theorem for vector payoffs. Pacific Journal of Mathematics, 6(1), 1–8. https://doi.org/10.2140/pjm.1956.6.1.

    Article  MathSciNet  MATH  Google Scholar 

  3. Breinbjerg, J. (2017). Equilibrium arrival times to queues with general service times and non-linear utility functions. European Journal of Operational Research, 261(2), 595–605. https://doi.org/10.1016/j.ejor.2017.03.010.

    Article  MathSciNet  MATH  Google Scholar 

  4. Busoniu, L., Babuska, R., & Schutter, B. D. (2008). A comprehensive survey of multiagent reinforcement learning. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 38(2), 156–172. https://doi.org/10.1109/TSMCC.2007.913919.

    Article  Google Scholar 

  5. Corley, H. W. (1985). Games with vector payoffs. Journal of Optimization Theory and Applications, 47(4), 491–498. https://doi.org/10.1007/BF00942194.

    Article  MathSciNet  MATH  Google Scholar 

  6. Corley, H. W. (2020). A regret-based algorithm for computing all pure nash equilibria for noncooperative games in normal form. Theoretical Economics Letters, 10(06), 1253–1259. https://doi.org/10.4236/tel.2020.106076.

    Article  Google Scholar 

  7. Dragomir, S. S., & Pearce, C. E. (2012). Jensen’s inequality for quasiconvex functions. Numerical Algebra, Control and Optimization, 2(2), 279–291. https://doi.org/10.3934/naco.2012.2.279.

    Article  MathSciNet  MATH  Google Scholar 

  8. Fudenberg, D., & Tirole, J. (1991). Game theory. MIT Press.

    MATH  Google Scholar 

  9. Hayes, C. F., Rădulescu, R., Bargiacchi, E., Källström, J., Macfarlane, M., Reymond, M., et al. (2022). A practical guide to multi-objective reinforcement learning and planning. Autonomous Agents and Multi-Agent Systems, 36(1), 26. https://doi.org/10.1007/s10458-022-09552-y.

    Article  Google Scholar 

  10. Hayes, C. F., Reymond, M., Roijers, D. M., Howley, E., & Mannion, P. (2021). Distributional Monte Carlo Tree Search for Risk-Aware and Multi-Objective Reinforcement Learning. In Proceedings of the 20th international conference on autonomous agents and multiagent systems (AAMAS 2021) (p. 3). IFAAMAS

  11. He, J., Li, Y., Li, H., Tong, H., Yuan, Z., Yang, X., & Huang, W. (2020). Application of game theory in integrated energy system systems: A review. IEEE Access, 8, 93380–93397. https://doi.org/10.1109/ACCESS.2020.2994133.

    Article  Google Scholar 

  12. Ismaili, A. (2018). On existence, mixtures, computation and efficiency in multi-objective games. In T. Miller, N. Oren, Y. Sakurai, I. Noda, B. T. R. Savarimuthu, & T. Cao Son (Eds.), PRIMA 2018: Principles and practice of multi-agent systems (pp. 210–225). Springer.

    Chapter  Google Scholar 

  13. Koppelman, F. S. (1981). Non-linear utility functions in models of travel choice behavior. Transportation, 10(2), 127–146. https://doi.org/10.1007/BF00165262.

    Article  Google Scholar 

  14. Leyton-Brown, K., & Shoham, Y. (2008). Essentials of game theory: A concise multidisciplinary introduction. Synthesis Lectures on Artificial Intelligence and Machine Learning, 2(1), 1–88.

    Article  MATH  Google Scholar 

  15. Lozovanu, D., Solomon, D., & Zelikovsky, A. (2005). Multiobjective games and determining Pareto–Nash equilibria. The Bulletin of Academy of Sciences of Moldova, Mathematics, 3(49), 115–122.

    MathSciNet  MATH  Google Scholar 

  16. Nash, J. (1951). Non-cooperative games. The Annals of Mathematics, 54(2), 286. https://doi.org/10.2307/1969529.

    Article  MathSciNet  MATH  Google Scholar 

  17. Nowé, A., Vrancx, P., & De Hauwere, Y. M. (2012). Game theory and multi-agent reinforcement learning. In M. Wiering & M. van Otterlo (Eds.), Reinforcement learning: State-of-the-art (pp. 441–470). Springer. https://doi.org/10.1007/978-3-642-27645-3_14.

    Chapter  Google Scholar 

  18. Porter, R., Nudelman, E., & Shoham, Y. (2008). Simple search methods for finding a Nash equilibrium. Games and Economic Behavior, 63(2), 642–662. https://doi.org/10.1016/j.geb.2006.03.015.

    Article  MathSciNet  MATH  Google Scholar 

  19. Rădulescu, R. (2021). Decision making in multi-objective multi-agent systems: A utility-based perspective. Ph.D. thesis, Vrije Universiteit Brussel, Brussels

  20. Rădulescu, R., Mannion, P., Roijers, D. M., & Nowé, A. (2020). Multi-objective multi-agent decision making: A utility-based analysis and survey. Autonomous Agents and Multi-Agent Systems, 34(1), 10–10. https://doi.org/10.1007/s10458-019-09433-x.

    Article  Google Scholar 

  21. Rădulescu, R., Mannion, P., Zhang, Y., Roijers, D. M., & Nowé, A. (2020). A utility-based analysis of equilibria in multi-objective normal-form games. The Knowledge Engineering Review, 35, e32. https://doi.org/10.1017/S0269888920000351.

    Article  Google Scholar 

  22. Rădulescu, R., Verstraeten, T., Zhang, Y., Mannion, P., Roijers, D. M., & Nowé, A. (2022). Opponent learning awareness and modelling in multi-objective normal form games. Neural Computing and Applications, 34(3), 1759–1781. https://doi.org/10.1007/s00521-021-06184-3.

    Article  Google Scholar 

  23. Roijers, D. M., Steckelmacher, D., & Nowé, A. (2018). Multi-objective reinforcement learning for the expected utility of the return. In Adaptive learning agents workshop at AAMAS Stockholm, Sweden

  24. Roijers, D. M., Vamplew, P., Whiteson, S., & Dazeley, R. (2013). A survey of multi-objective sequential decision-making. Journal of Artificial Intelligence Research, 48, 67–113. https://doi.org/10.1613/jair.3987.

    Article  MathSciNet  MATH  Google Scholar 

  25. Roijers, D. M., & Whiteson, S. (2017). Multi-objective decision making. Synthesis Lectures on Artificial Intelligence and Machine Learning, 11(1), 1–129.

    Article  MATH  Google Scholar 

  26. Roijers, D. M., Zintgraf, L. M., Libin, P., Reymond, M., Bargiacchi, E., & Nowé, A. (2021). Interactive multi-objective reinforcement learning in multi-armed bandits with gaussian process utility models. In F. Hutter, K. Kersting, J. Lijffijt, & I. Valera (Eds.), Machine learning and knowledge discovery in databases (pp. 463–478). Springer. https://doi.org/10.1007/978-3-030-67664-3_28.

    Chapter  Google Scholar 

  27. Rosenthal, R. W. (1973). A class of games possessing pure-strategy Nash equilibria. International Journal of Game Theory, 2(1), 65–67. https://doi.org/10.1007/BF01737559.

    Article  MathSciNet  MATH  Google Scholar 

  28. Roughgarden, T. (2016). Introduction and Examples (pp. 1–10). Cambridge University Press. https://doi.org/10.1017/CBO9781316779309.002.

    Book  MATH  Google Scholar 

  29. Röpke, W. (2021). Communication in multi-objective games. Master’s thesis, Vrije Universiteit Brussel

  30. Röpke, W., Roijers, D. M., Nowé, A., & Rădulescu, R. (2021). On nash equilibria for multi-objective normal-form games under scalarised expected returns versus expected scalarised returns. In MODeM workshop

  31. Röpke, W., Rădulescu, R., Roijers, D. M., & Nowé, A. (2021). Communication strategies in multi-objective normal-form games. In Adaptive and learning agents workshop at AAMAS

  32. Samuelson, L. (2016). Game theory in economics and beyond. Journal of Economic Perspectives, 30(4), 107–130. https://doi.org/10.1257/jep.30.4.107.

    Article  MathSciNet  Google Scholar 

  33. Shapley, L. S., & Rigby, F. D. (1959). Equilibrium points in games with vector payoffs. Naval Research Logistics Quarterly, 6(1), 57–61. https://doi.org/10.1002/nav.3800060107.

    Article  MathSciNet  Google Scholar 

  34. Stein, N. D., Ozdaglar, A., & Parrilo, P. A. (2008). Separable and low-rank continuous games. International Journal of Game Theory, 37(4), 475–504. https://doi.org/10.1007/s00182-008-0129-2.

    Article  MathSciNet  MATH  Google Scholar 

  35. Varian, H. R. (2014). Chapter 3—preferences. In Intermediate microeconomics: A modern approach: Ninth Edition. W. W. Norton & Company

  36. Voorneveld, M., Grahn, S., & Dufwenberg, M. (2000). Ideal equilibria in noncooperative multicriteria games. Mathematical Methods of Operations Research, 52(1), 65–77. https://doi.org/10.1007/s001860000069.

    Article  MathSciNet  MATH  Google Scholar 

  37. Wierzbicki, A. P. (1995). Multiple criteria games—Theory and applications. Journal of Systems Engineering and Electronics, 6(2), 65–81.

    Google Scholar 

  38. Zapata, A., Mármol, A. M., Monroy, L., & Caraballo, M. A. (2019). A maxmin approach for the equilibria of vector-valued games. Group Decision and Negotiation, 28(2), 415–432. https://doi.org/10.1007/s10726-018-9608-4.

    Article  Google Scholar 

  39. Zeleny, M. (1975). Games with multiple payoffs. International Journal of Game Theory, 4(4), 179–191. https://doi.org/10.1007/BF01769266.

    Article  MathSciNet  MATH  Google Scholar 

  40. Zintgraf, L. M., Roijers, D. M., Linders, S., Jonker, C. M., & Nowé, A. (2018). Ordered Preference Elicitation Strategies for Supporting Multi-Objective Decision Making. In Proceedings of the 17th international conference on autonomous agents and multiagent systems, AAMAS ’18 (pp. 1477–1485). International foundation for autonomous agents and multiagent systems, Stockholm, Sweden

Download references

Acknowledgements

The first author is supported by the Research Foundation – Flanders (FWO), grant number 1197622N. This research was supported by funding from the Flemish Government under the “Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen” program.

Author information

Authors and Affiliations

  1. Artificial Intelligence Lab, Vrije Universiteit Brussel, Brussels, Belgium

    Willem Röpke, Diederik M. Roijers, Ann Nowé & Roxana Rădulescu

  2. Urban Innovation and R&D, City of Amsterdam, Amsterdam, The Netherlands

    Diederik M. Roijers

Authors
  1. Willem Röpke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Diederik M. Roijers
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ann Nowé
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Roxana Rădulescu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Willem Röpke.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Part of this work was carried out by the first author for his thesis [29] under the supervision of the other authors. Some preliminary results in this article were presented in the Multi-Objective Decision Making Workshop 2021 [30].

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Röpke, W., Roijers, D.M., Nowé, A. et al. On nash equilibria in normal-form games with vectorial payoffs. Auton Agent Multi-Agent Syst 36, 53 (2022). https://doi.org/10.1007/s10458-022-09582-6

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10458-022-09582-6

Keywords

Search

Navigation