Advertisement

On Incremental Passivity in Network Games

  • Lacra PavelEmail author
Conference paper
Part of the Static & Dynamic Game Theory: Foundations & Applications book series (SDGTFA)

Abstract

In this paper, we show how control principles and passivity properties can be used in analysing and designing learning rules/dynamics for agents playing a network game. We focus on two instances: (1) agents learning about the others’ actions and (2) agents learning about the game (reinforcement-learning). In both cases, we show the trade-off between game properties and agent learning dynamics properties, underpinned by passivity/monotonicity and the balancing principle.

References

  1. 1.
    Alpcan, T., Başar, T.: Distributed Algorithms for Nash Equilibria of Flow Control Games, pp. 473–498. Birkhäuser Boston (2005)Google Scholar
  2. 2.
    Başar, T., Olsder, G.: Dynamic Noncooperative Game Theory: Second Edition. Classics in Applied Mathematics, SIAM (1999)Google Scholar
  3. 3.
    Bauso, D.: Game Theory with Engineering Applications. SIAM, ser. Advances in Design and Control Series (2016)Google Scholar
  4. 4.
    Benaïm, M.: Dynamics of stochastic approximation algorithms. in Le Seminaire de Probabilites, Lecture Notes, Graduate Texts in Mathematics 1709, 1–68 (1999)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Bramoullé, Y., Kranton, R., D’Amours, M.: Strategic interaction and networks. The American Economic Review 104(3), 898–930 (2014)CrossRefGoogle Scholar
  6. 6.
    Bürger, M., Zelazo, D., Allgöwer, F.: Duality and network theory in passivity-based cooperative control. Automatica 50(8), 2051–2061 (2014)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Candogan, O., Bimpikis, K., Ozdaglar, A.: Optimal pricing in networks with externalities. Operations Research 60(4), 883–905 (2012)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Facchinei, F., Pang, J.: Finite-Dimensional Variational Inequalities and Complementarity Problems. Springer New York (2007)zbMATHGoogle Scholar
  9. 9.
    Fox, M.J., Shamma, J.S.: Population Games, Stable Games, and Passivity. In: 51st IEEE Conf. on Decision and Control. pp. 7445–7450 (Dec 2012)Google Scholar
  10. 10.
    Frihauf, P., Krstic, M., Başar, T.: Nash Equilibrium Seeking in Noncooperative Games. IEEE Trans. on Automatic Control 57(5), 1192–1207 (2012)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Gadjov, D., Pavel, L.: A Passivity-Based Approach to Nash Equilibrium Seeking over Networks. IEEE Trans. on Automatic Control  https://doi.org/10.1109/TAC.2018.2833140
  12. 12.
    Gao, B., Pavel, L.: On Passivity and Reinforcement Learning in Finite Games. In: 51st IEEE Conf. on Decision and Control (Dec 2018)Google Scholar
  13. 13.
    Gharesifard, B., Cortes, J.: Distributed convergence to Nash equilibria in two-network zero-sum games. Automatica 49(6), 1683–1692 (2013)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Govindan, S., Reny, P.J., Robson, A.J.: A short proof of Harsanyi’s purification theorem. Games Econ. Behav. 45, 369–374 (2003)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Grammatico, S., Parise, F., Colombino, M., Lygeros, J.: Decentralized Convergence to Nash Equilibria in Constrained Deterministic Mean Field Control. IEEE Trans. on Automatic Control 61(11), 3315–3329 (2016)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Hines, G., Arcak, M., Packard, A.: Equilibrium-independent passivity: A new definition and numerical certification. Automatica 47(9), 1949–1956 (2011)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Hiriart-Urruty, J.B., Lemaréchal, C.: Fundamentals of Convex Analysis (2001)Google Scholar
  18. 18.
    Hofbauer, J., Sandholm, W.H.: Stable games and their dynamics. J. Econ. Theory 144(4), 1665–1693 (2009)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Khalil, H.: Nonlinear Systems. Prentice Hall (2002)Google Scholar
  20. 20.
    Koshal, J., Nedic, A., Shanbhag, U.V.: Distributed algorithms for aggregative games on graphs. Operations Research 64(3), 680–704 (2016)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Leslie, D., Collins, E.: Individual Q-Learning in Normal Form Games. SIAM J. Control and Optimiz. 44(2), 495–514 (2005)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Li, N., Marden, J.R.: Designing games for distributed optimization. IEEE Journal of Selected Topics in Signal Processing 7(2), 230–242 (2013)CrossRefGoogle Scholar
  23. 23.
    Marden, J.R., Shamma, J.S.: Game theory and distributed control. Handbook of Game Theory 4, 2818–2833 (2013)Google Scholar
  24. 24.
    Mertikopoulos, P., Sandholm, W.: Learning in Games via Reinforcement and Regularization. Mathematics of Operations Research 41(4), 1297–1324 (2016)MathSciNetCrossRefGoogle Scholar
  25. 25.
    P. Coucheney, B. Gaujal, Mertikopoulos, P.: Penalty-Regulated Dynamics and Robust Learning Procedures in Games. Mathematics of Operat. Research 40(3), 611–633 (2015)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Parise, F., Gentile, B., Grammatico, S., Lygeros, J.: Network aggregative games: Distributed convergence to Nash equilibria. In: Proc. of the 54th IEEE CDC. pp. 2295–2300 (2015)Google Scholar
  27. 27.
    Pavel, L.: Game theory for control of optical networks. Birkhäuser-Springer (2012)Google Scholar
  28. 28.
    Pavlov, A., Marconi, L.: Incremental passivity and output regulation. System & Control Letters 57, 400– 409 (2008)MathSciNetCrossRefGoogle Scholar
  29. 29.
    R. Cominetti, E. Melo, Sorin, S.: A payoff-based learning procedure and its application to traffic games. Games and Economic behaviour 70(1), 71–83 (2010)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Salehisadaghiani, F., Pavel, L.: Distributed Nash equilibrium seeking: A gossip-based algorithm. Automatica 72, 209–216 (2016)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Scutari, G., Facchinei, F., Pang, J.S., Palomar, D.P.: Real and Complex Monotone Communication Games. IEEE Trans. on Inform. Theory 60(7), 4197–4231 (2014)MathSciNetCrossRefGoogle Scholar
  32. 32.
    Shamma, J.S., Arslan, G.: Dynamic fictitious play, dynamic gradient play, and distributed convergence to Nash equilibria. IEEE Trans. on Automatic Control 50(3), 312–327 (2005)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Sorin, S.: Exponential weight algorithm in continuous time. Mathematical Programming 116(1), 513–528 (2009)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Swenson, B., Kar, S., Xavier, J.: Empirical centroid fictitious play: An approach for distributed learning in multi-agent games. IEEE Trans. on Signal Processing 63(15), 3888–3901 (2015)MathSciNetCrossRefGoogle Scholar
  35. 35.
    Wang, X., Xiao, N., Wongpiromsarn, T., Xie, L., Frazzoli, E., Rus, D.: Distributed consensus in noncooperative congestion games: an application to road pricing. In: IEEE 10th Int. Conf. on Control and Automation (ICCA). pp. 1668–1673 (2013)Google Scholar
  36. 36.
    Zhou, Z., Mertikopoulos, P., Moustakas, A.L., Bambos, N., Glynn, P.: Mirror descent learning in continuous games. In: 56th IEEE Conf. on Decision and Control. pp. 5776–5783 (Dec 2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.University of TorontoTorontoCanada

Personalised recommendations