Game-Theoretic Learning in Distributed Control

  • Jason R. MardenEmail author
  • Jeff S. Shamma
Reference work entry


In distributed architecture control problems, there is a collection of interconnected decision-making components that seek to realize desirable collective behaviors through local interactions and by processing local information. Applications range from autonomous vehicles to energy to transportation. One approach to control of such distributed architectures is to view the components as players in a game. In this approach, two design considerations are the components’ incentives and the rules that dictate how components react to the decisions of other components. In game-theoretic language, the incentives are defined through utility functions, and the reaction rules are online learning dynamics. This chapter presents an overview of this approach, covering basic concepts in game theory, special game classes, measures of distributed efficiency, utility design, and online learning rules, all with the interpretation of using game theory as a prescriptive paradigm for distributed control design.


Learning in games Evolutionary games Multiagent systems Distributed decision systems 


  1. Alos-Ferrer C, Netzer N (2010) The logit-response dynamics. Games Econ Behav 68:413–427Google Scholar
  2. Altman E, Bonneau N, Debbah M (2006) Correlated equilibrium in access control for wireless communications. In: 5th International Conference on NetworkingGoogle Scholar
  3. Babichenko Y (2012) Completely uncoupled dynamics and Nash equilibria. Games Econ Behav 76:1–14Google Scholar
  4. Blondel VD, Hendrickx JM, Olshevsky A, Tsitsiklis JN (2005a) Convergence in multiagent coordination, consensus, and flocking. In: IEEE Conference on Decision and Control, pp 2996–3000Google Scholar
  5. Blondel VD, Hendrickx JM, Olshevsky A, Tsitsiklis JN (2005b) Convergence in multiagent coordination, consensus, and flocking. In: Proceedings of the Joint 44th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC’05), SevilleGoogle Scholar
  6. Blume L (1993) The statistical mechanics of strategic interaction. Games Econ Behav 5:387–424Google Scholar
  7. Blume L (1997) Population games. In: Arthur B, Durlauf S, and Lane D (eds) The economy as an evolving complex system II. Addison-Wesley, Reading, pp 425–460Google Scholar
  8. Borowski H, Marden JR, Frew EW (2013) Fast convergence in semi-anonymous potential games. In: Proceedings of the IEEE Conference on Decision and Control, pp 2418–2423Google Scholar
  9. Borowski HP, Marden JR, Shamma JS (2014) Learning efficient correlated equilibria. In: Proceedings of the IEEE Conference on Decision and Control, pp 6836–6841Google Scholar
  10. Cortes J, Martinez S, Karatas T, Bullo F (2002) Coverage control for mobile sensing networks. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA’02), Washington, DC, pp 1327–1332Google Scholar
  11. Daskalakis C, Goldberg PW, Papadimitriou CH (2009) The complexity of computing a Nash equilibrium. SIAM J Comput 39(1):195–259Google Scholar
  12. Fudenberg D, Levine DK (1995) Consistency and cautious fictitious play. Games Econ Behav 19:1065–1089Google Scholar
  13. Fudenberg D, Levine DK (1998) The theory of learning in games. MIT Press, CambridgeGoogle Scholar
  14. Fudenberg D, Tirole J (1991) Game theory. MIT Press, CambridgeGoogle Scholar
  15. Gopalakrishnan R, Marden JR, Wierman A (2014) Potential games are necessary to ensure pure Nash equilibria in cost sharing games. Math Oper Res 39(4):1252–1296Google Scholar
  16. Hart S (2005) Adaptive heuristics. Econometrica 73(5):1401–1430Google Scholar
  17. Hart S, Mansour Y (2010) How long to equilibrium? The communication complexity of uncoupled equilibrium procedures. Games Econ Behav 69(1):107–126Google Scholar
  18. Hart S, Mas-Colell A (2000) A simple adaptive procedure leading to correlated equilibrium. Econometrica 68(5):1127–1150Google Scholar
  19. Hart S, Mas-Colell A (2003) Uncoupled dynamics do not lead to Nash equilibrium. Am Econ Rev 93(5):1830–1836Google Scholar
  20. Jadbabaie A, Lin J, Morse AS (2003) Coordination of groups of mobile autonomous agents using nearest neighbor rules. IEEE Trans Autom Control 48(6):988–1001Google Scholar
  21. Kearns MJ, Littman ML, Singh SP (2001) Graphical models for game theory. In: Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence, pp 253–260Google Scholar
  22. Lambert III TJ, Epelman MA, Smith RL (2005) A fictitious play approach to large-scale optimization. Oper Res 53(3):477–489Google Scholar
  23. Marden JR (2012) State based potential games. Automatica 48:3075–3088Google Scholar
  24. Marden JR (2015) Selecting efficient correlated equilibria through distributed learning. In: American Control Conference, pp 4048–4053Google Scholar
  25. Marden JR, Shamma JS (2012) Revisiting log-linear learning: asynchrony, completeness and a payoff-based implementation. Games Econ Behav 75(2):788–808Google Scholar
  26. Marden JR, Shamma JS (2015) Game theory and distributed control. In: Young HP, Zamir S (eds) Handbook of game theory with economic applications, vol 4. Elsevier Science, pp 861–899Google Scholar
  27. Marden JR, Wierman A (2013) Distributed welfare games. Oper Res 61:155–168Google Scholar
  28. Marden JR, Arslan G, Shamma JS (2009) Joint strategy fictitious play with inertia for potential games. IEEE Trans Autom Control 54:208–220Google Scholar
  29. Martinez S, Cortes J, Bullo F (2007) Motion coordination with distributed information. Control Syst Mag 27(4):75–88Google Scholar
  30. Monderer D, Shapley LS (1996) Fictitious play property for games with identical interests. J Econ Theory 68:258–265Google Scholar
  31. Montanari A, Saberi A (2009) Convergence to equilibrium in local interaction games. In: 50th Annual IEEE Symposium on Foundations of Computer ScienceGoogle Scholar
  32. Murphey RA (1999) Target-based weapon target assignment problems. In: Pardalos PM, Pitsoulis LS (eds) Nonlinear assignment problems: algorithms and applications. Kluwer Academic, AlexandraGoogle Scholar
  33. Nisan N, Roughgarden T, Tardos E, Vazirani VV (2007) Algorithmic game theory. Cambridge University Press, New YorkGoogle Scholar
  34. Olfati-Saber R (2006) Flocking for multi-agent dynamic systems: algorithms and theory. IEEE Trans Autom Control 51:401–420Google Scholar
  35. Olfati-Saber R, Murray RM (2003) Consensus problems in networks of agents with switching topology and time-delays. IEEE Trans Autom Control 49(9):1520–1533Google Scholar
  36. Olfati-Saber R, Fax JA, Murray RM (2007) Consensus and cooperation in networked multi-agent systems. Proc IEEE 95(1):215–233Google Scholar
  37. Roughgarden T (2005) Selfish routing and the price of anarchy. MIT Press, CambridgeGoogle Scholar
  38. Roughgarden T (2015) Intrinsic robustness of the price of anarchy. J ACM 62(5):32:1–32:42Google Scholar
  39. Shah D, Shin J (2010) Dynamics in congestion games. In: ACM SIGMETRICS, pp 107–118Google Scholar
  40. Shamma JS (2014) Learning in games. In: Baillieul J, Samad T (eds) Encyclopedia of Systems and Control. Springer, LondonGoogle Scholar
  41. Shoham Y, Powers R, Grenager T (2007) If multi-agent learning is the answer, what is the question? Artif Intell 171(7):365–377Google Scholar
  42. Touri B, Nedic A (2011) On ergodicity, infinite flow, and consensus in random models. IEEE Trans Autom Control 56(7):1593–1605Google Scholar
  43. Tsitsiklis JN (1987) Decentralized detection by a large number of sensors. Technical report. MIT, LIDSGoogle Scholar
  44. Tsitsiklis JN, Bertsekas DP, Athans M (1986) Distributed asynchronous deterministic and stochastic gradient optimization algorithms. IEEE Trans Autom Control 35(9):803–812MathSciNetCrossRefGoogle Scholar
  45. Vetta A, “Nash equilibria in competitive societies, with applications to facility location, traffic routing and auctions,” Proceedings of the 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002, pp 416–425Google Scholar
  46. Wang B, Han Z, Liu KJR (2009) Peer-to-peer file sharing game using correlated equilibrium. In: 43rd Annual Conference on Information Sciences and Systems, CISS 2009, pp 729–734Google Scholar
  47. Wolpert D, Tumor K (1999) An overview of collective intelligence. In: Bradshaw JM (ed) Handbook of Agent Technology. AAAI Press/MIT Press, Cambridge, USAGoogle Scholar
  48. Young HP (1998) Individual strategy and social structure. Princeton University Press, PrincetonGoogle Scholar
  49. Young HP (2004) Strategic learning and its limits. Oxford University Press, New YorkCrossRefGoogle Scholar
  50. Zhu M, Martínez S (2013) Distributed coverage games for energy-aware mobile sensor networks. SIAM J Control Optim 51(1):1–27MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Electrical and Computer EngineeringUniversity of CaliforniaSanta BarbaraUSA
  2. 2.Computer, Electrical and Mathematical Science and Engineering Division (CEMSE)King Abdullah University of Science and Technology (KAUST)ThuwalSaudi Arabia

Personalised recommendations