Abstract
This paper considers optimization problems for a new kind of control systems based on non-equilibrium dynamic games. To be precise, the authors consider the infinitely repeated games between a human and a machine based on the generic 2×2 game with fixed machine strategy of finite k-step memory. By introducing and analyzing the state transfer graphes (STG), it will be shown that the system state will become periodic after finite steps under the optimal strategy that maximizes the human’s averaged payoff, which helps us to ease the task of finding the optimal strategy considerably. Moreover, the question whether the optimizer will win or lose is investigated and some interesting phenomena are found, e.g., for the standard Prisoner’s Dilemma game, the human will not lose to the machine while optimizing her own averaged payoff when k = 1; however, when k ≥ 2, she may indeed lose if she focuses on optimizing her own payoff only. The robustness of the optimal strategy and identification problem are also considered. It appears that both the framework and the results are beyond those in the classical control theory and the traditional game theory.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
K. J. Astrom and B. Wittenmark, Adaptive Control, 2nd ed., Addison-Wesley, Reading, MA, 1995.
L. Guo and H. Chen, The Astrom-Wittenmark self-tuning regulator revised and ELS-based adapptive trakers, IEEE Trans. on Automatic Control, 1991, 36: 802–812.
L. Guo and L. Ljung, Performance analysis of general tracking algorithms, IEEE Trans. on Automatic Control, 1995, 40: 1388–1402.
L. Guo, Self-convergence of weighted least-squares with applications to stochastic adaptive control, IEEE Trans. on Automatic Control, 1996, 41: 79–89.
T. L. Duncan, L. Guo, and B. Pasik-Duncan, Continous-time linear-quadratic Gaussian adaptive control, IEEE Trans. on Automatic Control, 1999, 44: 1653–1662.
G. C. Goodwin and K. S. Sin, Adaptive Filtering, Prediction and Control, Prentice-Hall, Englewood Cliffs NJ, 1984.
P. R. Kumar and P. Varaiya, Stochastic Systems: Estimation, Identification and Adaptive Control, Prentice Hall, Englewood Cliffs NJ, 1986.
M. Kristic, I. Kanellakopoulos, and P. Kokotoric, Nonlinear Adaptive Control Design, A Wiley-Interscience Publication, John Wiley & Sons, Inc., Canada, 1995.
L. Guo, Adaptive systems theory: some basic concepts, methods and results, Journal of Systems Science & Complexity, 2003, 16(2): 293–306.
J. Holland, Hidden Order: How Adaptation Builds Complexity, Addison-Wesley, Reading, MA: 1995.
J. Holland, Studying complex adaptive systems, Journal of System Science & Complexity, 2006, 19(1): 1–8.
T. Basar and G. J. Olsder, Dynamic Noncooperative Game Theory, the Society for Industrial Applied Mathematics, Academic Press, New York, 1999.
P. Y. Nie, L. Chen, and M. Fukushima, Dynamic programming approach to discrete time dynamic feedback Stackelberg games with independent and dependent followers, European Journal of Operational Research, 2006, 169: 310–328.
J. M. Smith, Evolution and the Theory of Games, Cambridge University Press, Cambridge, New York, 1982.
J. W. Weibull. Evolutionary Game Theory, MIT Press, Cambridge, MA, 1995.
J. Hofbauer and K. Sigmund, Evolutionary game dynamics, Bulletin of the American Mathematical Society, 2003, 40: 479–519.
S. R. Bulo and I. M. Bomze, Infection ans immunization: A new class of evolutionary game dynamics, Games and Economic Behavior, 2011, 71: 193–211.
D. Fudenberg and D. K. Levine, The Theory of Learning in Games, MIT Press, Cambridge, MA, 1998.
E. Kalai and E. Lehrer, Rational learning leads to Nash equilibrium, Econometria, 1993, 61: 1019–1045.
D. P. Foster and H. P. Young, Regret testing: A simple payoff-based procedure for learning Nash equilibrium, Theoretical Economics, 2006, 1: 341–367.
J. R. Marden, G. Arslan, and J. S. Shamma, Joint strategy fictitious play with inertia for potential games, IEEE Trans on Automatic Control, 2009, 54: 208–220.
H. P. Young, Learning by trial and error, Games and Economic Behavior, 2009, 65: 626–643.
R. Axelrod, The Evolution of Cooperation, Basic Books, New York, 1984.
L. Davis, Genetic Algorithms and Simulated Annealing, Morgan Kaufman Publishers, Inc., Los Altos, CA, 1987.
R. Axelrod, The Complexity of Cooperation: Agent-Based Models of Competition and Collaboration, Princeton University Press, Princeton, New Jersey, 1997.
A. Rubinstein, Finite automata play the repeated prisoner’s dilemma, Journal of Economic Theory, 1986, 39: 83–96.
G. Szabó and C. Töke, Evolutionary prisoner’s dilemma game on a square lattice, Physical Review E, 1998, 58: 69–73.
M. A. Nowak, Five rules for the evolution of cooperation, Science, 2006, 314: 1560–1563.
Y. Mu and L. Guo, Optimization and idntification in a non-equilibrium dynamic game, Proceedings of Joint 48th IEEE CDC and 28th CCC, Shanghai, 2009.
X. Hu, U. Jonsson, B. Wahlberg, and B. K. Ghosh, Three Decades of Progress in Control Sciences, Springer, Berlin, 2010.
M. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming, JohnWiley & Sons, Inc, New York, 1994.
J. B. Jensen and G. Gutin, Digraphs: Theory, Algorithms and Applications, Spring-Verlag, London, 2001.
D. B. Johnson, Finding all the elementary circuits of a directed graph, SIAM J. Comp, 1975, 4: 77–84.
Author information
Authors and Affiliations
Corresponding author
Additional information
This paper was supported by the National Natural Science Foundation of China under Grant No. 60821091 and by the Knowledge Innovation Project of Chinese Academy of Sciences under Grant No. KJCX3-SYW-S01.
This paper was recommended for publication by Editor Yiguang HONG.
Rights and permissions
About this article
Cite this article
Mu, Y., Guo, L. Towards a theory of game-based non-equilibrium control systems. J Syst Sci Complex 25, 209–226 (2012). https://doi.org/10.1007/s11424-012-1065-6
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11424-012-1065-6