Abstract
A general class of no-regret learning algorithms, called Φ-no-regret learning algorithms is defined, which spans the spectrum from no-internal-regret learning to no-external-regret learning, and beyond. Φ describes the set of strategies to which the play of a learning algorithm is compared: a learning algorithm satisfies Φ-no-regret iff no regret is experienced for playing as the algorithm prescribes, rather than playing according to any of the transformations of the algorithm’s play prescribed by elements of Φ. Analogously, a class of game-theoretic equilibria, called Φ-equilibria, is defined, and it is shown that the empirical distribution of play of Φ-no-regret algorithms converges to the set of Φ-equilibria. Perhaps surprisingly, the strongest form of no-regret algorithms in this class are no-internal-regret algorithms. Thus, the tightest game-theoretic solution concept to which Φ-no-regret algorithms (provably) converge is correlated equilibrium. In particular, Nash equilibrium is not a necessary outcome of learning via any Φ-no-regret learning algorithms.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Aumann, R.: Subjectivity and correlation in randomized strategies. Journal of Mathematical Economics 1, 67–96 (1974)
Blackwell, D.: An analog of the minimax theorem for vector payoffs. Pacific Journal of Mathematics 6, 1–8 (1956)
Brown, G.: Iterative solutions of games by fictitious play. In: Koopmans, T. (ed.) Activity Analysis of Production and Allocation. Wiley, New York (1951)
Cournot, A.: Recherches sur les Principes Mathematics de la Theorie de la Richesse. Hachette (1838)
Foster, D., Vohra, R.: A randomization rule for selecting forecasts. Operations Research 41(4), 704–709 (1993)
Foster, D., Vohra, R.: Regret in the on-line decision problem. Games and Economic Behavior 21, 40–55 (1997)
Freund, Y., Schapire, R.: A decision-theoretic generalization of on-line learning and an application to boosting. In: Vitányi, P.M.B. (ed.) EuroCOLT 1995. LNCS, vol. 904, pp. 23–37. Springer, Heidelberg (1995)
Freund, Y., Schapire, R.: Game theory, on-line prediction, and boosting. In: Proceedings of the 9th Annual Conference on Computational Learning Theory, pp. 325–332. ACM Press, New York (1996)
Fudenberg, D., Levine, D.K.: Conditional universal consistency. Games and Economic Behavior (Forthcoming)
Fudenberg, D., Levine, D.K.: Universal consistency and cautious fictitious play. Journal of Economic Dyanmics and Control 19, 1065–1090 (1995)
Hannan, J.: Approximation to Bayes risk in repeated plays. In: Dresher, M., Tucker, A.W., Wolfe, P. (eds.) Contributions to the Theory of Games, vol. 3, pp. 97–139. Princeton University Press, Princeton (1957)
Hart, S., Mas Colell, A.: A simple adaptive procedure leading to correlated equilibrium. Econometrica 68, 1127–1150 (2000)
Hart, S., Mas Colell, A.: A general class of adaptive strategies. Economic Theory 98, 26–54 (2001)
Jafari, A.: On the Notion of Regret in Infinitely Repeated Games. Master’s Thesis, Brown University, Providence (May 2003)
Robinson, J.: An iterative method of solving a game. Annals of Mathematics 54, 298–301 (1951)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Greenwald, A., Jafari, A. (2003). A General Class of No-Regret Learning Algorithms and Game-Theoretic Equilibria. In: Schölkopf, B., Warmuth, M.K. (eds) Learning Theory and Kernel Machines. Lecture Notes in Computer Science(), vol 2777. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45167-9_2
Download citation
DOI: https://doi.org/10.1007/978-3-540-45167-9_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40720-1
Online ISBN: 978-3-540-45167-9
eBook Packages: Springer Book Archive