Average Reward Timed Games

  • Bo Thomas Adler
  • Luca de Alfaro
  • Marco Faella
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3829)


We consider real-time games where the goal consists, for each player, in maximizing the average reward he or she receives per time unit. We consider zero-sum rewards, so that a reward of +r to one player corresponds to a reward of –r to the other player. The games are played on discrete-time game structures which can be specified using a two-player version of timed automata whose locations are labeled by reward rates. Even though the rewards themselves are zero-sum, the games are not, due to the requirement that time must progress along a play of the game.

Since we focus on control applications, we define the value of the game to a player to be the maximal average reward per time unit that the player can ensure. We show that, in general, the values to players 1 and 2 do not sum to zero. We provide algorithms for computing the value of the game for either player; the algorithms are based on the relationship between the original, infinite-round game, and a derived game that is played for only finitely many rounds. As memoryless optimal strategies exist for both players in both games, we show that the problem of computing the value of the game is in NP∩coNP.


Average Reward Reward Rate Strongly Connect Component Game Structure Concurrency Theory 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [ABM04]
    Alur, R., Bernadsky, M., Madhusudan, P.: Optimal reachability for weighted timed games. In: Díaz, J., Karhumäki, J., Lepistö, A., Sannella, D. (eds.) ICALP 2004. LNCS, vol. 3142, pp. 122–133. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  2. [AD94]
    Alur, R., Dill, D.L.: A theory of timed automata. Theor. Comp. Sci. 126, 183–235 (1994)zbMATHCrossRefMathSciNetGoogle Scholar
  3. [AH97]
    Alur, R., Henzinger, T.A.: Modularity for timed and hybrid systems. In: Mazurkiewicz, A., Winkowski, J. (eds.) CONCUR 1997. LNCS, vol. 1243, pp. 74–88. Springer, Heidelberg (1997)Google Scholar
  4. [AMAS98]
    Asarin, E., Maler, O., Pnueli, A., Sifakis, J.: Controller synthesis for timed automata. In: Proc. IFAC Symposium on System Structure and Control, pp. 469–474. Elsevier, Amsterdam (1998)Google Scholar
  5. [BBL04]
    Bouyer, P., Brinksma, E., Larsen, K.G.: Staying alive as cheaply as possible. In: Alur, R., Pappas, G.J. (eds.) HSCC 2004. LNCS, vol. 2993, pp. 203–218. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  6. [BCFL04]
    Bouyer, P., Cassez, F., Fleury, E., Larsen, K.G.: Optimal strategies in priced timed game automata. In: Lodaya, K., Mahajan, M. (eds.) FSTTCS 2004. LNCS, vol. 3328, pp. 148–160. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  7. [Chu63]
    Church, A.: Logic, arithmetics, and automata. In: Proc. International Congress of Mathematicians 1962, pp. 23–35. Institut Mittag-Leffler (1963)Google Scholar
  8. [Con92]
    Condon, A.: The complexity of stochastic games. Information and Computation 96, 203–224 (1992)zbMATHCrossRefMathSciNetGoogle Scholar
  9. [dAFH+03]
    de Alfaro, L., Faella, M., Henzinger, T.A., Majumdar, R., Stoelinga, M.: The element of surprise in timed games. In: Amadio, R.M., Lugiez, D. (eds.) CONCUR 2003. LNCS, vol. 2761, pp. 144–158. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  10. [dAHS02]
    de Alfaro, L., Henzinger, T.A., Stoelinga, M.: Timed interfaces. In: Sangiovanni-Vincentelli, A.L., Sifakis, J. (eds.) EMSOFT 2002. LNCS, vol. 2491, pp. 108–122. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  11. [EJ91]
    Emerson, E.A., Jutla, C.S.: Tree automata, mu-calculus and determinacy (extended abstract). In: Proc. 32nd IEEE Symp. Found. of Comp. Sci, pp. 368–377. IEEE Computer Society Press, Los Alamitos (1991)CrossRefGoogle Scholar
  12. [EM79]
    Ehrenfeucht, A., Mycielski, J.: Positional strategies for mean payoff games. Int. Journal of Game Theory 8(2), 109–113 (1979)zbMATHCrossRefMathSciNetGoogle Scholar
  13. [HHM99]
    Henzinger, T.A., Horowitz, B., Majumdar, R.: Rectangular hybrid games. In: Baeten, J.C.M., Mauw, S. (eds.) CONCUR 1999. LNCS, vol. 1664, pp. 320–335. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  14. [Kar78]
    Karp, R.M.: A characterization of the minimum cycle mean in a digraph. Discrete Mathematics 23, 309–311 (1978)zbMATHMathSciNetGoogle Scholar
  15. [MPS95]
    Maler, O., Pnueli, A., Sifakis, J.: On the synthesis of discrete controllers for timed systems. In: Mayr, E.W., Puech, C. (eds.) STACS 1995. LNCS, vol. 900, pp. 229–242. Springer, Heidelberg (1995)Google Scholar
  16. [PR89]
    Pnueli, A., Rosner, R.: On the synthesis of a reactive module. In: Proceedings of the 16th Annual Symposium on Principles of Programming Languages, pp. 179–190. ACM Press, New York (1989)Google Scholar
  17. [RW89]
    Ramadge, P.J.G., Wonham, W.M.: The control of discrete event systems. IEEE Transactions on Control Theory 77, 81–98 (1989)Google Scholar
  18. [ZP96]
    Zwick, U., Paterson, M.: The complexity of mean payoff games on graphs. Theor. Comp. Sci. 158, 343–359 (1996)zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Bo Thomas Adler
    • 1
  • Luca de Alfaro
    • 1
  • Marco Faella
    • 1
    • 2
  1. 1.School of EngineeringUniversity of CaliforniaSanta CruzUSA
  2. 2.Dipartimento di Scienze FisicheUniversità di Napoli “Federico II”Italy

Personalised recommendations