A Markov Decision Evolutionary Game for Individual Energy Management

  • Yezekael Hayel
  • Hamidou Tembine
  • Eitan Altman
  • Rachid El-Azouzi
Part of the Annals of the International Society of Dynamic Games book series (AISDG, volume 11)


We study in this paper a noncooperative game with an infinite number of players that are involved in many local interactions, each involving a randomly selected pair of players. Each player has to choose between an aggressive or a nonaggressive action. The expected lifetime of an individual as well as its expected total fitness during its lifetime (given as the total amount of packets it transmits during the lifetime) depend on the level of aggressiveness (power level) of all actions it takes during its life. The instantaneous reward of each player depends on the level of aggressiveness of his action as well as on that of his opponent. We model this as a Markov Decision Evolutionary Game which is an extension of the evolutionary game paradigm introduced in 1972 by Maynard Smith, and study the structure of equilibrium policies.


Time Slot Pure Strategy Markov Decision Process Mobile Terminal Global Optimum Solution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Altman, E., Hayel, Y.: A Stochastic Evolutionary Game of Energy Management in a Distributed Aloha Network. In: Proceedings of IEEE INFOCOM (2008)Google Scholar
  2. 2.
    Altman, E., Hayel, Y., Tembine, H., El Azouzi, R.: Markov Decision Evolutionary Games with Expected Average Fitness. In: Brown, J.S., Vincent, T.L.S. (eds.) Evolutionary Ecology Research Journal, Special issue in honour of Thomas Vincent (2009)Google Scholar
  3. 3.
    Altman, E., El Azouzi, R., Hayel, Y., Tembine, H.: The Evolution of Transport Protocols: An Evolutionary Game Perspective. Computer Networks 5310, 1751–1759 (2009)CrossRefGoogle Scholar
  4. 4.
    Axelrod, R.: The Evolution of Cooperation (Revised ed.). Basic Books, New York, NY (2006)Google Scholar
  5. 5.
    Haigh, J.: Game theory and evolution. Advances in Applied Probability 7(1), 8–11 (1975)MATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Hayel, Y., Tembine, H., Altman, E., El Azouzi, R.: A Markov Decision Evolutionary Game for Individual Energy Management. Technical Report, CERI (2009)Google Scholar
  7. 7.
    Hofbauer, J., Sigmund, K.: Evolutionary Games and Population Dynamics. Cambridge University Press, Cambridge, UK (1998)MATHGoogle Scholar
  8. 8.
    Hofbauer, J., Sigmund, K.: Evolutionary game dynamics. American Mathematical Society 40(4) (2003)Google Scholar
  9. 9.
    Houston, A.I., McNamara, J.M.: Evolutionarily stable strategies in the repeated hawk-dove game. Behavioral Ecology 2(3), 219–227 (1991)CrossRefGoogle Scholar
  10. 10.
    Maynard Smith, J.: Game Theory and the Evolution of Fighting. In: Maynard Smith, J. (ed) On Evolution, pp. 8–28. Edinburgh University Press, Edinburgh (1972)Google Scholar
  11. 11.
    Maynard Smith, J.: Evolution and the Theory of Games. Cambridge University Press, Cambridge, UK (1982)Google Scholar
  12. 12.
    McNamara, J.M., Merad, S., Collins, E.J.: The Hawk-Dove Game as an Average-Cost Problem. Adv. Appl. Probab. 23(4), 667–682 (1984)CrossRefMathSciNetGoogle Scholar
  13. 13.
    Puterman, M.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York, NY (1994)MATHGoogle Scholar
  14. 14.
    Sandholm, W.H.: Potential games with continuous player sets. Journal of Economic Theory 97(1), 81–108 (2001)MATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Sandholm, W.H.: Population Games and Evolutionary Dynamics. MIT, Cambridge, MA (to be published)Google Scholar
  16. 16.
    Susu, A., Acquaviva, A., Atienza, D., De Micheli, G.: Stochastic Modeling and Analysis for Environmentally Powered Wireless Sensor Nodes. Proceedings of WiOpt (2008)Google Scholar
  17. 17.
    Tembine, H., Altman, E., El Azouzi, R., Sandholm, W.H.: Evolutionary game dynamics with migration for hybrid power control in wireless communications. 47th IEEE Conference on Decision and Control, pp. 4479–4484, Cancun, Mexico (2008)Google Scholar
  18. 18.
    Tembine, H., Altman, e., El Azouzi, R., Hayel, Y.: Evolutionary Games in Wireless Networks. IEEE Transactions on Systems, Man, and Cybernetics: Part B, special issue on Game TheoryGoogle Scholar
  19. 19.
    Vincent, T.L., Vincent, T.L.S.: Evolution and control system design. IEEE Control Systems Magazine 20(5), 20–35 (2000)CrossRefGoogle Scholar
  20. 20.
    Weibull, J.W.: Evolutionary Game Theory. MIT, Cambridge, MA (1995)MATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Yezekael Hayel
    • 1
  • Hamidou Tembine
    • 1
  • Eitan Altman
    • 2
  • Rachid El-Azouzi
    • 1
  1. 1.Laboratoire Informatique d’Avignon/CERIUniversité d’AvignonAvignonFrance
  2. 2.INRIA Sophia Antipolis MéditerranéeSophia AntipolisFrance

Personalised recommendations