Skip to main content
SpringerLink
Log in

Sensitivity Analysis and Evolutionary Games

  • Chapter
Constructive Computation in Stochastic Models with Applications

  • 1626 Accesses

Abstract

In this chapter, we present sensitivity analysis for performance measures of an irreducible perturbed Markov chain which is either discretetime or continuous-time. By using the UL- and LU-types of RG-factorizations, we can express the nth derivatives of the performance measures, including the stationary, transient and discounted cases. Furthermore, we apply the sensitivity analysis to study symmetric evolutionary games by perturbed birth death processes and asymmetric evolutionary games by perturbed QBD processes.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 169.00
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alos-Ferrer C. and I. Neustadt (2005). Best-response dynamics in a birth-death model of evolution in games. In an initial version

    Google Scholar 

  2. Altman E., K.E. Avrachenkov and R. Núñez-Queija (2004). Perturbation analysis for denumerable Markov chains with application to queueing models. Advances in Applied Probability 36: 839–853

    Article  MATH  MathSciNet  Google Scholar 

  3. Amir M. and S.K. Berninghaus (1996). Another approach to mutation and learning in games. Games and Economic Behavior 14: 19–43

    Article  MATH  MathSciNet  Google Scholar 

  4. Bao G., C.G. Cassandras and M.A. Zazanis (1996). First-and second-derivatives estimators for cyclic closed-queueing networks. IEEE Transaction on Automatic Control 41: 1106–1124

    Article  MATH  MathSciNet  Google Scholar 

  5. Bielecki T.R. and L. Stettner (1998). Ergodic control of singularly perturbed Markov process in discrete time with general state and compact action spaces. Applied Mathematics and Optimization 38: 261–281

    Article  MATH  MathSciNet  Google Scholar 

  6. Binmore, K., L. Samuelson and R. Vaughan (1995). Musical chairs: Modeling noisy evolution. Games and Economic Behavior 11:1–35

    Article  MATH  MathSciNet  Google Scholar 

  7. Björnerstedt, J. and J.W. Weibull (1996). Nash equilibrium and evolution by imitation. In The Rational Foundations of Economic Behavior, Arrow, K. J. et al. (eds), St. Martin’s Press: New York, 155–181

    Google Scholar 

  8. Blume, L.E. (2003). How noise matters. Games and Economic Behavior 44: 251–271

    Article  MATH  MathSciNet  Google Scholar 

  9. Cao X.R. (1994). Realization Probabilities: The Dynamics of Queuing Systems, Springer-Verlag: London

    Book  MATH  Google Scholar 

  10. Cao X.R. (1998). The Maclaurin series for performance functions of Markov chains. Advances in Applied Probability 30: 676–692

    Article  MATH  MathSciNet  Google Scholar 

  11. Cao X.R. (1998). The relations among potentials, perturbation analysis, and Markov decision processes. Discrete Event Dynamic Systems 8: 71–87

    Article  MATH  MathSciNet  Google Scholar 

  12. Cao X.R. (2003). From perturbation analysis to Markov decision processes and reinforcement learning. Discrete Event Dynamic Systems 13: 9–39

    Article  MATH  MathSciNet  Google Scholar 

  13. Cao X.R. (2003). Semi-Markov decision problems and performance sensitivity analysis. IEEE Transaction on Automatic Control 48: 758–769

    Article  Google Scholar 

  14. Cao X.R. (2007). Stochastic Learning and Optimization: A Sensitivity-Based Approach, Springer-Verlag: New York

    MATH  Google Scholar 

  15. Cao X.R. and H.F. Chen (1997). Perturbation realization, potentials, and sensitivity analysis of Markov processes. IEEE Transaction on Automatic Control 42: 1382–1393

    Article  MATH  MathSciNet  Google Scholar 

  16. Cao X.R. and Y.W. Wan (1998). Algorithms for sensitivity analysis of Markov systems through potentials and perturbation realization. IEEE Transaction on Control Systems Technology 6: 482–494

    Article  Google Scholar 

  17. Cao X.R., X.M. Yuan, and L. Qiu (1996). A single sample path-based performance sensitivity formula for Markov chains. IEEE Transaction on Automatic Control 41: 1814–1817

    Article  MATH  MathSciNet  Google Scholar 

  18. Cassandras C.G. (1993). Discrete-Event Systems: Modeling and Performance Analysis, Homewood, IL, Aksen Associates

    Google Scholar 

  19. Chang C.S. and F. Nelson (1993). Perturbation analysis of the M/M/1 queue in a Markovian environment via the matrix-geometric method. Stochastic Models 9: 233–246

    Article  MATH  MathSciNet  Google Scholar 

  20. Chen H.C. and Y. Chow (2008). Equilibrium selection in evolutionary games with imperfect monitoring. Journal of Applied Probability 45: 388–402

    Article  MATH  MathSciNet  Google Scholar 

  21. Corradi V. and R. Sarin (2000). Continuous approximations of stochastic evolutionary game dynamics. Journal of Economic Theory 94: 163–191

    Article  MATH  MathSciNet  Google Scholar 

  22. Cressman R. (1992). The Stability Concept of Evolutionary Game Theory. A Dynamic Approach, Springer-Verlag: Berlin

    MATH  Google Scholar 

  23. Delebecque F. (1983). A reduction process for perturbed Markov chains. SIAM Journal on Applied Mathematics 43: 325–350

    Article  MATH  MathSciNet  Google Scholar 

  24. Ellison, G. (1993). Learning, local interaction, and coordination. Econometrica 61: 1047–1071

    Article  MATH  MathSciNet  Google Scholar 

  25. Ellison, G. (2000). Basins of attraction, long run equilibria, and the speed of step-by-step evolution. Review of Economic Studies 67: 17–45

    Article  MATH  MathSciNet  Google Scholar 

  26. Fishman M.A. (2008). Asymmetric evolutionary games with non-linear pure strategy payoffs. Games and Economic Behavior 63: 77–90

    Article  MATH  MathSciNet  Google Scholar 

  27. Foster D. and P. Young (1990). Stochastic evolutionary game dynamics. Theoretical Population Biology 38: 219–232

    Article  MATH  MathSciNet  Google Scholar 

  28. Fudenberg D. and D.K. Levine (1998). The Theory of Learning in Games, MIT Press: Cambridge, MA

    MATH  Google Scholar 

  29. Fudenberg D. and L.A. Imhof (2006). Imitation processes with small mutations. Journal of Economic Theory 131: 251–262

    Article  MATH  MathSciNet  Google Scholar 

  30. Fudenberg D., M.A. Nowakb, C. Taylorb and L.A. Imhof (2006). Evolutionary game dynamics in finite populations with strong selection and weak mutation. Theoretical Population Biology 70: 352–363

    Article  MATH  Google Scholar 

  31. Gambin A., P. Krzyanowski and P. Pokarowski (2008). Aggregation algorithms for perturbed Markov chains with applications to networks modeling. SIAM Journal on Scientific Computing 31: 45–73

    Article  MATH  MathSciNet  Google Scholar 

  32. Glasserman P. (1991). Gradient Estimation Via Perturbation Analysis, Kluwer Academic Publishers

    Google Scholar 

  33. Gong W.B., J. Pan and C.G. Cassandras (1992). Obtaining derivatives of stationary queue length probabilities through the matrix-geometric solution method. Applied Mathematics Letters 5: 69–73

    Article  MATH  MathSciNet  Google Scholar 

  34. Grassmann W.K. (1996). Optimizing steady state Markov chains by state reduction. European Journal of Operational Research 89: 277–284

    Article  MATH  Google Scholar 

  35. Grassmann W.K. and X. Chen (1995). The use of derivatives for optimizing steady state queues. Journal of the Operational Research Society 46: 104–115

    MATH  Google Scholar 

  36. Harsanyi, J. and R. Selten (1988). A General Theory of Equilibrium Selection in Games, MIT Press: Cambridge, MA

    MATH  Google Scholar 

  37. Hart, S. (2002). Evolutionary dynamics and backward induction. Games and Economic Behavior 41: 227–264

    Article  MATH  MathSciNet  Google Scholar 

  38. Hart, S. and A. Mas-Colell (2000). A simple adaptive procedure leading to correlated equilibrium. Econometrica 68: 1127–1150

    Article  MATH  MathSciNet  Google Scholar 

  39. Hart, S. and A. Mas-Colell (2003). Uncoupled dynamics do not lead to Nash equilibrium. American Economic Review 93: 1830–1836

    Article  Google Scholar 

  40. He Q.M. (1995). Differentiability of matrices R and G in the matrix-analytic method. Stochastic Models 11: 123–132

    Article  MATH  Google Scholar 

  41. He Q.M. and M.F. Neuts (1997). On Episodic queues. SIAM Journal on Matrix Analysis and Applications 18: 223–248

    Article  MATH  MathSciNet  Google Scholar 

  42. Hines, W. (1987). Evolutionary stable strategies: a review of basic theory. Theoretical Population Biology 31: 195–272

    Article  MATH  MathSciNet  Google Scholar 

  43. Hipp C. (2006). Speedy convolution algorithms and Panjer recursions for phase-type distributions. Insurance: Mathematics & Economics 38: 176–188

    Article  MATH  MathSciNet  Google Scholar 

  44. Ho Y.C. and X.R. Cao (1991). Perturbation Analysis of Discrete-Event Dynamic Systems, Kluwer

    Google Scholar 

  45. Hofbauer, J. (2000). From Nash and Brown to Maynard Smith: Equilibria, dynamics, and ESS. Selection 1: 81–88

    Article  Google Scholar 

  46. Hofbauer, J. and W.H. Sandholm (2002). On the global convergence of stochastic fictitious play. Econometrica 70: 2265–2294

    Article  MATH  MathSciNet  Google Scholar 

  47. Hofbauer J. and K. Sigmund (1998). Evolutionary Games and Population Dynamics, Cambridge University Press

    Google Scholar 

  48. Hofbauer J. and K. Sigmund (2003). Evolutionary game dynamics. American Mathematical Society Bulletin (New Series) 40: 479–519

    Article  MATH  MathSciNet  Google Scholar 

  49. Hunter J.J. (1986). Stationary distributions of perturbed Markov chains. Linear Algebra and its Applications 82: 201–214

    Article  MATH  MathSciNet  Google Scholar 

  50. Hunter J.J. (2005). Stationary distributions and mean first passage times of perturbed Markov chains. Linear Algebra and its Applications 410: 217–243

    Article  MATH  MathSciNet  Google Scholar 

  51. Hunter J.J. (2006). Mixing times with applications to perturbed Markov chains. Linear Algebra and its Applications 417: 108–123

    Article  MATH  MathSciNet  Google Scholar 

  52. Kandori, M., G.J. Mailath and R. Rob (1993). Learning, mutation, and long run equilibria in games. Econometrica 61: 29–56

    Article  MATH  MathSciNet  Google Scholar 

  53. Kandori, M. and R. Rob (1995). Evolution of equilibria in the long run: A general theory and applications. Journal of Economic Theory 65: 383–414

    Article  MATH  MathSciNet  Google Scholar 

  54. Korolyuk V.S. and A.F. Turbin (1993). Mathematical Foundations of the State Lumping of Large System, Kluwer Academic Pulishers

    Google Scholar 

  55. Latouche G. (1988). Perturbation analysis of phase-type queue with weakly correlated arrivals. Advances in Applied Probability 20: 896–912

    Article  MATH  MathSciNet  Google Scholar 

  56. Li Q.L. and J. Cao (2004). Two types of RG-factorizations of quasi-birth-and-death processes and their applications to stochastic integral functionals. Stochastic Models 20: 299–340

    Article  MATH  MathSciNet  Google Scholar 

  57. Li Q.L. and L.M. Liu (2004). An algorithmic approach on sensitivity analysis of perturbed QBD processes. Queueing Systems 48: 365–397

    Article  MATH  MathSciNet  Google Scholar 

  58. Li Q.L. and Y.Q. Zhao (2002). The RG-factorizations in block-structured Markov renewal processes with applications. Technical Report 381, Laboratory for Research in Statistics and Probability, Carleton University and University of Ottawa, Canada, 1–40

    Google Scholar 

  59. Li Q.L. and Y.Q. Zhao (2004). The RG-factorization in block-structured Markov renewal processes with applications. In Observation, Theory and Modeling of Atmospheric Variability, X. Zhu (ed), World Scientific, 545–568

    Google Scholar 

  60. Maynard Smith J. (1979). Evolutionary game theory. In Vito Volterra Symposium on Mathematical Models in Biology, Rome, 73–81; or in Lecture Notes in Biomath., 39, Springer: Berlin-New York, 1980

    Google Scholar 

  61. Maynard Smith, J. (1982). Evolution and the Theory of Game, MIT Press: Cambridge, MA

    Google Scholar 

  62. Maynard Smith, J. and G.R. Price (1973). The logic of animal conflict. Nature 246: 15–18

    Article  Google Scholar 

  63. Nöldeke, G. and L. Samuelson (1993). An evolutionary analysis of backward and forward induction. Games and Economic Behavior 5: 425–454

    Article  MATH  MathSciNet  Google Scholar 

  64. Rhode, P. and M. Stegeman (1996). A comment on “Learning, mutation, and long-run equilibria in games. Econometrica 64: 443–449

    Article  MATH  MathSciNet  Google Scholar 

  65. Roberts G.O., J.S. Rosenthal and P.O. Schwartz (1998). Convergence properties of perturbed Markov chains. Journal of Applied Probability 35: 1–11

    Article  MATH  MathSciNet  Google Scholar 

  66. Robson A.J. (1990). Efficiency in evolutionary games: Darwin, Nash and the secret handshake. Journal of Theoretical Biology 144: 379–396

    Article  MathSciNet  Google Scholar 

  67. Robson A.J. and F. Vega-Redondo (1996). Efficient equilibrium selection in evolutionary games with random matching. Journal of Economic Theory 70: 65–92

    Article  MATH  MathSciNet  Google Scholar 

  68. Samuelson L. (1997). Evolutionary Games and Equilibrium Selection, MIT Press: Cambridge, MA

    MATH  Google Scholar 

  69. Schlag, K.H. (1998). Why imitate, and if so, how? A boundedly rational approach to multi-armed bandits. Journal of Economic Theory 78: 130–156

    Article  MATH  MathSciNet  Google Scholar 

  70. Solan E. and N. Vieille (2003). Perturbed Markov chains. Journal of Applied Probability 40: 107–122

    Article  MATH  MathSciNet  Google Scholar 

  71. Suri R. and M.A. Zazanis (1988). Perturbation analysis gives strongly consistent sensitivity estimates for the M/G/1 queue. Management Science 34: 39–64

    Article  MATH  MathSciNet  Google Scholar 

  72. Suri S. (2007). Computational evolutionary game theory. In Algorithmic game theory, Cambridge University Press, 717–736

    Google Scholar 

  73. Tadj L. and A. Touzene (2003). A QBD approach to evolutionary game theory. Applied Mathematical Modelling 27: 913–927

    Article  MATH  Google Scholar 

  74. Takada T. and J. Kigami (1991). The dynamical attainability of ESS in evolutionary games. Journal of Mathematical Biology 29: 513–529

    Article  MATH  MathSciNet  Google Scholar 

  75. Tanaka, Y. (2000). A finite population ESS and a long run equilibrium in a n players coordination game. Mathematical Social Sciences 39: 195–206

    Article  MATH  MathSciNet  Google Scholar 

  76. Taylor C., D. Fudenberg, A. Sasaki and M.A. Nowak (2004). Evolutionary game dynamics in finite populations. Bulletin of Mathematical Biology 66: 1621–1644

    Article  MathSciNet  Google Scholar 

  77. Taylor, P.D. and L. Jonker (1978). Evolutionarily stable strategies and game dynamics. Mathematical Biosciences 40: 145–156

    Article  MATH  MathSciNet  Google Scholar 

  78. Vega-Redondo, F. (1996). Evolution, Games and Economic Behaviour, Oxford University Press

    Google Scholar 

  79. Vincent T.L. (1985). Evolutionary games. Journal of Optimization Theory and Applications 46: 605–612

    Article  MATH  MathSciNet  Google Scholar 

  80. Young, H.P. (1993). The evolution of conventions. Econometrica 61: 57–84

    Article  MATH  MathSciNet  Google Scholar 

  81. Young, H.P. (2004). Strategic Learning and Its Limits, Oxford University Press, Oxford

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Industrial Engineering, Tsinghua University, Beijing, 100084, P.R. China

    Quan-Lin Li

Authors
  1. Quan-Lin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Tsinghua University Press, Beijing and Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Li, QL. (2010). Sensitivity Analysis and Evolutionary Games. In: Constructive Computation in Stochastic Models with Applications. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-11492-2_11

Download citation

Publish with us

Policies and ethics

Search

Navigation