Automation and Remote Control

, Volume 79, Issue 4, pp 701–712 | Cite as

Stackelberg Equilibrium in a Dynamic Stimulation Model with Complete Information

  • D. B. Rokhlin
  • G. A. Ougolnitsky
Intellectual Control Systems, Data Analysis


We consider a stimulation model with Markov dynamics and discounted optimality criteria in case of discrete time and infinite planning horizon. In this model, the regulator has an economic impact on the executor, choosing a stimulating function that depends on the system state and the actions of the executor, who employs positional control strategies. System dynamics, revenues of the regulator and costs of the executor depend on the system state and the executor’s actions. We show that finding an approximate solution of the (inverse) Stackelberg game reduces to solving the optimal control problem with criterion equal to the difference between the revenue of the regulator and the costs of the executor. Here the ε-optimal strategy of the regulator is to economically motivate the executor to follow this optimal control strategy.


two-level incentive model inverse Stackelberg game discounted optimality criterion Bellman equation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    von Stackelberg, H., Marktform und Gleichgewicht, Vienna: Springer, 1934.Google Scholar
  2. 2.
    Basar, T. and Olsder, G.J., Dynamic Noncooperative Game Theory, Philadelphia: SIAM, 1999.zbMATHGoogle Scholar
  3. 3.
    Dockner, E., Jørgensen, S., Van Long, N., and Sorger, G., Differential Games in Economics and Control Science, Cambridge: Cambridge Univ. Press, 2000.CrossRefzbMATHGoogle Scholar
  4. 4.
    Van Long, N., A Survey of Dynamic Games in Economics, Singapore: World Scientific, 2010.CrossRefzbMATHGoogle Scholar
  5. 5.
    Li, T. and Sethi, S.P., A Review of Dynamic Stackelberg Game Models, Discrete Cont. Dyn.–B, 2017, vol. 22, no. 1, pp. 125–159.MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Ho, Y.-C., Luh, P., and Muralidharan, R., Information Structure, Stackelberg Games, and Incentive Controllability, IEEE Trans. Automat. Control, 1981, vol. 26, no. 2, pp. 454–460.MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Olsder, G.J., Phenomena in Inverse Stackelberg Games. Part 1: Static Problems, J. Optim. Theory Appl., 2009, vol. 143, no. 3, pp. 589–600.MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Olsder, G.J., Phenomena in Inverse Stackelberg Games. Part 2: Dynamic Problems, J. Optim. Theory Appl., 2009, vol. 143, no. 3, pp. 601–618.MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Groot, N., De Schutter, B., and Hellendoorn, H., Reverse Stackelberg Games. Part I: Basic Framework, Control Applications (CCA), 2012 IEEE Int. Conf. on Control Applications, 2012, pp. 421–426.CrossRefGoogle Scholar
  10. 10.
    Groot, N., De Schutter, B., and Hellendoorn, H., Reverse Stackelberg Games. Part II: Results and Open Issues, Control Applications (CCA), 2012 IEEE Int. Conf. on Control Applications, 2012, pp. 427–432.CrossRefGoogle Scholar
  11. 11.
    Germeier, Yu.B., On Games of Two Players with a Fixed Sequence of Moves, Dokl. Akad. Nauk USSR, 1971, vol. 198, no. 5, pp. 1001–1004.zbMATHGoogle Scholar
  12. 12.
    Germeier, Yu.B., Igry s neprotivopolozhnymi interesami (Non-Antagonistic Games), Moscow: Nauka, 1976.Google Scholar
  13. 13.
    Kononenko, A.F., Game-Theory Analysis of a Two-level Hierarchical Control System, USSR Comput. Math. Math. Phys., 1974, vol. 14, no. 5, pp. 72–81.CrossRefzbMATHGoogle Scholar
  14. 14.
    Gorelov, M.A. and Kononenko, A.F., Dynamic Models of Conflicts. III. Hierarchical Games, Autom. Remote Control, 2015, vol. 76, no. 2, pp. 264–277.MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Shen, H. and Başar, T., Incentive-Based Pricing for Network Games with Complete and Incomplete Information, in Advances in Dynamic Game Theory: Numerical Methods, Algorithms, and Applications to Ecology and Economics, Jørgensen, S., Quincampoix, M., and Vincent, Th.L., Eds., Boston: Birkhäuser, 2007, pp. 431–458.CrossRefGoogle Scholar
  16. 16.
    Staňková, K., Olsder, G.J., and Bliemer, M.C.J., Comparison of Different Toll Policies in the Dynamic Second-best Optimal Toll Design Problem. Case Study on a Three-Link Network, Eur. J. Transp. Infrast. Res., 2009, vol. 4. no. 9, pp. 331–346.Google Scholar
  17. 17.
    Luh, P., Ho, Y., and Muralidharan, R., Load Adaptive Pricing: An Emerging Tool for Electric Utilities, IEEE Trans. Autom. Control, 1982, vol. 27, no. 2, pp. 320–329.CrossRefzbMATHGoogle Scholar
  18. 18.
    Burkov, V.N., Goubko, M., Korgin, N., and Novikov, D., Introduction to Theory of Control in Organizations, Boca Raton: CRC Press, 2015.CrossRefzbMATHGoogle Scholar
  19. 19.
    Novikov, D.A., Stimulirovanie v sotsial’no-ekonomicheskikh sistemakh (bazovye matematicheskie modeli) (Stimulation in Social-Economic Systems (Basic Mathematical Models)), Moscow: Inst. Probl. Upravlen. RAN, 1998.Google Scholar
  20. 20.
    Novikov, D.A. and Shokhina, T.E., Incentive Mechanisms in Dynamic Active Systems, Autom. Remote Control, 2003, vol. 64, no. 12, pp. 1912–1921.MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Sundaram, R.K., A First Course in Optimization Theory, Cambridge: Cambridge Univ. Press, 1996.CrossRefzbMATHGoogle Scholar
  22. 22.
    Papageorgiou, N.S. and Kyritsi-Yiallourou, S.Th., Handbook of Applied Analysis, Dordrecht: Springer, 2009.zbMATHGoogle Scholar
  23. 23.
    Hernández-Lerma, O. and Lasserre, J.B., Discrete-Time Markov Control Processes: Basic Optimality Criteria, New York: Springer, 1996.CrossRefzbMATHGoogle Scholar
  24. 24.
    Maitra, A., Discounted Dynamic Programming on Compact Metric Spaces, Sankhyā: Indian J. Statistics. Ser. A, 1968, vol. 30, no. 2, pp. 211–216.MathSciNetzbMATHGoogle Scholar
  25. 25.
    Schäl, M., Average Optimality in Dynamic Programming with General State Space, Math. Oper. Res., 1993, vol. 18, no. 1, pp. 163–172.MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Bertsekas, D. and Shreve, S., Stochastic Optimal Control: The Discrete Time Case, Belmont: Athena Scientific, 1996.zbMATHGoogle Scholar
  27. 27.
    Feinberg, E.A. and Lewis, M.E., Optimality Inequalities for Average Cost Markov Decision Processes and the Stochastic Cash Balance Problem, Math. Oper. Res., 2007, vol. 32, no. 4, pp. 769–783.MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Cruz-Suárez, D., Montes-de-Oca, R., and Salem-Silva, F., Conditions for the Uniqueness of Optimal Policies of Discounted Markov Decision Processes, Math. Oper. Res., 2004, vol. 60, no. 3, pp. 415–436.MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Breton, M., Alj, A., and Haurie, A., Sequential Stackelberg Equilibria in Two-Person Games, J. Optim. Theor. Appl., 1998, vol. 59, no. 1, pp. 71–97.MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Blackwell, D., Discounted Dynamic Programming, Ann. Math. Statist., 1965, vol. 36, no. 1, pp. 226–235.MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Shreve, S.E. and Bertsekas, D.P., Universally Measurable Policies in Dynamic Programming, Math. Oper. Res., 1979, vol. 4, no. 1, pp. 15–30.MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Morgan, J., Constrained Well-Posed Two-Level Optimization Problems, in Nonsmooth Optimization and Related Topics, Clarke, F.H., Dem’yanov, V.F., and Giannessi, F., Eds., Boston: Springer, 1989, pp. 307–325.CrossRefGoogle Scholar
  33. 33.
    Patrone, F., Well-Posedness for Nash Equilibria and Related Topics, in Recent Developments in Well-Posed Variational Problems, Lucchetti, R. and Revalski, J., Eds., Dordrecht: Springer, 1995, pp. 211–227.CrossRefGoogle Scholar
  34. 34.
    Montes-De-Oca, R. and Lemus-Rodríguez, E., When Are the Value Iteration Maximizers Close to an Optimal Stationary Policy of a Discounted Markov Decision Process? Closing the Gap between the Borel Space Theory and Actual Computations, WSEAS Trans. Math., 2010, vol. 9, no. 3, pp. 151–160.MathSciNetGoogle Scholar

Copyright information

© Pleiades Publishing, Ltd. 2018

Authors and Affiliations

  1. 1.Southern Federal UniversityRostov-on-DonRussia

Personalised recommendations