Automation and Remote Control

, Volume 79, Issue 4, pp 701–712 | Cite as

Stackelberg Equilibrium in a Dynamic Stimulation Model with Complete Information

Intellectual Control Systems, Data Analysis


We consider a stimulation model with Markov dynamics and discounted optimality criteria in case of discrete time and infinite planning horizon. In this model, the regulator has an economic impact on the executor, choosing a stimulating function that depends on the system state and the actions of the executor, who employs positional control strategies. System dynamics, revenues of the regulator and costs of the executor depend on the system state and the executor’s actions. We show that finding an approximate solution of the (inverse) Stackelberg game reduces to solving the optimal control problem with criterion equal to the difference between the revenue of the regulator and the costs of the executor. Here the ε-optimal strategy of the regulator is to economically motivate the executor to follow this optimal control strategy.


two-level incentive model inverse Stackelberg game discounted optimality criterion Bellman equation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    von Stackelberg, H., Marktform und Gleichgewicht, Vienna: Springer, 1934.Google Scholar
  2. 2.
    Basar, T. and Olsder, G.J., Dynamic Noncooperative Game Theory, Philadelphia: SIAM, 1999.MATHGoogle Scholar
  3. 3.
    Dockner, E., Jørgensen, S., Van Long, N., and Sorger, G., Differential Games in Economics and Control Science, Cambridge: Cambridge Univ. Press, 2000.CrossRefMATHGoogle Scholar
  4. 4.
    Van Long, N., A Survey of Dynamic Games in Economics, Singapore: World Scientific, 2010.CrossRefMATHGoogle Scholar
  5. 5.
    Li, T. and Sethi, S.P., A Review of Dynamic Stackelberg Game Models, Discrete Cont. Dyn.–B, 2017, vol. 22, no. 1, pp. 125–159.MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Ho, Y.-C., Luh, P., and Muralidharan, R., Information Structure, Stackelberg Games, and Incentive Controllability, IEEE Trans. Automat. Control, 1981, vol. 26, no. 2, pp. 454–460.MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Olsder, G.J., Phenomena in Inverse Stackelberg Games. Part 1: Static Problems, J. Optim. Theory Appl., 2009, vol. 143, no. 3, pp. 589–600.MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Olsder, G.J., Phenomena in Inverse Stackelberg Games. Part 2: Dynamic Problems, J. Optim. Theory Appl., 2009, vol. 143, no. 3, pp. 601–618.MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Groot, N., De Schutter, B., and Hellendoorn, H., Reverse Stackelberg Games. Part I: Basic Framework, Control Applications (CCA), 2012 IEEE Int. Conf. on Control Applications, 2012, pp. 421–426.CrossRefGoogle Scholar
  10. 10.
    Groot, N., De Schutter, B., and Hellendoorn, H., Reverse Stackelberg Games. Part II: Results and Open Issues, Control Applications (CCA), 2012 IEEE Int. Conf. on Control Applications, 2012, pp. 427–432.CrossRefGoogle Scholar
  11. 11.
    Germeier, Yu.B., On Games of Two Players with a Fixed Sequence of Moves, Dokl. Akad. Nauk USSR, 1971, vol. 198, no. 5, pp. 1001–1004.MATHGoogle Scholar
  12. 12.
    Germeier, Yu.B., Igry s neprotivopolozhnymi interesami (Non-Antagonistic Games), Moscow: Nauka, 1976.Google Scholar
  13. 13.
    Kononenko, A.F., Game-Theory Analysis of a Two-level Hierarchical Control System, USSR Comput. Math. Math. Phys., 1974, vol. 14, no. 5, pp. 72–81.CrossRefMATHGoogle Scholar
  14. 14.
    Gorelov, M.A. and Kononenko, A.F., Dynamic Models of Conflicts. III. Hierarchical Games, Autom. Remote Control, 2015, vol. 76, no. 2, pp. 264–277.MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Shen, H. and Başar, T., Incentive-Based Pricing for Network Games with Complete and Incomplete Information, in Advances in Dynamic Game Theory: Numerical Methods, Algorithms, and Applications to Ecology and Economics, Jørgensen, S., Quincampoix, M., and Vincent, Th.L., Eds., Boston: Birkhäuser, 2007, pp. 431–458.CrossRefGoogle Scholar
  16. 16.
    Staňková, K., Olsder, G.J., and Bliemer, M.C.J., Comparison of Different Toll Policies in the Dynamic Second-best Optimal Toll Design Problem. Case Study on a Three-Link Network, Eur. J. Transp. Infrast. Res., 2009, vol. 4. no. 9, pp. 331–346.Google Scholar
  17. 17.
    Luh, P., Ho, Y., and Muralidharan, R., Load Adaptive Pricing: An Emerging Tool for Electric Utilities, IEEE Trans. Autom. Control, 1982, vol. 27, no. 2, pp. 320–329.CrossRefMATHGoogle Scholar
  18. 18.
    Burkov, V.N., Goubko, M., Korgin, N., and Novikov, D., Introduction to Theory of Control in Organizations, Boca Raton: CRC Press, 2015.CrossRefMATHGoogle Scholar
  19. 19.
    Novikov, D.A., Stimulirovanie v sotsial’no-ekonomicheskikh sistemakh (bazovye matematicheskie modeli) (Stimulation in Social-Economic Systems (Basic Mathematical Models)), Moscow: Inst. Probl. Upravlen. RAN, 1998.Google Scholar
  20. 20.
    Novikov, D.A. and Shokhina, T.E., Incentive Mechanisms in Dynamic Active Systems, Autom. Remote Control, 2003, vol. 64, no. 12, pp. 1912–1921.MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Sundaram, R.K., A First Course in Optimization Theory, Cambridge: Cambridge Univ. Press, 1996.CrossRefMATHGoogle Scholar
  22. 22.
    Papageorgiou, N.S. and Kyritsi-Yiallourou, S.Th., Handbook of Applied Analysis, Dordrecht: Springer, 2009.MATHGoogle Scholar
  23. 23.
    Hernández-Lerma, O. and Lasserre, J.B., Discrete-Time Markov Control Processes: Basic Optimality Criteria, New York: Springer, 1996.CrossRefMATHGoogle Scholar
  24. 24.
    Maitra, A., Discounted Dynamic Programming on Compact Metric Spaces, Sankhyā: Indian J. Statistics. Ser. A, 1968, vol. 30, no. 2, pp. 211–216.MathSciNetMATHGoogle Scholar
  25. 25.
    Schäl, M., Average Optimality in Dynamic Programming with General State Space, Math. Oper. Res., 1993, vol. 18, no. 1, pp. 163–172.MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Bertsekas, D. and Shreve, S., Stochastic Optimal Control: The Discrete Time Case, Belmont: Athena Scientific, 1996.MATHGoogle Scholar
  27. 27.
    Feinberg, E.A. and Lewis, M.E., Optimality Inequalities for Average Cost Markov Decision Processes and the Stochastic Cash Balance Problem, Math. Oper. Res., 2007, vol. 32, no. 4, pp. 769–783.MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    Cruz-Suárez, D., Montes-de-Oca, R., and Salem-Silva, F., Conditions for the Uniqueness of Optimal Policies of Discounted Markov Decision Processes, Math. Oper. Res., 2004, vol. 60, no. 3, pp. 415–436.MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Breton, M., Alj, A., and Haurie, A., Sequential Stackelberg Equilibria in Two-Person Games, J. Optim. Theor. Appl., 1998, vol. 59, no. 1, pp. 71–97.MathSciNetCrossRefMATHGoogle Scholar
  30. 30.
    Blackwell, D., Discounted Dynamic Programming, Ann. Math. Statist., 1965, vol. 36, no. 1, pp. 226–235.MathSciNetCrossRefMATHGoogle Scholar
  31. 31.
    Shreve, S.E. and Bertsekas, D.P., Universally Measurable Policies in Dynamic Programming, Math. Oper. Res., 1979, vol. 4, no. 1, pp. 15–30.MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    Morgan, J., Constrained Well-Posed Two-Level Optimization Problems, in Nonsmooth Optimization and Related Topics, Clarke, F.H., Dem’yanov, V.F., and Giannessi, F., Eds., Boston: Springer, 1989, pp. 307–325.CrossRefGoogle Scholar
  33. 33.
    Patrone, F., Well-Posedness for Nash Equilibria and Related Topics, in Recent Developments in Well-Posed Variational Problems, Lucchetti, R. and Revalski, J., Eds., Dordrecht: Springer, 1995, pp. 211–227.CrossRefGoogle Scholar
  34. 34.
    Montes-De-Oca, R. and Lemus-Rodríguez, E., When Are the Value Iteration Maximizers Close to an Optimal Stationary Policy of a Discounted Markov Decision Process? Closing the Gap between the Borel Space Theory and Actual Computations, WSEAS Trans. Math., 2010, vol. 9, no. 3, pp. 151–160.MathSciNetGoogle Scholar

Copyright information

© Pleiades Publishing, Ltd. 2018

Authors and Affiliations

  1. 1.Southern Federal UniversityRostov-on-DonRussia

Personalised recommendations