Discrete Event Dynamic Systems

, Volume 24, Issue 2, pp 103–132 | Cite as

A tutorial on event-based optimization—a new optimization framework



In many practical systems, the control or decision making is triggered by certain events. The performance optimization of such systems is generally different from the traditional optimization approaches, such as Markov decision processes or dynamic programming. The goal of this tutorial is to introduce, in an intuitive manner, a new optimization framework called event-based optimization. This framework has a wide applicability to aforementioned systems. With performance potential as building blocks, we develop two intuitive optimization algorithms to solve the event-based optimization problem. The optimization algorithms are proposed based on an intuitive principle, and theoretical justifications are given with a performance sensitivity based approach. Finally, we provide a few practical examples to demonstrate the effectiveness of the event-based optimization framework. We hope this framework may provide a new perspective to the optimization of the performance of event-triggered dynamic systems.


Event-based optimization Performance potential Discrete-event dynamic systems Sensitivity-based approach 


  1. Altman E (1999) Constrained Markov decision processes. Chapman & Hall/CRCGoogle Scholar
  2. Altman E, Avrachenkov KE, Nún̋ez-Queija R (2004) Perturbation analysis for denumerable Markov chains with application to queueing models. Adv. Appl. Probab. 36:839–853CrossRefMATHGoogle Scholar
  3. Bertsekas DP, Tsitsiklis JN (1996) Neuro-dynamic programming. Athena, Belmont, MAGoogle Scholar
  4. Barto A, Mahadevan S (2003) Recent advances in hierarchical reinforcement learning. In: Special issue on reinforcement learning. discrete event dynamic systems: theory and applications 13:41–77Google Scholar
  5. Cao XR (1994) Realization probabilities—the dynamics of queueing systems. Springer, New YorkGoogle Scholar
  6. Cao XR (2005) Basic ideas for event-based optimization of Markov systems. Discrete Event Dynamic Systems: Theory and Applications 15:169–197CrossRefMATHMathSciNetGoogle Scholar
  7. Cao XR (2007) Stochastic learning and optimization—a sensitivity-based approach. Springer, New YorkCrossRefMATHGoogle Scholar
  8. Cao XR, Chen HF (1997) Potentials, perturbation realization, and sensitivity analysis of Markov processes. IEEE Trans Autom Control 42:1382–1393CrossRefMATHMathSciNetGoogle Scholar
  9. Cao XR, Zhang JY (2008) Event-based optimization of Markov systems. IEEE Trans Autom Control 53:1076–1082CrossRefMathSciNetGoogle Scholar
  10. Cao XR, Ren ZY, Bhatnagar S, Fu M, Marcus S (2002) A time aggregation approach to Markov decision processes. Automatica 38:929–943CrossRefMATHMathSciNetGoogle Scholar
  11. Ho YC, Cao XR (1991) Perturbation analysis of discrete event systems. Kluwer Academic Publishers, NorwellCrossRefMATHGoogle Scholar
  12. Jia QS (2011) On solving event-based optimization with average reward over infinite stages. IEEE Trans Autom Control 56:2912–2917CrossRefGoogle Scholar
  13. Leahu H, Heidergott B, Hordijk A (2013) Perturbation analysis of waiting times in the G/G/1 queue. Discrete Event Dyn Syst Theory Appl 23:277–305CrossRefMATHMathSciNetGoogle Scholar
  14. Marbach P, Tsitsiklis JN (2001) Simulation-based optimization of Markov reward processes. IEEE Trans Autom Control 46:191–209CrossRefMATHMathSciNetGoogle Scholar
  15. Parr RE (1998) Hierarchical control and learning for Markov decision processes. Ph.D. Dissertation, University of California at BerkeleyGoogle Scholar
  16. Puterman ML (1994) Markov decision processes: discrete stochastic dynamic programming. WileyGoogle Scholar
  17. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge, MAGoogle Scholar
  18. Wang DX, Cao XR (2011) Event-based optimization for POMDP and its application in portfolio management. In: The Proceedings of the 18th IFAC World Congress, 28 Aug–2 Sept 2011. Milano, Italy, pp 3228–3233Google Scholar
  19. Whitt W (1978) Approximation of dynamic programs, I. Math Oper Res 3:231–243CrossRefMATHMathSciNetGoogle Scholar
  20. Whitt W (1979) Approximation of dynamic programs, II. Math Oper Res 4:179–185CrossRefMATHMathSciNetGoogle Scholar
  21. Xia L (2012) Optimal control of customer admission to an open Jackson network. In: The Proceedings of the 31st Chinese Control Conference, 25–27 July 2013. Hefei, China, pp 2362–2367Google Scholar
  22. Xia L (2013) Event-based optimization of admission control in open queueing networks. Discrete Event Dyn Syst Theory Appl. doi: 10.1007/s10626-013-0167-1 Google Scholar
  23. Xia L, Cao XR (2012) Performance optimization of queueing systems with perturbation realization. Eur J Oper Res 218:293–304CrossRefMATHMathSciNetGoogle Scholar
  24. Yao C, Cassandras CG (2012) Using infinitesimal perturbation analysis of stochastic flow models to recover performance sensitivity estimates of discrete event systems. Discrete Event Dyn Syst Theory Appl 22:197–219CrossRefMATHMathSciNetGoogle Scholar
  25. Zhao Y, Yan CB, Zhao Q, Huang N, Li J, Guan X (2010a) Efficient simulation method for general assembly systems with material handling based on aggregated event-scheduling. IEEE Trans Autom Sci Eng 7(4):762–775CrossRefGoogle Scholar
  26. Zhao Y, Zhao Q, Guan X (2010b) Stochastic optimal control for a class of manufacturing systems based on event-based optimization. In: Huang J, Liu K-Z, Ohta Y (eds) Theory and applications of complex systems and robust control. Tsinghua University Press, Beijing, China, pp 63–83Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Center for Intelligent and Networked Systems (CFINS), Department of Automation, TNListTsinghua UniversityBeijingChina
  2. 2.Department of Finance and the Key Laboratory of System Control and Information Processing of Ministry of Education, Department of AutomationShanghai Jiao Tong UniversityShanghaiChina
  3. 3.Institute of Advanced StudyHong Kong University of Science and TechnologyHong KongChina

Personalised recommendations