Stochastic flow models with delays, blocking and applications to multi-intersection traffic light control


We extend Stochastic Flow Models (SFMs), used for a large class of discrete event and hybrid systems, by including the delays which typically arise in flow movements, as well as blocking effects due to space constraints. We apply this framework to the multi-intersection traffic light control problem by including transit delays for vehicles moving from one intersection to the next and possible blocking between two intersections. Using Infinitesimal Perturbation Analysis (IPA) for this SFM with delays and possible blocking, we derive new on-line gradient estimates of several congestion cost metrics with respect to the controllable green and red cycle lengths. The IPA estimators are used to iteratively adjust light cycle lengths to improve performance and, in conjunction with a standard gradient-based algorithm, to obtain optimal values which adapt to changing traffic conditions. We introduce two new cost metrics to better capture congestions and show that the inclusion of delays and possible blocking in our analysis lead to improved performance relative to models that ignore delays and/or blocking effects.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16


  1. Anderson MP, Woessner WW, Hunt RJ (2015) Applied groundwater modeling: simulation of flow and advective transport. Academic Press, Cambridge

  2. Armony M, Israelit S, Mandelbaum A, Marmor YN, Tseytlin Y, Yom-Tov GB, et al. (2015) On patient flow in hospitals: a data-based queueing-science perspective. Stoch Syst 5(1):146–194

  3. Cassandras CG, Wardi Y, Melamed B, Sun G, Panayiotou CG (2002) Perturbation analysis for on-line control and optimization of stochastic fluid models. IEEE Trans Autom Control 47(8):1234–1248

  4. Cassandras CG, Lafortune S (2009) Introduction to discrete event systems. Springer, Berlin

  5. Cassandras CG, Wardi Y, Panayiotou CG, Yao C (2010) Perturbation analysis and optimization of stochastic hybrid systems. Eur J Control 6(6):642–664

  6. Chen R, Cassandras CG (2018) Stochastic flow models with delays and applications to multi-intersection traffic light control. Proc 2018 Intl Worksh Discret Event Syst 51 (7):39–44

  7. Fleck JL, Cassandras CG, Geng Y (2016) Adaptive quasi-dynamic traffic light control. IEEE Trans Control Syst Technol 24(3):830–842

  8. Fu MC, Howell WC (2003) Application of perturbation analysis to traffic light signal timing. Proc IEEE, Conf on Decision and Control, pp 4837–4840

  9. Geng Y, Cassandras CG (2012) Traffic light control using infinitesimal perturbation analysis. In: 2012 IEEE 51St annual conf. on decision and control (CDC). IEEE, pp 7001–7006

  10. Geng Y, Cassandras CG (2015) Multi-intersection traffic light control with blocking. Discret Event Dyn Syst 25(1-2):7–30

  11. Head L, Ciarallo F, Kaduwela DL (1996) A perturbation analysis approach to traffic signal optimization. INFORMS National Meeting

  12. Panayiotou CG, Howell WC, Fu MC (2005) Online traffic light control through gradient estimation usinf stochastic flow models. Proc IFAC World Congress

  13. Wardi Y, Adams R, Melamed B (2010) A unified approach to infinitesimal perturbation analysis in stochastic flow models: the single-stage case. IEEE Trans Autom Control 55(1):89–103

  14. Yao C, Cassandras CG (2011) Perturbation analysis of stochastic hybrid systems and applications to resource contention games. Front Electr Electron Eng China 6 (3):453–467

  15. Yin S, Ding SX, Abandan Sari AH, Hao H (2013) Data-driven monitoring for stochastic systems and its application on batch process. Intl J Syst Sci 44(7):1366–1376

Download references


This work was supported in part by NSF under grants ECCS-1509084, DMS-1664644, and CNS-1645681, by AFOSR under grant FA9550-19-1-0158, by ARPA-E’s NEXTCAR program under grant DE-AR0000796, and by the MathWorks.

Author information

Correspondence to Rui Chen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Applications-2020

Guest Editors: Francesco Basile, Jan Komenda, and Christoforos Hadjicostis

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Chen, R., Cassandras, C.G. Stochastic flow models with delays, blocking and applications to multi-intersection traffic light control. Discrete Event Dyn Syst 30, 125–153 (2020).

Download citation


  • Performance evaluation
  • Optimization
  • Discrete approaches for hybrid systems
  • Applications