Distributed Q-learning Controller for a Multi-Intersection Traffic Network

  • Sahar AraghiEmail author
  • Abbas Khosravi
  • Douglas Creighton
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9489)


This paper proposes a Q-learning based controller for a network of multi intersections. According to the increasing amount of traffic congestion in modern cities, using an efficient control system is demanding. The proposed controller designed to adjust the green time for traffic signals by the aim of reducing the vehicles’ travel delay time in a multi-intersection network. The designed system is a distributed traffic timing control model, applies individual controller for each intersection. Each controller adjusts its own intersection’s congestion while attempt to reduce the travel delay time in whole traffic network. The results of experiments indicate the satisfied efficiency of the developed distributed Q-learning controller.


Traffic Congestion Traffic Signal Green Time Reinforcement Learning Method Traffic Control System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Abdoos, M., Mozayani, N., Bazzan, A.: Traffic light control in non-stationary environments based on multi agent q-learning. In: International IEEE Conference on Intelligent Transportation Systems (ITSC), pp. 1580–1585, Washington, DC, USA (2011)Google Scholar
  2. 2.
    Abdulhai, B., Pringle, R., Karakoulas, G.: Reinforcement learning for true adaptive traffic signal control. J. Transp. Eng. 129(3), 278–285 (2003)CrossRefGoogle Scholar
  3. 3.
    Adler, J.L., Satapathy, G., Manikonda, V., Bowles, B., Blue, V.J.: A multi-agent approach to cooperative traffic management and route guidance. Transp. Res. B Methodol. 39(4), 297–318 (2005)CrossRefGoogle Scholar
  4. 4.
    Araghi, S., Khosravi, A., Creighton, D.: A review on computational intelligence methods for controlling traffic signal timing. Expert Syst. Appl. 42(3), 1538–1550 (2015)CrossRefGoogle Scholar
  5. 5.
    Araghi, S., Khosravi, A., Johnstone, M., Creighton, D.: Intelligent traffic light control of isolated intersections using machine learning methods, pp. 3621–3626 (2013)Google Scholar
  6. 6.
    Araghi, S., Khosravi, A., Johnstone, M., Creighton, D.: Q-learning method for controlling traffic signal phase time in a single intersection. In: IEEE Conference on Intelligent Transportation Systems, Proceedings, pp. 1261–1265 (2013)Google Scholar
  7. 7.
    Balaji, P., Sachdeva, G., Srinivasan, D., Tham, C.K.: Multi-agent system based urban traffic management. In: IEEE Congress on Evolutionary Computation, CEC 2007, pp. 1740–1747, IEEE (2007)Google Scholar
  8. 8.
    Cai, C., Wong, C.K., Heydecker, B.G.: Adaptive traffic signal control using approximate dynamic programming. Transp. Res. C Emerg. Technol. 17(5), 456–474 (2009)CrossRefGoogle Scholar
  9. 9.
    Chin, Y., Bolong, N., Kiring, A., Yang, S., Teo, K.: Q-learning based traffic optimization in management of signal timing plan. Int. J. Simul. Syst. Sci. Technol. 12(3), 29–35 (2011)Google Scholar
  10. 10.
    Doniec, A., Mandiau, R., Piechowiak, S., Espié, S.: A behavioral multi-agent model for road traffic simulation. Eng. Appl. Artif. Intell. 21(8), 1443–1454 (2008)CrossRefGoogle Scholar
  11. 11.
    Hirankitti, V., Krohkaew, J., Hogger, C.J.: A multi-agent approach for intelligent traffic-light control. In: World Congress on Engineering, vol. 1, Citeseer (2007)Google Scholar
  12. 12.
    Kosonen, I.: Multi-agent fuzzy signal control based on real-time simulation. Transp. Res. C Emerg. Technol. 11(5), 389–403 (2003)CrossRefGoogle Scholar
  13. 13.
    Kronborg, P., Davidsson, F.: MOVA and IHOVRA: traffic signal control for isolated intersections. Traffic Eng. Control 34(4), 195–200 (1993)Google Scholar
  14. 14.
    Nair, B., Cai, J.: A fuzzy logic controller for isolated signalized intersection with traffic abnormality considered. In: 2007 IEEE Intelligent Vehicles Symposium, pp. 1229–1233 (2007)Google Scholar
  15. 15.
    Prashanth, L., Bhatnagar, S.: Reinforcement learning with function approximation for traffic signal control. IEEE Trans. Intell. Transp. Syst. 12(2), 412–421 (2011)CrossRefGoogle Scholar
  16. 16.
    Spall, J.C., Chin, D.C.: Traffic-responsive signal timing for system-wide traffic control. Transp. Res. C Emerg. Technol. 5(3–4), 153–163 (1997)CrossRefGoogle Scholar
  17. 17.
    Sutton, R.S.: Generalization in reinforcement learning: successful examples using sparse coarse coding. In: Touretzky, D.S., Mozer, M.C., Hasselmo, M.E. (eds.) Advances in Neural Information Processing Systems, vol. 8, pp. 1038–1044. MIT Press, Cambridge (1996)Google Scholar
  18. 18.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning an Introduction. MIT Press, Cambridge (1998)Google Scholar
  19. 19.
    Teknomo, K.: Application of microscopic pedestrian simulation model. Transp. Res. F Traffic Psychol. Behav. 9(1), 15–27 (2006)CrossRefGoogle Scholar
  20. 20.
    Thorpe, T.L., Anderson, C.W.: Traffic light control using SARSA with three state representations. Technical report, IBM Corporation (1996)Google Scholar
  21. 21.
    Vincent, R., Peirce, J.: MOVA: traffic responsive, self-optimising signal control for isolated intersections. Technical report RR 170, Crowthorne: TRRL, Transport and Road Research Laboratory (1988)Google Scholar
  22. 22.
    Wiering, M., Vreeken, J., van Veenen, J., Koopman, A.: Simulation and optimization of traffic in a city. In: IEEE Intelligent Vehicles Symposium, pp. 453–458 (2004)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Sahar Araghi
    • 1
    Email author
  • Abbas Khosravi
    • 1
  • Douglas Creighton
    • 1
  1. 1.Center for Intelligent Systems Research (CISR)Deakin UniversityWaurn PondsAustralia

Personalised recommendations