Traffic Congestion Prediction and Intelligent Signalling Based on Markov Decision Process and Reinforcement Learning

  • S. SuryaEmail author
  • N. Rakesh
Conference paper
Part of the Lecture Notes in Networks and Systems book series (LNNS, volume 14)


As the advancement of vehicular traffic, traffic congestion alleviation is desperately required in urban cities. A fixed duration traffic light in an intersection can make a few empty lanes and at the same time create other congested lanes. Dynamic scheduling of the signals is renowned as a solution for traffic congestion mitigation in urban areas. Static phase timing is not an optimal solution for reducing the congestion at the signals. So there is a pressing need of efficient algorithms for congestion prediction by considering historical and real time traffic data. The proposed work adopts to optimize a standard traffic a junction of two roads, one with North–South orientation and other with East–West orientation stop light dynamically with reinforcement learning and with markov decision process. It considers inflow and out flow of traffic at each lines and also waiting time of the vehicle for scheduling the signal timings. By using this method the overall waiting time of vehicles considerably reduced.


Traffic congestion Reinforcement learning Queue learning Markov decision process 


  1. 1.
    Atote BS, Bedekar M, Panicker SS (2015) Traffic signal control for urban area. Int J Eng SciGoogle Scholar
  2. 2.
    Lienert E. Simulation of genetic algorithm: traffic light efficiency. Senior research paperGoogle Scholar
  3. 3.
    Misbahuddin S, Zubairi JA, Saggaf A, Basuni J, Wadany SA, Al-Sofi A (2014) IoT based dynamic road traffic management for smart citiesGoogle Scholar
  4. 4.
    Sarath S, Chinnu R, Gopika PS (2016) Real time smart traffic control system using dynamic background. Int J Control Theor Appl 99:4249–4255Google Scholar
  5. 5.
    Zhao Y. Predicting traffic congestion with driving behavior. Big Data Developer & AnalystGoogle Scholar
  6. 6.
    Nayak RR, Sahana SK, Bagalkot AS, Soumya M, Roopa J, Govinda Raju M, Ramavenkateswaransmart N (2013) Traffic congestion control using wireless communication. Senior research paperGoogle Scholar
  7. 7.
    Glick J (2015) Reinforcement learning for adaptive traffic signal control. Final project, CS 229 (Machine Learning). Stanford University, 11 Dec 2015Google Scholar
  8. 8.
    Bacon J, Bejan AI, Beresford AR, Evans D, Gibbens RJ, Moody K (2011) Using real-time road traffic data to evaluate congestion. Springer, BerlinGoogle Scholar
  9. 9.
    Abidin AF, Kolberg M (2015) Towards improved vehicle arrival time prediction in public transportation: integrating SUMO and Kalman Filter models. In: 2015 17th UKSIMAMSS international conference on modelling and simulation Google Scholar
  10. 10.
    Behrisch M, Bieker L, Erdmann J, Krajzewicz D (2011) SUMO—simulation of urban mobility. In: The third international conference on advances in system simulationGoogle Scholar
  11. 11.
    Jain V, Sharma A, Dhananjay A, Subramanian L (2012) Traffic density estimation for noisy camera sources. In: TRB 91st annual meeting, Washington D.C., Jan 2012Google Scholar
  12. 12.
    Li F et al (2013) Efficient and scalable IoT service delivery on cloud. In: IEEE CLOUD, 2013 Google Scholar
  13. 13.
    Dan P (2013) Urban traffic congestion prediction based on routes information. In: 2013 IEEE 8th international symposium on applied computational intelligence and informatics (SACI)Google Scholar
  14. 14.
    Huaping Z, Xia H. The analysis of the traffic flow of the intersection based on the queuing theory. Sci Technol Inf Google Scholar
  15. 15.
    Chen T, Tan Z, Liu Y (2013) The improving method of the intersection traffic problems of the urban road. Traffic Eng 7:56–58Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.Department of CSEAmrita School of Engineering, Amrita Vishwa Vidyapeetham, Amrita UniversityBengaluruIndia

Personalised recommendations