Educational Driving Through Intelligent Traffic Simulation

  • Bogdan Vajdea
  • Aurelia CiupeEmail author
  • Bogdan Orza
  • Serban Meza
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12149)


Modelling driving dynamics in experimental educational scenarios represents a key enhancement of a SMART city, where citizen-oriented politics promote traffic rules knowledge retention and awareness. We propose an instructional design of mapping simulations of real-world urban networks as use cases for practicing traffic rules. Traffic simulations have been implemented through Reinforcement Learning agents, using a modified Policy Proximal Optimization (PPO) strategy, demonstrating a good sample efficiency. The proposed objective function and the selected policy positive and negative rewards empower the car agent to reach a predefined destination, from a predefined start position, while adapting to the route line. Results validate the applicability of the proposed approach to educational simulations, within a generic gamified environment. The approach proposes a further extension towards adaption to complex lane design (e.g. traffic signs) and player’s in-game behavior.


Reinforcement learning Agent Autonomous navigation Traffic simulation Instructional design Educational simulations 


  1. 1.
    Ingook, J., Donghun, K., Donghun, L., Youngsung, S.,: An agent-based simulation modeling with deep reinforcement learning for smart traffic signal control. In: IEEE International Conference on Information and Communication Technology Convergence, South Korea (2018)Google Scholar
  2. 2.
    Li, N., Oyler, D.W., Zhang, M., Yildiz, Y., Kolmanovsky, I., Girard, A.R.: Game theoretic modeling of driver and vehicle interactions for verification and validation of autonomous vehicle control systems. IEEE Trans. Control Syst. Technol. 26(5), 1782–1797 (2017)CrossRefGoogle Scholar
  3. 3.
    Oyler, D.W., Yildiz, Y., Girard, A.R., Li, N.I., Kolmanovsky, I.V.: A game theoretical model of traffic with multiple interacting drivers for use in autonomous vehicle development. In: IEEE American Control Conference, pp. 1705–1710, Boston (2016)Google Scholar
  4. 4.
    Genders, W., Razavi, S.: Evaluating reinforcement learning state representations for adaptive traffic signal control. Proc. Comput. Sci. 130, 26–33 (2018)CrossRefGoogle Scholar
  5. 5.
    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. In: arXiv preprint arXiv:1707.06347, pp. 1–12 (2017)
  6. 6.
    Campos, C., Leitão, J., Coelho, A.: Integrated modeling of road environments for driving simulation. In: 10th International Conference on Computer Graphics Theory and Applications, pp. 70–80, Berlin (2015)Google Scholar
  7. 7.
    Fransman, A., Richter, B., Raath, S.: An interactive computer program for South African urban primary school children to learn about traffic signs and rules. Afr. Saf. Promot.: J. Inj. Violence Prevent. 16(1), 57–67 (2018)Google Scholar
  8. 8.
    Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V.: CARLA: an open urban driving simulator. In: arXiv preprint arXiv:1711.03938, pp. 1–16 (2016)
  9. 9.
    Paden, B., Cap, M., Yong, S., Yershov, D., Frazzoli, E.: A survey of motion planning and control techniques for self-driving urban vehicles. IEEE Trans. Intell. Veh. 1(1), 33–55 (2016)CrossRefGoogle Scholar
  10. 10.
    Jraidi, I., Chalfoun, P., Frasson, C.: Implicit strategies for intelligent tutoring systems. In: International Conference on Intelligent Tutoring Systems, pp. 1–10 (2012)Google Scholar
  11. 11.
    Isele, D., Rahimi, R., Cosgun, A., Subramanian, K., Fujimura, K.: Navigating occluded intersections with autonomous vehicles using deep reinforcement learning. In: IEEE International Conference on Robotics and Automation, pp. 2034–2039, Brisbane (2018)Google Scholar
  12. 12.
    Mirowski, P., et al.: Learning to navigate in cities without a map. In: arXiv:1804.00168 (2018)
  13. 13.
    Oyer, W., Yildiz, Y., Girard, A., Li, N., Kolmanovsky, I.: A game theoretical model of traffic with multiple interacting drivers for use in autonomous vehicle development. In: Codevilla, F., Miiller, M., López, A., Koltun, V., Dosovitskiy, A. (eds.) American Control Conference on End-to-end Driving via Conditional Imitation Learning, Boston (2016). In: IEEE International Conference on Robotics and Automation, pp. 1–9, Brisbane (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Bogdan Vajdea
    • 1
  • Aurelia Ciupe
    • 1
    Email author
  • Bogdan Orza
    • 1
  • Serban Meza
    • 1
  1. 1.Multimedia Systems and Applications LaboratoryTechnical University of Cluj-NapocaCluj-NapocaRomania

Personalised recommendations