Adaptive Traffic Signal Control Methods Based on Deep Reinforcement Learning

Abstract

Smart cities are characterized by their use of intelligent transportation systems (ITS), which utilize advanced traffic signal control methods to achieve effective and efficient traffic operations. Recently, due to significant progress in artificial intelligence, research has focused on machine learning-based frameworks of adaptive traffic signal control (ATSC). In particular, deep reinforcement learning (DRL) can be formulated as a model-free technique and applied to optimal action selection problems. We propose a DRL-based ATSC method for two kinds of neural network models: deep neural networks (DNN) and convolutional neural networks (CNN). In the training processes, the microscopic simulator Vissim builds a virtual intersection that the agent uses to scan all possible observations and makes interactive decisions. For each testing scenario, five random experiments are generated. The average system total delay (ASTD) over the five experiments is compared for the two proposed neural network models and a fixed timing plan by way of the Webster delay formulas. Based on these preliminary tests, the DNN-model (CNN-model) signal control agent performed a lowest value of ASTD for the unsaturated (oversaturated) cases. Moreover, the CNN-model has better feature extraction capabilities than the DNN-model particularly for the oversaturated cases. We found that situations with specific traffic maneuvers, such as a spillback of a protected left-turn bay, are well learned by the proposed CNN-model, even the training scenario remains only unsaturated.