Intelligent traffic signal optimization: UCB-DQN a promising approach for enhanced mobility and safety

Abstract

Traffic lights are vital for traffic management, particularly at intersections. Traditional fixed-time systems, widely used in real-world settings, struggle to adapt to dynamic traffic patterns, leading to increased waiting times, congestion, and inefficient road use. These issues undermine the overall effectiveness of transportation networks. To address these challenges, dynamic traffic light control systems have emerged, adjusting to real-time traffic conditions. Reinforcement Learning (RL) and Deep Reinforcement Learning (DRL) algorithms, which learn through direct interaction with the environment, have shown potential in optimizing traffic light timings by adapting to changing traffic flows. In this work, we propose a novel traffic light control method that combines the Upper Confidence Bound (UCB) strategy with the Deep Q-Network (DQN). The UCB guides the exploration phase of the DQN, helping it focus on useful strategies, improving performance, and reducing learning time. This guidance allows the DQN to avoid unproductive exploration. We evaluate our approach using the Simulation of Urban Mobility (SUMO) tool on a scenario involving a single intersection with various entities: regular vehicles, pedestrians, and emergency vehicles. Our results show that the proposed model outperforms both the traditional DQN and Double DQN (DDQN) in reducing total waiting time and speeding up the learning process. Specifically, our model achieves a lower total waiting time compared to traditional DQN across all training episodes, highlighting its effectiveness in real-time traffic control.