Dynamic road network optimization has been used for improving traffic flow in an infrequent and localized manner. The development of intelligent systems and technology provides an opportunity to improve the frequency and scale of dynamic road network optimization. However, such improvements are hindered by the high computational complexity of the existing algorithms that generate the optimization plans. We present a novel solution that integrates machine learning and road network optimization. Our solution consists of two complementary parts. The first part is an efficient algorithm that uses reinforcement learning to find the best road network configurations at real-time. The second part is a dynamic routing mechanism, which helps connected vehicles adapt to the change of the road network. Our extensive experimental results demonstrate that the proposed solution can substantially reduce the average travel time in a variety of scenarios, whilst being computationally efficient and hence applicable to real-life situations.

Intelligent traffic management solutions that leverage machine learning have gained a lot of interest in recent years. These techniques, however, cannot be deployed in real-world settings at a desirable pace due to technological barriers. Thus, easily customizable, realistic simulation environments are needed to train and verify the effectiveness of machine learning algorithms for traffic control. We propose an easily extendable traffic simulation system named e-SMARTS to allow researchers to experiment with novel data-driven traffic management algorithms in a setup that mimics real-world traffic conditions. We demonstrate the flexibility of e-SMARTS using widely researched traffic management solutions for Autonomous Intersection Management (AIM). In the demonstration, we present several pluggable algorithms for AIM and show that these computationally efficient algorithms can achieve effective and safe results.

Graph combinatorial optimization (CO) is a widely studied problem with use-cases stemming from many fields. Typically, in real-world applications, the features of a graph tend to change over time (e.g. traffic congestion, or travel time), thus, finding a solution to the dynamic graph CO problem is critical. In recent years, using deep learning techniques to find heuristic solutions for NP-hard CO problems has gained much interest as these learned heuristics can find near-optimal solutions efficiently. However, most of the existing methods for learning heuristics focus on static CO problems. The dynamic nature makes NP-hard CO problems much more challenging to learn, and the existing methods fail to find reasonable solutions. We propose a novel architecture named Graph Temporal Attention with Reinforcement Learning (GTA-RL) to learn heuristic solutions for dynamic versions of graph CO problems. We then extend our architecture to learn heuristics for the real-time version of CO problems where all input features of a problem are not known a priori, but rather learned in real-time. A detailed experimental evaluation against several state-of-the-art learning-based algorithms and optimal solvers demonstrates the efficiency and effectiveness of our approach.

With increasing connectivity and autonomy in traffic eco-systems, Autonomous Intersection Management (AIM) has attracted strong attention from the research community. AIM helps optimize traffic by coordinating the trajectory of connected vehicles around intersections. Most of the existing AIM solutions are developed for single-objective optimization problems that are focused on improving traffic flow. A complete AIM solution needs to perform bi-objective optimization that considers both traffic flow and safety. However, the computational complexity for achieving both objectives is significantly high with the existing solutions, especially when traffic demand is stochastic. We address the limitations of the existing solutions using deep reinforcement learning (deep RL) that helps solve complex problems efficiently. Our solution uses two types of RL agents. The first type is intersection-level agents, which generate theoretically sound trajectory plans for individual vehicles approaching intersections. The second type is vehicle-level agents that control vehicles' actual trajectories around the intersections based on the plans. Both agents incorporate traffic flow and safety constraints into their decision making. Our experimental results show that our solution achieves a high safety level with a minimum impact on travel time.

Due to the importance of road freight, there is a significant cost of delaying freight vehicles on the road. In this work, we focus on freight vehicle optimization by reducing delays at intersections. Our simulation study evaluates the effectiveness of an autonomous intersection management strategy that prioritizes connected freight vehicles using intelligent traffic lights. We simulate a wide range of traffic scenarios on our microscopic traffic simulator. Our results show that the strategy can help reduce the delay of freight vehicles with a minimal impact on other vehicles in a real road network. Our simulations also reveal the scenarios where the strategy works best and where it should be avoided. Effects of individual parameters are also measured through simulations.

Autonomous intersection management has the ability to reduce congestion at intersections significantly, compared to classical traffic signal control in the era of connected autonomous vehicles. Autonomous intersection management requires time and speed adjustment for vehicles arriving at an intersection for collision-free passing through the intersection. Due to its computational complexity, this problem has been studied only when vehicle arrival times towards the vicinity of the intersection are known beforehand or with other simplifying scenarios which limits the applicability of these solutions for real-time settings. To solve the real-time autonomous traffic intersection management problem, we propose a reinforcement learning (RL) based multiagent architecture and a novel RL algorithm coined multi-discount Q-learning. In multi-discount Q-learning, we introduce a simple yet effective way to solve a Markov Decision Process by preserving both short-term and long-term goals, which is crucial for collision-free speed control. Our experimental results using microscopic simulations show that our RL-based multiagent solution can achieve near-optimal performance efficiently when minimizing the travel time through an intersection.