Ramp metering or navigation guidance plays a vital role in alleviating congestion. However, dynamically integrating ramp metering and navigation guidance to achieve a “1+1>2” effect remains a challenge. To fill the technological gap, this paper proposes a reinforcement learning-based dynamic cooperative ramp metering and navigation guidance (DCRMNG) approach to conduct cooperative traffic control on the expressway system, which simultaneously adjusts demand and supply. Traffic congestion estimation (TCE) and route generation (RG) models are established for future traffic conditions prediction, traffic congestion evaluation, and guiding route generation. To formulate coordinative control strategies for two distinct groups of agents, a heterogeneous-agent Markov decision process (HMDP) is developed. The reward function is carefully crafted to promote collaboration and accelerate the optimization algorithms convergence. Then, a novel heterogeneous-agent proximal policy optimization (HAPPO) algorithm is introduced to solve the DCRMNG approach. Finally, a real-world scenario with an expressway and its parallel arterial road in Hangzhou, China is simulated to assess the performance of the HAPPO-based DCRMNG approach. The results reveal that the proposed approach has the capability to improve the overall network traffic efficiency, mitigate the “navigation jam” issue and achieve intelligent and precise control of the expressway system, exhibiting outstanding performance in various traffic scenarios.

Addressing traffic congestion is significant in enhancing urban mobility. Traditional navigation systems use real-time traffic status to push the temporally shortest path to drivers, forming selfish-routing, which guide traffic flow to the same roadway, thereby causing imbalance traffic flow distribution and navigation-induced congestion. Besides, navigation based on real-time detected traffic metrics may cause the congestion oscillation during periods with high travel demand variations. This paper proposes the Traffic Prediction and Reinforcement Learning-based Navigation Guidance (TP-RLNG) approach for active control of traffic flow on urban expressways. The TP-RLNG introduces a differentiated guidance approach with multiple origin-destination (OD) pairs, replacing the conventional all-or-nothing route guidance strategies. To enhance the stability, the TP-RLNG integrates traffic prediction and reinforcement learning (RL)-based dynamic optimization to prospectively harmonize traffic supply and demand overall the expressway system. In contrast to the reinforcement learning navigation guidance (RLNG) (no prediction) approach, the integration of traffic prediction model in TP-RLNG enhances the anticipation of critical nodes and enables proactive traffic management. We use SUMO-based traffic simulation to examine the efficacy of TP-RLNG and alternative approaches in controlling traffic flow within the Hangzhou Liushi Expressway network under varying demand patterns. The results underscore the ability of TP-RLNG to enhance road network efficiency and mitigate urban expressway congestion. The findings indicate that, relative to RLNG approaches, our methodology achieves a reduction in average travel time of 9.3%.

Segment routing (SR) is an emerging architecture that can benefit traffic engineering (TE). Nowadays, TE in SR networks (SR-TE) is often solved as an optimization problem to optimize network performance such as link utilization. As network size grows rapidly, implementing SR-TE suffers from scalability issues, including long computation time, high control overhead and expensive deployment cost. In this paper, we propose Smart Node Selection (SNS), a scalable SR-TE method with learning-based node selection (NS). NS is a recently proposed technique for reducing computation time of SR-TE. It first selects a subset of nodes as candidate intermediate nodes to route traffic, then builds linear programming (LP) models that can be solved efficiently. However, existing NS methods use simple heuristics and consider only network topology, which may lead to unsatisfying network performance. To address this problem, we for the first time formulates NS as a reinforcement learning task, which learns a selection policy to achieve better trade-offs between TE performance and computation time, considering both topology and traffic. Besides, we extend NS with additional selection policies and a customized training algorithm, making it a unified framework for scalable SR-TE, which reduces not only computation time, but also control overhead and deployment cost. Performance evaluations on various real-world topologies and traffic matrices show that SNS significantly reduces computation time and control overhead of existing LP models while offering good network performance, and can also be used in partially deployed SR networks to reduce deployment cost.

Irregularly Sampled Medical Time Series (ISMTS) are commonly found in the healthcare domain, where different variables exhibit unique temporal patterns while interrelated. However, many existing methods fail to efficiently consider the differences and correlations among medical variables together, leading to inadequate capture of fine-grained features at the variable level in ISMTS. We propose Knowledge-Empowered Dynamic Graph Network (KEDGN), a graph neural network empowered by variables' textual medical knowledge, aiming to model variable-specific temporal dependencies and inter-variable dependencies in ISMTS. Specifically, we leverage a pre-trained language model to extract semantic representations for each variable from their textual descriptions of medical properties, forming an overall semantic view among variables from a medical perspective. Based on this, we allocate variable-specific parameter spaces to capture variable-specific temporal patterns and generate a complete variable graph to measure medical correlations among variables. Additionally, we employ a density-aware mechanism to dynamically adjust the variable graph at different timestamps, adapting to the time-varying correlations among variables in ISMTS. The variable-specific parameter spaces and dynamic graphs are injected into the graph convolutional recurrent network to capture intra-variable and inter-variable dependencies in ISMTS together. Experiment results on four healthcare datasets demonstrate that KEDGN significantly outperforms existing methods.

Semi-supervised learning (SSL) has been actively studied due to its ability to alleviate the reliance of deep learning models on labeled data. Although existing SSL methods based on pseudo-labeling strategies have made great progress, they rarely consider time-series data's intrinsic properties (e.g., temporal dependence). Learning representations by mining the inherent properties of time series has recently gained much attention. Nonetheless, how to utilize feature representations to design SSL paradigms for time series has not been explored. To this end, we propose a Time Series SSL framework via Temporal-Frequency Co-training (TS-TFC), leveraging the complementary information from two distinct views for unlabeled data learning. In particular, TS-TFC employs time-domain and frequency-domain views to train two deep neural networks simultaneously, and each view's pseudo-labels generated by label propagation in the representation space are adopted to guide the training of the other view's classifier. To enhance the discriminative of representations between categories, we propose a temporal-frequency supervised contrastive learning module, which integrates the learning difficulty of categories to improve the quality of pseudo-labels. Through co-training the pseudo-labels obtained from temporal-frequency representations, the complementary information in the two distinct views is exploited to enable the model to better learn the distribution of categories. Extensive experiments on 106 UCR datasets show that TS-TFC outperforms state-of-the-art methods, demonstrating the effectiveness and robustness of our proposed model.

Noisy labels seriously degrade the generalization ability of Deep Neural Networks (DNNs) in various classification tasks. Existing studies on label-noise learning mainly focus on computer vision, while time series also suffer from the same issue. Directly applying the methods from computer vision to time series may reduce the temporal dependency due to different data characteristics. How to make use of the properties of time series to enable DNNs to learn robust representations in the presence of noisy labels has not been fully explored. To this end, this paper proposes a method that expands the distribution of Confident instances by Time-Warping (CTW) to learn robust representations of time series. Specifically, since applying the augmentation method to all data may introduce extra mislabeled data, we select confident instances to implement Time-Warping. In addition, we normalize the distribution of the training loss of each class to eliminate the model's selection preference for instances of different classes, alleviating the class imbalance caused by sample selection. Extensive experimental results show that CTW achieves state-of-the-art performance on the UCR datasets when dealing with different types of noise. Besides, the t-SNE visualization of our method verifies that augmenting confident data improves the generalization ability. Our code is available at https://github.com/qianlima-lab/CTW.

Large-scale Deterministic Network (LDN) is developed to achieve deterministic transmission in large-scale networks, which can provide bounded delay and jitter with the Cycle Specified Queuing and Forwarding (CSQF) and shaping mechanisms. Routing for time-sensitive flows in LDN requires meeting the delay and bandwidth constraints, like the Multi-Constrained Path (MCP) problem. However, there is a discrepancy that the constraints in the MCP problem are definite while in LDN they are uncertain. It is because the flow’s rate can be adjusted by the shaper, which influences the waiting time at the ingress node and reserved bandwidth at the path. We call the routing problem in LDN the Multi Variable Constraints Routing (MVCR) problem. Due to the uncertainty of constraints, existing routing algorithms may encounter overlong runtime or early rejection if applied to the MVCR problem. In this paper, we propose a Heuristic Fast Routing solution to address the MVCR problem in LDN, called HFR-L. Firstly, we establish a pathbook in advance and design a metric for selecting routes from it with taking the variable flow’s rate into consideration. Then, given the possibility of network changes or the absence of feasible paths in pathbook, we design an algorithm to compute routes in real time, which is based on an extended Lagrange Relaxation based Aggregated Cost (LARAC) algorithm and continuously adjusts the flow’s rate to balance the constraints. The experiments show that our HFR-L has excellent routing performance and fast execution speed in both global and online scenarios, confirming its feasibility to be used in LDN.

Traffic engineering (TE) is fundamental and important in modern communication networks. Deep reinforcement learning (DRL)-based TE solutions can solve TE in a data-driven and model-free way thus have attracted much attention recently. However, most of these solutions ignore that TE is a real-world application and there are challenges applying DRL to real-world TE like: (1) Efficiency. Existing learning-from-scratch DRL agent needs long-time interactions to find solutions better than traditional methods. (2) Safety. Existing DRL-based solutions make TE decisions without considering safety constraints, poor decisions may be made and cause significant performance degradation. In this paper, we propose a safe training approach for DRL-based TE, which tries to address the above two problems. It focuses on making full use of data and ensuring safety so that DRL agent for TE can learn more quickly and possibly poor decisions will not be applied to real environment. We implemented the proposed method in ns-3 and simulation results show that our method performs better with faster convergence rate compared to other DRL-based methods while ensuring the safety of the performed TE decisions.

Segment routing (SR) is an emerging architecture that can benefit traffic engineering (TE). To solve TE in SR networks (we call it SR-TE), linear programming (LP) is often used. But LP methods proposed so far for SR-TE are computationally expensive thus do not scale well in practice. To achieve trade-off between performance and time, we can select a set of nodes as candidates for intermediate nodes to route all traffic instead of considering all the nodes. However, existing node selection methods are all rule-based and only pay attention to the structure of network topology without considering flows, so they are not flexible and may lead to poor performance. In this paper, we for the first time formulate node selection for SR-TE as a reinforcement learning (RL) task. When performing node selection, we consider the impact of both topology and traffic matrix. Also, a customized training algorithm for our task is proposed because existing RL algorithms can not be used directly. Performance evaluations on various real-world topologies and traffic matrices show that our method can achieve good TE performance with much less running time.