Estimating trip travel time is indispensable in intelligent transportation systems, especially for the mobile navigation Apps and ride-hailing services. However, the accurate travel time estimation (TTE) is a non-trivial task due to the challenges of time-varying traffic conditions, complex road networks, and external influences. Learning to estimate the travel time from trajectories is prevalent in emerging studies. The explosion of trajectory data facilitates the implementation of end-to-end deep learning models, making them a powerful tool for the TTE. Whereas, the inherent geographic contexts of trajectories are neglected in previous efforts. The inadequate information usage and incomplete feature representation would inevitably result in somewhat unsatisfactory performances and the weak generalization ability. To tackle such problems, we propose a novel deep representation learning approach. Firstly, GPS points of trajectories are encoded via geographic grids, which eases the modeling of spatiotemporal features. Secondly, the learnable matrices of geographic grids and traveled distance undergo a multi-head self-attention module with the positional encoding to capture their inner dependencies. And then, the external influential features of the departure time, day-of-week, and weather information are incorporated into the model through the embedding manipulation. Finally, across-attention module is leveraged to merge all representation results and gain the desired output. Validated on two real-world datasets from Beijing and Chengdu, China, our method significantly outperforms state-of-the-art baselines by 4%-11%. The implementation code of our model is available at the repository:https://github.com/hang-xu-suda/ATRLNN

Public cloud GPU clusters are becoming emerging platforms for training distributed deep learning jobs. Under this training paradigm, the job scheduler is a crucial component to improve user experiences, i.e., reducing training fees and job completion time, which can also save power costs for service providers. However, the scheduling problem is known to be NP-hard. Most existing work divides it into two easier sub-tasks, i.e., ordering task and placement task, which are responsible for deciding the scheduling orders of jobs and placement orders of GPU machines, respectively. Due to the superior adaptation ability, learning-based policies can generally perform better than traditional heuristic-based methods. Nevertheless, there are still two main challenges that have not been well-solved. First, most learning-based methods only focus on ordering or placement policy independently, while ignoring their cooperation. Second, the unbalanced machine performances and resource contention impose huge overhead and uncertainty on job duration, but rarely be considered in existing work. To tackle these issues, this paper presents a dual-agent scheduler framework abstracted from the two sub-tasks to jointly learn the ordering and placement policies and make better-informed scheduling decisions. Specifically, we design an ordering agent with a scalable squeeze-and-communicate strategy for better cooperation; for the placement agent, we propose a novel Random Walk Gaussian Process to learn the performance similarities of GPU machines while being aware of the uncertain performance fluctuation. Finally, the dual-agent is jointly optimized with multi-agent reinforcement learning. Extensive experiments conducted on the real-world production cluster trace demonstrate the superiority of our model.

An increasing number of vehicles cause the deteriorating congestion problem, which leads to the excessive time spent in commuting. Thus, finding fast driving paths gathers growing interest from travelers and governments. However, effective route planning in traffic networks is deemed to be a considerable challenge due to the complex variations of traffic conditions. The existing traffic state aware routing strategies include two main categories: trip planning based on the traffic state before departure or short-term traffic prediction. The former may incur newly emerged jams because of the rapid en-route evolution of traffic state. The nonnegligible prediction errors would make the latter method deviate from the optimal paths. Moreover, these two deterministic routing strategies may easily result in late arrival for important events. To address this nontrivial problem, we propose a sophisticated route planning approach. Specifically, our method employs empirical observations of traffic prediction results to develop statistical models of error distributions. Then, the empirical error distributions are incorporated with the speed prediction values to constitute the stochastic and time-varying (STV) road network model. Thirdly, the least expected time (LET) routing problem in this STV network is defined. To determine the LET paths, we develop the time-varying K-fastest paths algorithm to generate the candidate paths and the discrete numerical method to compare their expected travel time. Finally, we collect the real-world traffic speed and trajectory data for experiments. The comparison results validate that our approach achieves the best performance and the improvement over other baselines is significant in peak hours.

Social psychology and real experiences show that cognitive consistency plays an important role to keep human society in order: if people have a more consistent cognition about their environments, they are more likely to achieve better cooperation. Meanwhile, only cognitive consistency within a neighborhood matters because humans only interact directly with their neighbors. Inspired by these observations, we take the first step to introduce neighborhood cognitive consistency (NCC) into multi-agent reinforcement learning (MARL). Our NCC design is quite general and can be easily combined with existing MARL methods. As examples, we propose neighborhood cognition consistent deep Q-learning and Actor-Critic to facilitate large-scale multi-agent cooperations. Extensive experiments on several challenging tasks (i.e., packet routing, wifi configuration and Google football player control) justify the superior performance of our methods compared with state-of-the-art MARL approaches.

Communication is a crucial factor for the big multi-agent world to stay organized and productive. Recently, Deep Reinforcement Learning (DRL) has been applied to learn the communication strategy and the control policy for multiple agents. However, the practical limited bandwidth in multi-agent communication has been largely ignored by the existing DRL methods. Specifically, many methods keep sending messages incessantly, which consumes too much bandwidth. As a result, they are inapplicable to multi-agent systems with limited bandwidth. To handle this problem, we propose a gating mechanism to adaptively prune less beneficial messages. We evaluate the gating mechanism on several tasks. Experiments demonstrate that it can prune a lot of messages with little impact on performance. In fact, the performance may be greatly improved by pruning redundant messages. Moreover, the proposed gating mechanism is applicable to several previous methods, equipping them the ability to address bandwidth restricted settings.