Reinforcement Learning (RL) over motion skill space has been verified to generate more diverse behaviors than that over low-level control space, and has exhibited superior autonomous driving performance in complex traffic scenarios. However, the incomplete observations pose challenges in achieving efficient skill exploration under unsupervised conditions, hampering the driving performance and applicability. In this paper, we propose a dynamic-aware RL with hybrid network (Da-HnRL) to develop a pseudo-hierarchical planning framework for better motion skill learning in challenging dense traffics. Based on the semi-POMDP modeling, we construct a hybrid network with skip connections as the RL backbone, facilitating a better understanding of the underlying system dynamics. Then we design an efficiency-oriented reward shaping mechanism to incentivize active skill exploration, promoting enhanced trade-off between exploration and exploitation. Furthermore, we provide a comprehensive scoring mechanism for policy identification, ensuring the near-optimality. We validate the proposed methods on challenging dense-traffic tasks. The results demonstrate the superiority of our approach over previous methods, with improved learning efficiency, driving stability and generalization.

It has been found that many real networks, such as power grids and the Internet, are non-robust, i.e., attacking a small set of nodes would cause the paralysis of the entire network. Thus, the Network Enhancement Problem~(NEP), i.e., improving the robustness of a given network by modifying its structure, has attracted increasing attention. Heuristics have been proposed to address NEP. However, a hand-engineered heuristic often has significant performance limitations. A recently proposed model solving NEP by reinforcement learning has shown superior performance than heuristics on in-distribution datasets. However, their model shows considerably inferior out-of-distribution generalization ability when enhancing networks against the degree-based targeted attack. In this paper, we propose a more effective model with stronger generalization ability by incorporating domain knowledge including measurements of local network structures and decision criteria of heuristics. We further design a hierarchical attention model to utilize the network structure directly, where the query range changes from local to global. Finally, we propose neural confined local search~(NCLS) to realize the effective search of a large neighborhood, which exploits a learned model to confine the neighborhood to avoid exhaustive enumeration. We conduct extensive experiments on synthetic and real networks to verify the ability of our models.

Autonomous driving (AD) has experienced rapid development in recent years, and the reinforcement learning (RL) pipeline in trial-and-error manner can surpass human driving ability. However, the poor performance in sample efficiency and generalization limits RL applying in the challenging urban traffic scenarios. In this paper, we build a context-enhanced meta-RL framework with data-reused adaptation for challenging urban AD. At both the meta-learning and adaptation stages, the context-enhanced state representation is designed to reduce the perceptual gap in variant urban scenarios, improving the sample efficiency and robustness. At adaptation stage, the meta-training data with context-enhanced features are reused through propensity estimation to constrain the optimization objective of new tasks, aiming to maintain the good driving performance of meta-trained policy and fast adapt to the new tasks. Extensive experiments are conducted in CARLA simulator with various urban environments and task settings. The learning curves and quantitative comparisons validate the good sample efficiency and generalization of our proposed method, with state-of-the-art driving performance on urban AD benchmarks.

Temporal point process (TPP) is a powerful tool to model the generation of irregular event sequences. With the rise of deep learning, neural TPPs have been proposed and made a great success. In neural TPPs, a state is extracted to characterize the influence of past events and updated accordingly when a new event is observed. Existing works can be categorized into two frameworks; each has pros and cons. Thus, we integrate these frameworks to overcome their shortages. To propose a model under the integrated framework, we first propose a novel and theoretically well-founded continuous-time layer normalized gated recurrent unit (CLNGRU) model, which is simpler, more robust, and more expressive than a pioneering work. Then we propose an enhanced version of CLNGRU (CLNGRU-E) under the integrated framework. We conduct experiments on synthetic and real datasets. Experimental results show the superiority of our models against the state-of-the-art methods on both generative and predictive tasks. Finally, we show the advantages of our models in recovering the underlying generative processes from sampled sequences.

For modern recommender systems, the issue of data sparsity still often arises in many cases. To address this challenge, Cross-Domain Recommendation (CDR) has been explored by transferring knowledge learned from the source domain to alleviate the data sparsity in the target domain, and existing methods typically assume intra-domain samples share more commonalities than variabilities. However, it can also cause negative effects when there exists a high diversity among intra-domain samples. In this paper, we propose a novel Mixed Gate Control (MGC) model to fulfill 1) the division of samples into groups according to their commonalities and 2) the transfer of knowledge based on groups. We stack up multiple MGC layers into Multiple Layers MGC (ML-MGC) and apply it to a large-scale bundle retrieval system in a listed internet company. We conduct experiments on its own commercial dataset and two public real-world datasets. Experimental results show the superiority of our model against state-of-the-art methods for the CDR task. Finally, we present a case study to illustrate the groups identified by our model.

Abstract Shortest path distance query is one of the most fundamental problems in graph theory and applications. Nowadays, the scale of graphs becomes so large that traditional algorithms for shortest path are not available to answer the exact distance query quickly. Many methods based on two-hop labeling have been proposed to solve this problem. However, they cost too much either in preprocessing or query phase to handle large networks containing as many as tens of millions of vertices. In this paper, we propose a novel $k$-hub labeling method to address this problem in large networks with less preprocessing cost while keeping the query time in the microsecond level on average. Technically, two types of labels are presented in our construction, one for distance queries when the actual distance is at most $k-2$, which we call local label, and the other for further distance queries, which we call hub label. Our approach of $k$-hub labeling is essentially different from previous widely used two-hop labeling framework since we construct labels by using hub network structure. We conduct extensive experiments on large real-world networks and the results demonstrate the higher efficiency of our method in preprocessing phase and the much smaller space size of constructed index compared to previous efficient two-hop labeling method, with a comparatively fast query speed.