Traffic simulation, complementing real-world data with a long-tail distribution, allows for effective evaluation and enhancement of the ability of autonomous vehicles to handle accident-prone scenarios. Simulating such safety-critical scenarios is nontrivial, however, from log data that are typically regular scenarios, especially in consideration of dynamic adversarial interactions between the future motions of autonomous vehicles and surrounding traffic participants. To address it, this paper proposes an innovative and efficient strategy, termed IntSim, that explicitly decouples the driving intentions of surrounding actors from their motion planning for realistic and efficient safety-critical simulation. We formulate the adversarial transfer of driving intention as an optimization problem, facilitating extensive exploration of diverse attack behaviors and efficient solution convergence. Simultaneously, intention-conditioned motion planning benefits from powerful deep models and large-scale real-world data, permitting the simulation of realistic motion behaviors for actors. Specially, through adapting driving intentions based on environments, IntSim facilitates the flexible realization of dynamic adversarial interactions with autonomous vehicles. Finally, extensive open-loop and closed-loop experiments on real-world datasets, including nuScenes and Waymo, demonstrate that the proposed IntSim achieves state-of-the-art performance in simulating realistic safety-critical scenarios and further improves planners in handling such scenarios.

Continual learning aims to acquire new knowledge while retaining past information. Class-incremental learning (CIL) presents a challenging scenario where classes are introduced sequentially. For video data, the task becomes more complex than image data because it requires learning and preserving both spatial appearance and temporal action involvement. To address this challenge, we propose a novel exemplar-free framework that equips separate spatiotemporal adapters to learn new class patterns, accommodating the incremental information representation requirements unique to each class. While separate adapters are proven to mitigate forgetting and fit unique requirements, naively applying them hinders the intrinsic connection between spatial and temporal information increments, affecting the efficiency of representing newly learned class information. Motivated by this, we introduce two key innovations from a causal perspective. First, a causal distillation module is devised to maintain the relation between spatial-temporal knowledge for a more efficient representation. Second, a causal compensation mechanism is proposed to reduce the conflicts during increment and memorization between different types of information. Extensive experiments conducted on benchmark datasets demonstrate that our framework can achieve new state-of-the-art results, surpassing current example-based methods by 4.2% in accuracy on average. The codes are accessible in https://github.com/tychen-SJTU/CSTA.

Network planning is crucial to facilitate network service under limited network operation costs. However, adapting the network topology (i.e., connections and capacities for physical and IP links) to time-varying and stochastic network states is challenging due to high computational overhead of problem solving. Existing deep learning based methods suffer from prohibitive computational complexity due to simultaneously solving IP link capacities and allocated traffics or iteratively exploring search space with excessive samples via heuristic rules. In this paper, we propose GDPlan, the first generative framework that leverages conditional graph diffusion model to address this challenge. To achieve high solving efficiency, GDPlan decouples network planning into two separate stages, i.e., generation of diverse high-quality solutions to IP link capacity and refinement of the solutions under feasibility constraints. Specifically, we develop generation-oriented graph signal modeling that reformulates solving IP link capacities as generation of graph topology conditioned on the graph signals related to traffic demands, physical links and network costs. Consequently, energy-guided controllable graph generation is achieved to capture significant structural patterns of graph topology with score-based graph diffusion model and produce diverse solutions with a guarantee of feasibility under specific objectives or constraints. GDPlan successfully achieves graph generation based solution to network planning with an order of magnitude higher solving speed than the commercial solver Gurobi. Experimental results demonstrate that, compared with Gurobi, the proposed GDPlan obtains lowest 3.7% average gap with only about 7.5% average running time.

Data augmentation methods to enhance perception performance in adverse weather have recently attracted considerable attention. Most of the LiDAR data augmentation methods post-process the existing dataset by physics-based models or machine-learning methods. However, due to the limited environmental annotations and the fixed vehicle trajectories in existing datasets, it is challenging to edit the scene and expand the diversity of traffic flow and scenario. To this end, we propose a simulator-based physical modeling approach to augment LiDAR data in rainy weather, enhancing the performance of the perception model. We complete the modeling task of the rainy weather effect in the CARLA simulator and establish a data collection pipeline for LiDAR. Furthermore, we pay special attention to the spray generated by vehicles in rainy weather and simulate this phenomenon through the Spray Emitter method we developed. In addition, considering the influence of different weather conditions on point cloud intensity, we develop a prediction network to forecast the intensity of the LiDAR echo. This enables us to complete the rainy weather simulation of 4D point cloud data. In the experiment, we observe that the model augmented by our synthetic dataset improves the performance for 3D object detection in rainy weather. Both code and dataset are available at https://github.com/PJLab-ADG/PCSim#rainypcsim.

With the advent of data science, the analysis of network or graph data has become a very timely research problem. A variety of recent works have been proposed to generalize neural networks to graphs, either from a spectral graph theory or a spatial perspective. The majority of these works, however, focus on adapting the convolution operator to graph representation. At the same time, the pooling operator also plays an important role in distilling multiscale and hierarchical representations, but it has been mostly overlooked so far. In this article, we propose a parameter-free pooling operator, called iPool, that permits to retain the most informative features in arbitrary graphs. With the argument that informative nodes dominantly characterize graph signals, we propose a criterion to evaluate the amount of information of each node given its neighbors and theoretically demonstrate its relationship to neighborhood conditional entropy. This new criterion determines how nodes are selected and coarsened graphs are constructed in the pooling layer. The resulting hierarchical structure yields an effective isomorphism-invariant representation of networked data on arbitrary topologies. The proposed strategy achieves superior or competitive performance in graph classification on a collection of public graph benchmark data sets and superpixel-induced image graph data sets.

A number of models have been proposed to capture rich semantic representation in Visual Question Answering (VQA). In this paper, we illustrate the compositionality of general cognitive ability in VQA and take the linguistic structure of language into consideration in semantic representation. We decompose the question into several components by the semantic tree and apply a tree-structured model to distill the sentence representation. In addition, we exploit the complementary image of the new dataset and optimize the classifier used to predict answers. We design a dual path network for the new VQA 2.0 dataset in training process to lead the model to effectively take advantage of the property of the dataset. Experiments show that our method could obtain more useful information and improve the performance.

Most state-of-the-art methods for representation learning are supervised, which require a large number of labeled data. This paper explores a novel unsupervised approach for learning visual representation. We introduce an image-wise discrimination criterion in addition to a pixel-wise reconstruction criterion to model both individual images and the difference between original images and reconstructed ones during neural network training. These criteria induce networks to focus on not only local features but also global high-level representations, so as to provide a competitive alternative to supervised representation learning methods, especially in the case of limited labeled data. We further introduce a competition mechanism to drive each component to increase its capability to win its adversary. In this way, the identity of representations and the likeness of reconstructed images to original ones are alternately improved. Experimental results on several tasks demonstrate the effectiveness of our approach.

Given a set of labeled data with semantic descriptions, zero-shot learning aims at recognizing objects from unseen classes, where no instances of the classes are used during training. Most existing methods solve this problem via embedding images and labels into an embedding space and computing similarity across different information sources. However, the similarity calculation could be unreliable when the training and testing data distributions are inconsistent. In this paper, we propose a novel zero-shot learning model that forms a neighborhood-preserving structure in the semantic embedding space and utilize it to predict classifiers for unseen classes. By constructing a locally connected graph for class embeddings, we exploit the structural constraint of embeddings of similar classes and retain the global structure in the semantic embedding space to obtain an effective representation of semantic information. Experiment results on three benchmark datasets demonstrate that the proposed method generates effective semantic representations and out-performs state-of-the-art methods.

This paper proposes a hybrid wavelet convolution network (HWCN) which is composed of a scattering convolution component and a convolution neural component. The hierarchical end-to-end network implements sparse-coding and high-dimensional reconstruction for inverse problem through cascade convolutions. With the pre-defined scattering convolutions from nonlinear operators, the network can be tailored in accordance with the frequency property to provide sparse code candidates, and the convolution neural component could automatically select and weight these candidates for sparse coding. Given a tiny dataset, HWCN could train complex deep network with better generalization by regularization from scattering convolutions, and thereby is a competitive alternative to convolutional neural networks (CNN). Moreover, we further demonstrate that HWCN is a superior selection of sparse-coding based image super-resolution and achieves state-of-the-art performance.