Motion planning is a critical module in autonomous driving, with the primary challenge of uncertainty caused by interactions with other participants. As most previous methods treat prediction and planning as separate tasks, it is difficult to model these interactions. Furthermore, since the route path navigates ego vehicles to a predefined destination, it provides relatively stable intentions for ego vehicles and helps constrain uncertainty. On this basis, we construct Int2Planner, an Intention-based Integrated motion Planner achieves multi-modal planning and prediction. Instead of static intention points, Int2Planner utilizes route intention points for ego vehicles and generates corresponding planning trajectories for each intention point to facilitate multi-modal planning. The experiments on the private dataset and the public nuPlan benchmark show the effectiveness of route intention points, and Int2Planner achieves state-of-the-art performance. We also deploy it in real-world vehicles and have conducted autonomous driving for hundreds of kilometers in urban areas. It further verifies that Int2Planner can continuously interact with the traffic environment.

Autonomous driving is a challenging task that requires perceiving and understanding the surrounding environment for safe trajectory planning. While existing vision-based end-to-end models have achieved promising results, these methods are still facing the challenges of vision understanding, decision reasoning and scene generalization. To solve these issues, a generative planning with 3D-vision language pre-training model named GPVL is proposed for end-to-end autonomous driving. The proposed paradigm has two significant aspects. On one hand, a 3D-vision language pre-training module is designed to bridge the gap between visual perception and linguistic understanding in the bird's eye view. On the other hand, a cross-modal language model is introduced to generate reasonable planning with perception and navigation information in an auto-regressive manner. Experiments on the challenging nuScenes dataset demonstrate that the proposed scheme achieves excellent performances compared with state-of-the-art methods. Besides, the proposed GPVL presents strong generalization ability and real-time potential when handling high-level commands in various scenarios. It is believed that the effective, robust and efficient performance of GPVL is crucial for the practical application of future autonomous driving systems.

We present GDFusion, a temporal fusion method for vision-based 3D semantic occupancy prediction (VisionOcc). GDFusion opens up the underexplored aspects of temporal fusion within the VisionOcc framework, focusing on both temporal cues and fusion strategies. It systematically examines the entire VisionOcc pipeline, identifying three fundamental yet previously overlooked temporal cues: scene-level consistency, motion calibration, and geometric complementation. These cues capture diverse facets of temporal evolution and make distinct contributions across various modules in the VisionOcc framework. To effectively fuse temporal signals across heterogeneous representations, we propose a novel fusion strategy by reinterpreting the formulation of vanilla RNNs. This reinterpretation leverages gradient descent on features to unify the integration of diverse temporal information, seamlessly embedding the proposed temporal cues into the network. Extensive experiments on nuScenes demonstrate that GDFusion significantly outperforms established baselines, achieving 2.2%–4.7% mIoU improvement and reducing memory consumption by 30%–72%. Codes are available at https: //github.com/cdb342/GDFusion.

Vision-based 3D semantic occupancy prediction is a critical task in 3D vision that integrates volumetric 3D reconstruction with semantic understanding. Existing methods, however, often rely on modular pipelines. These modules are typically optimized independently or use pre-configured inputs, leading to cascading errors. In this paper, we address this limitation by designing a novel causal loss that enables holistic, end-to-end supervision of the modular 2D-to-3D transformation pipeline. Grounded in the principle of 2D-to-3D semantic causality, this loss regulates the gradient flow from 3D voxel representations back to the 2D features. Consequently, it renders the entire pipeline differentiable, unifying the learning process and making previously non-trainable components fully learnable. Building on this principle, we propose the Semantic Causality-Aware 2D-to-3D Transformation, which comprises three components guided by our causal loss: Channel-Grouped Lifting for adaptive semantic mapping, Learnable Camera Offsets for enhanced robustness against camera perturbations, and Normalized Convolution for effective feature propagation. Extensive experiments demonstrate that our method achieves state-of-the-art performance on the Occ3D benchmark, demonstrating significant robustness to camera perturbations and improved 2D-to-3D semantic consistency.

Reconstructing pedestrian motion from dynamic sensors, with a focus on pedestrian intention, is crucial for advancing autonomous driving safety. However, this task is challenging due to data limitations arising from technical complexities, safety, and cost concerns. We introduce the Pedestrian Motion Reconstruction (PMR) dataset, which focuses on pedestrian intention to reconstruct behavior using multiple perspectives and modalities. PMR is developed from a mixed reality platform that combines real-world realism with the extensive, accurate labels of simulations, thereby reducing costs and risks. It captures the intricate dynamics of pedestrian interactions with objects and vehicles, using different modalities for a comprehensive understanding of human-vehicle interaction. Analyses show that PMR can naturally exhibit pedestrian intent and simulate extreme cases. PMR features a vast collection of data from 54 subjects interacting across 12 urban settings with 7 objects, encompassing 12,138 sequences with diverse weather conditions and vehicle speeds. This data provides a rich foundation for modeling pedestrian intent through multi-view and multi-modal insights. We also conduct comprehensive benchmark assessments across different modalities to thoroughly evaluate pedestrian motion reconstruction methods.

Unmanned sanitation vehicles (USVs) promise cleaner cities, yet efficiently coordinating multiple USVs in large urban areas remains challenging due to constraints such as limited waste capacity and battery life. In this demo, we present MRTC, a multi-robot task cooperation system. First, Dynamic Task Assignment employs an Actor-Critic policy within a Markov decision framework to allocate cleaning tasks and decide the required number of USVs. Second, Single-USV Path Planning refines each route via a fast two-layer iterative search. Over an eight-month real-world deployment in three urban testbeds, our MRTC system markedly improved cleaning efficiency while lowering operating costs. Operating over a combined 10,775 km of routes per month, the system achieved average monthly savings of 20,575 kWh of energy and 2,744 labour hours. A demonstration video is available at https://llq978.github.io/Demo/.

Although machine learning models have been widely used in electricity theft detection, most of them lack interpretability, which hinders user trust and policy enforcement. To this end, this paper aims to investigate the interpretability of machine learning models in electricity theft detection. Specifically, a comprehensive theoretical analysis is conducted to reveal why the interpretability is needed in electricity theft detection. Then, a model-agnostic explainable artificial intelligence (XAI) framework is proposed to uncover the potential start and end times of fraudulent behavior, and to clarify the rationale behind identifying fraudulent users within machine learning models by calculating the importance score of each data point. Simulation results demonstrate that the XAI framework provides class-discriminative data points to interpret fraudulent activities, enabling suspicious users to understand why the machine learning model identified them as suspicious and guiding model improvement. Moreover, compared with benchmarks (e.g., Shapley additive explanations, local interpretable model-agnostic explanations, and gradient-weighted class activation mapping techniques), the harmonic mean of overlap and coverage (HMOC) of the proposed XAI framework is improved by 10.26% to 54.73%, indicating more trustworthy interpretations.

Recent advances in neural networks have significantly improved electricity theft detection, achieving higher detection accuracy compared to earlier methods (e.g., support vector machine and decision tree). However, the performance of these networks is still restricted by the class imbalance issue, which causes the neural networks to bias toward classifying unknown users as the majority class (i.e., normal users). While previous works have developed oversampling and data augmentation techniques to alleviate this problem at the data level, these techniques usually replicate existing fraudulent samples or generate similar ones, which can lead to overfitting and, thus, limit model performance. To this end, this article aims to mitigate the class imbalance issue from a novel perspective at the algorithmic level. Specifically, a sample-weighted (SW) loss is proposed to efficiently train neural networks by assigning different weights to samples based on their importance, in contrast to most existing works, which treat all samples equally. Notably, the proposed SW loss is independent of any specific model architecture, meaning that it can be seamlessly integrated with various neural networks to update their weights for electricity theft detection. Simulation results on real-world datasets show that the proposed SW loss outperforms baselines (e.g., binary cross entropy loss, class-balanced loss, oversampling, and data augmentation), with an increase of about 0.27% to 9.78% in mean average precision and 0.14% to 2.92% in the area under the curve, respectively.

Instance segmentation is a fundamental research in computer vision, especially in autonomous driving. However, manual mask annotation for instance segmentation is quite time-consuming and costly. To address this problem, some prior works attempt to apply weakly supervised manner by exploring 2D or 3D boxes. However, no one has ever successfully segmented 2D and 3D instances simultaneously by only using 2D box annotations, which could further reduce the annotation cost by an order of magnitude. Thus, we propose a novel framework called Multimodal Weakly Supervised Instance Segmentation (MWSIS), which incorporates various fine-grained label correction modules for both 2D and 3D modalities, along with a new multimodal cross-supervision approach. In the 2D pseudo label generation branch, the Instance-based Pseudo Mask Generation (IPG) module utilizes predictions for self-supervised correction. Similarly, in the 3D pseudo label generation branch, the Spatial-based Pseudo Label Generation (SPG) module generates pseudo labels by incorporating the spatial prior information of the point cloud. To further refine the generated pseudo labels, the Point-based Voting Label Correction (PVC) module utilizes historical predictions for correction. Additionally, a Ring Segment-based Label Correction (RSC) module is proposed to refine the predictions by leveraging the depth prior information from the point cloud. Finally, the Consistency Sparse Cross-modal Supervision (CSCS) module reduces the inconsistency of multimodal predictions by response distillation. Particularly, transferring the 3D backbone to downstream tasks not only improves the performance of the 3D detectors, but also outperforms fully supervised instance segmentation with only 5% fully supervised annotations. On the Waymo dataset, the proposed framework demonstrates significant improvements over the baseline, especially achieving 2.59% mAP and 12.75% mAP increases for 2D and 3D instance segmentation tasks, respectively. The code is available at https://github.com/jiangxb98/mwsis-plugin.

Many contemporary studies utilize grid-based models for neural field representation, but a systematic analysis of grid-based models is still missing, hindering the improvement of those models. Therefore, this paper introduces a theoretical framework for grid-based models. This frame-work points out that these models' approximation and generalization behaviors are determined by grid tangent ker-nels (GTK), which are intrinsic properties of grid-based models. The proposed framework facilitates a consistent and systematic analysis of diverse grid-based models. Furthermore, the introduced framework motivates the development of a novel grid-based model named the Multiplicative Fourier Adaptive Grid (MulFAGrid). The numerical analysis demonstrates that MulFAGrid exhibits a lower generalization bound than its predecessors, indicating its robust generalization performance. Empirical studies reveal that MulFAGrid achieves state-of-the-art performance in various tasks, including 2D image fitting, 3D signed distance field (SDF) reconstruction, and novel view synthesis, demonstrating superior representation ability. The project website is available at this link.

AlphaRotate is an open-source TensorFlow benchmark for performing scalable rotation detection on various datasets. It currently provides more than 16 popular rotation detection models under a single, well-documented API designed for use by both practitioners and researchers. AlphaRotate regards high performance, robustness, sustainability and scalability as the core concept of design, and all models are covered by continuous integration, code coverage, maintainability checks, and visual monitoring and analysis. AlphaRotate can be installed from PyPI and is released under the Apache-2.0 License. Source code is available at https://github.com/yangxue0827/RotationDetection.

Traditional CNN-based training for instance segmentation is time-consuming owing to large datasets and complex network modules, making direct searching of architecture challenging. In this paper, we introduce an efficient framework, named EASInst. It can discover practical backbone and encoder architectures for the improved sparse activation instance segmentation model. Specifically, we construct a supernet for both backbone and encoder modules of SparseInst based on a differentiable method. In addition, kernel sharing mask and channel pruning technology are employed. Moreover, Taylor-Loss and a novel DY-Loss are devised for instance segmentation to improve the accuracy. Experiments show that the searched architectures outperform the existing Resnet-based real-time instance segmentation methods, which achieve 38.5 mAP with 39.5 FPS on COCO test-dev set.

Camera calibration is crucial in computer vision tasks and applications, e.g., autonomous driving (AD). However, prevailing camera calibration models pose a time-consuming and labor-intensive off-board process in mass production settings, while simultaneously lacking exploration of real-world AD scenarios. To this end, inspired by recent advancements in bird's-eye-view (BEV) perception models, this paper proposes a novel multi-camera Calibration method via Reversed BEV representations for AD, termed CalibRBEV. Specifically, the proposed CalibRBEV model primarily comprises two stages. Initially, we innovatively reverse the BEV perception pipeline, reconstructing bounding boxes through an attention auto-encoder module to fully extract the latent reversed BEV representations. Subsequently, the obtained representations from encoder are interacted with the surrounding multi-view image features for further refinement and calibration parameters prediction. Extensive experimental results on nuScenes and Waymo datasets validate the effectiveness of our proposed model.

Machine learning techniques have been extensively developed in the field of electricity theft detection. However, almost all typical models primarily rely on electricity consumption data to identify fraudulent users, often neglecting other pertinent household information such as gas consumption data. This article aims to explore the untapped potential of gas consumption data, a critical yet overlooked factor in electricity theft detection. In particular, we perform theoretical, qualitative, and quantitative correlation analyses between gas and electricity consumption data. Then, we propose two model-agnostic frameworks (i.e., multichannel network and twin network frameworks) to seamlessly integrate gas consumption data into machine learning models. Simulation results show a significant improvement in model performance when gas consumption data are incorporated using our proposed frameworks. Also, our proposed gas and electricity convolutional neural network, based on the proposed framework, demonstrates superior performance compared to classical and recent machine learning models on datasets with varying fraudulent ratios.

The integrations of advanced metering infrastructure and smart meters make it possible to detect electricity thieves by analyzing electricity consumption readings. However, the detection accuracies of traditional models are limited due to their difficulty in capturing the periodicity and latent features from electricity consumption readings. To solve this problem, a graph attention network (GAT)-based model is proposed to improve the detection accuracy from a fresh viewpoint on graph domains. First, a new strategy is presented to transform raw one-dimensional electricity consumption readings into dynamic graphs, which represent the features and periodicity through feature matrices and correlation matrices, respectively. Then, a GAT is migrated from traditional graph inferences into electricity theft detection, in which necessary adjustments are made on structures to capture periodicity and latent features from dynamic graphs. Case studies show that the proposed model outperforms popular baselines for a wide range of training ratios and fraudulent ratios.

Offshore wind farms (OWFs) with modular multilevel converter high-voltage dc (MMC-HVdc) have become an important form of renewable energy utilization. However, if a fault occurs at the tie line between the MMC and the OWF, the fault steady-state current at the fault point will be equal to zero when the negative-sequence current is suppressed, so traditional differential protection may fail to operate. To cope with this issue, this article proposes a new pilot protection method based on Minkowski distance. For internal faults, OWF and MMC will have different transient currents, so the Minkowski distance will be much higher than 0, but it will be equal to 0 for external faults and normal operation since the fault currents on both sides are the opposite. Therefore, the internal fault can be detected reliably. The proposed method does not depend on the power frequency phasor extraction, so it is not affected by the frequency offset in the transient process. In addition, this method operates quickly and has a strong ability to withstand fault resistance and environmental noise. Moreover, since there is always a transient process when a fault occurs, the proposed method applies to different fault ride-through strategies. PSCAD simulation and real-time digital simulator experiments show that the proposed method is suitable for different fault locations and types.

Small and cluttered objects are common in real-world which are challenging for detection. The difficulty is further pronounced when the objects are rotated, as traditional detectors often routinely locate the objects in horizontal bounding box such that the region of interest is contaminated with background or nearby interleaved objects. In this paper, we first innovatively introduce the idea of denoising to object detection. Instance-level denoising on the feature map is performed to enhance the detection to small and cluttered objects. To handle the rotation variation, we also add a novel IoU constant factor to the smooth L1 loss to address the long standing boundary problem, which to our analysis, is mainly caused by the periodicity of angular (PoA) and exchangeability of edges (EoE). By combing these two features, our proposed detector is termed as SCRDet++. Extensive experiments are performed on large aerial images public datasets DOTA, DIOR, UCAS-AOD as well as natural image dataset COCO, scene text dataset ICDAR2015, small traffic light dataset BSTLD and our released S$^{2}$TLD by this paper. The results show the effectiveness of our approach. The released dataset S$^{2}$TLD is made public available, which contains 5,786 images with 14,130 traffic light instances across five categories.

In autonomous driving system, High-Definition (HD) map is an important basis for localization, perception and planning tasks. For the construction of HD map, land marking detection is the first step. Recent studies of land marking detection are mainly based on camera image data, while LiDAR-based land marking detection is rarely studied. The main reason is that there are few datasets specially developed for land marking detection. In this paper, a new LiDAR-based land marking dataset named OpenDenseLane is developed for the construction of HD map, and is released to support the academic research. OpenDenseLane contains 1,709 scenarios with 57,227 frames, and each frame includes two types of LiDAR point cloud, camera image and localization data. The LiDAR data in our dataset are dense point clouds to reduce the impact of sparse point distribution. OpenDense-Lane provides abundant annotations of ground signs, such as lane line, crosswalk and turn arrow. Experiments are conducted on the proposed dataset and the results of land marking detection and HD map construction are analysed. Open-DenseLane will be released at https://github.com/Thinklab-SJTU/OpenDenseLane.

Trajectory prediction plays an important role in modern au-tonomous driving system. The multi-modal characteristics of trajectory prediction makes it difficult to accurately predict the driving intention and future trajectory of vehicles, espe-cially in the complex conditions, such as lane changing and road intersection. To improve the prediction performance, ad-ditional features are needed. High-definition (HD) map fea-ture and tail light feature are employed in this paper, which are fused with trajectory feature to assist vehicle trajectory pre-diction. A prediction model based on temporal convolution network (TCN) and graph convolution network (GCN) is constructed with corresponding loss functions. Experiments are carried out on our simulation dataset and the results show the effectiveness of the proposed feature fusion method as well as the prediction model.

Recent studies show that differentiable architecture search (DARTS) suffers notable instability and collapse issue: skip-connect may gradually dominate the cell, leading to deteri-orating architectures. We conjecture that the domination of skip-connect is due to its superiority in gradient compen-sate. On this foundation, we propose a novel and stable method, called DistillDARTS, to stabilize DARTS by knowl-edge distillation and self-distillation scheme. Specifically, the distillation is able to serve as a substitute for skip-connect and smooth the back-propagated gradient distributions among layers of DARTS. By compensating gradients in shallow lay-ers, our method can relieve the dependence of gradient on skip-connect and hence mitigates the collapse issue. Exten-sive experiments on a range of benchmarks demonstrate that DistillDARTS can obtain sturdy architectures with few skip-connects without additional manual interventions, thus suc-cessfully improving the robustness of DARTS. Due to the im-proved stability, our proposed approach achieves the accuracy of 97.57% on CIFAR-10 and 75.8% on ImageNet.

Unique fault behaviors of renewable energy sources (RESs) may lead to the misoperation of traditional pilot protection. To cope with this issue, this article proposes a new pilot protection method using the improved Euclidean distance. For normal operation or external faults, the currents on both ends are completely opposite, so the Euclidean distance of current absolute values on both ends is equal to 0. However, it will be much larger than 0 for internal faults because transient currents on both ends will have a big difference at this time. Therefore, internal and external faults can be detected reliably. In order to facilitate the setting calculation, the Euclidean distance is normalized, and a stability factor is introduced to avoid invalid calculation results. The proposed method can be applied to different RES types and different fault ride through strategies. Meanwhile, it can withstand larger fault resistance and noise interference. Compared with other methods using the RES fault currents, this approach can operate correctly without any additional criteria when the circuit breaker recloses on a permanent fault or RESs output a low power. PSCAD simulation and real-time digital simulator experiment verify this method.

The Deep learning of optical flow has been an active area for its empirical success. For the difficulty of obtaining accurate dense correspondence labels, unsupervised learning of optical flow has drawn more and more attention, while the accuracy is still far from satisfaction. By holding the philosophy that better estimation models can be trained with betterapproximated labels, which in turn can be obtained from better estimation models, we propose a self-taught learning framework to continually improve the accuracy using self-generated pseudo labels. The estimated optical flow is first filtered by bidirectional flow consistency validation and occlusion-aware dense labels are then generated by edge-aware interpolation from selected sparse matches. Moreover, by combining reconstruction loss with regression loss on the generated pseudo labels, the performance is further improved. The experimental results demonstrate that our models achieve state-of-the-art results among unsupervised methods on the public KITTI, MPI-Sintel and Flying Chairs datasets.