Although multiview fusion has demonstrated potential in LiDAR segmentation, its dependence on computationally intensive point-based interactions, arising from the lack of fixed correspondences between views such as range view and Bird's-Eye View (BEV), hinders its practical deployment. This paper challenges the prevailing notion that multiview fusion is essential for achieving high performance. We demonstrate that significant gains can be realized by directly fusing Polar and Cartesian partitioning strategies within the BEV space. Our proposed BEV-only segmentation model leverages the inherent fixed grid correspondences between these partitioning schemes, enabling a fusion process that is orders of magnitude faster (170x speedup) than conventional point-based methods. Furthermore, our approach facilitates dense feature fusion, preserving richer contextual information compared to sparse point-based alternatives. To enhance scene understanding while maintaining inference efficiency, we also introduce a hybrid Transformer-CNN architecture. Extensive evaluation on the SemanticKITTI and nuScenes datasets provides compelling evidence that our method outperforms previous multiview fusion approaches in terms of both performance and inference speed, highlighting the potential of BEV-based fusion for LiDAR segmentation.

Conventional camera-based 3D object detectors in autonomous driving are limited to recognizing a predefined set of objects, which poses a safety risk when encountering novel or unseen objects in real-world scenarios. To address this limitation, we present OS-Det3D, a two-stage training framework designed for camera-based open-set 3D object detection. In the first stage, our proposed 3D object discovery network (ODN3D) uses geometric cues from LiDAR point clouds to generate class-agnostic 3D object proposals, each of which are assigned a 3D objectness score. This approach allows the network to discover objects beyond known categories, allowing for the detection of unfamiliar objects. However, due to the absence of class constraints, ODN3D-generated proposals may include noisy data, particularly in cluttered or dynamic scenes. To mitigate this issue, we introduce a joint selection (JS) module in the second stage. The JS module uses both camera bird’s eye view (BEV) feature responses and 3D objectness scores to filter out low-quality proposals, yielding high-quality pseudo ground truth for unknown objects. OS-Det3D significantly enhances the ability of camera 3D detectors to discover and identify unknown objects while also improving the performance on known objects, as demonstrated through extensive experiments on the nuScenes and KITTI datasets.

Traditional monocular visual simultaneous localization and mapping (SLAM) systems rely on point features or line features to estimate and optimize the camera trajectory and build a map of the surrounding environment. However, in complex scenarios such as underground parking, the performance of traditional point-line SLAM systems tends to degrade due to mirror reflection, illumination change, poor texture, and other interference. This paper proposes VPL-SLAM, a structural vertical line supported point-line monocular SLAM system that works well in complex environments such as underground parking or campus. The proposed system leverages structural vertical lines at all instances of the process. With the assistance of the structural vertical lines and global vertical direction, our system can output a more accurate visual odometry result. Furthermore, the resulting map of our system is a more reasonable structural line feature map than the previous point-line-based monocular SLAM systems. Our system has been tested with the popular autonomous driving dataset Kitti Odometry. In addition, to fully test the proposed SLAM system, we also test our system using a self-collected dataset, including underground parking and campus scenarios. As a result, our proposal reveals a more accurate navigation result and a more reasonable structural resulting map compared to state-of-the-art point-line SLAM systems such as Structure PLP-SLAM.

LiDAR point cloud segmentation is one of the most fundamental tasks for autonomous driving scene understanding. However, it is difficult for existing models to achieve both high inference speed and accuracy simultaneously. For example, voxel-based methods perform well in accuracy, while Bird’s-Eye-View (BEV)-based methods can achieve real-time inference. To overcome this issue, we develop an effective 3D-to-BEV knowledge distillation method that transfers rich knowledge from 3D voxel-based models to BEV-based models. Our framework mainly consists of two modules: the voxel-to-pillar distillation module and the label-weight distillation module. Voxel-to-pillar distillation distills sparse 3D features to BEV features for middle layers to make the BEV-based model aware of more structural and geometric information. Label-weight distillation helps the model pay more attention to regions with more height information. Finally, we conduct experiments on the SemanticKITTI dataset and Paris-Lille-3D. The results on SemanticKITTI show more than 5% improvement on the test set, especially for classes such as motorcycle and person, with more than 15% improvement. The code can be accessed at https://github.com/fengjiang5/Knowledge-Distillation-from-Cylinder3D-to-PolarNet.

3D point cloud semantic segmentation is one of the fundamental tasks for environmental understanding. Although significant progress has been made in recent years, the performance of classes with few examples or few points is still far from satisfactory. In this paper, we propose a novel multi-to-single knowledge distillation framework for the 3D point cloud semantic segmentation task to boost the performance of those hard classes. Instead of fusing all the points of multi-scans directly, only the instances that belong to the previously defined hard classes are fused. To effectively and sufficiently distill valuable knowledge from multi-scans, we leverage a multilevel distillation framework, i.e., feature representation distillation, logit distillation, and affinity distillation. We further develop a novel instance-aware affinity distillation algorithm for capturing high-level structural knowledge to enhance the distillation efficacy for hard classes. Finally, we conduct experiments on the SemanticKITTI dataset, and the results on both the validation and test sets demonstrate that our method yields substantial improvements compared with the baseline method. The code is available at https://github.com/skyshoumeng/M2SKD.

Monocular 3D object detection aims to locate objects in different scenes with just a single image. Due to the absence of depth information, several monocular 3D detection techniques have emerged that rely on auxiliary depth maps from the depth estimation task. There are multiple approaches to understanding the representation of depth maps, including treating them as pseudo-LiDAR point clouds, leveraging implicit end-to-end learning of depth information, or considering them as an image input. However, these methods have certain drawbacks, such as their reliance on the accuracy of estimated depth maps and suboptimal utilization of depth maps due to their image-based nature. While LiDAR-based methods and convolutional neural networks (CNNs) can be utilized for pseudo point clouds and depth maps, respectively, it is always an alternative. In this paper, we propose a framework named the Adaptive Distance Interval Separation Network (ADISN) that adopts a novel perspective on understanding depth maps, as a form that lies between LiDAR and images. We utilize an adaptive separation approach that partitions the depth map into various subgraphs based on distance and treats each of these subgraphs as an individual image for feature extraction. After adaptive separations, each subgraph solely contains pixels within a learned interval range. If there is a truncated object within this range, an evident curved edge will appear, which we can leverage for texture extraction using CNNs to obtain rich depth information in pixels. Meanwhile, to mitigate the inaccuracy of depth estimation, we designed an uncertainty module. To take advantage of both images and depth maps, we use different branches to learn localization detection tasks and appearance tasks separately. Our approach significantly enhances the baseline and outperforms depth-assisted techniques, as shown by our extensive experiments on the KITTI monocular 3D object detection benchmark.

Abstract Siamese network based trackers formulate tracking as a similarity matching problem between a target template and a search region. Virtually all popular Siamese trackers use cross‐correlation to measure the similarity between the deep feature of template and search image. However, the emphasis for feature extraction in different parts of the image are the same. Besides, the global matching between the template and search region also seriously neglects the part‐level information and the deformation of targets during tracking. In this study, to tackle the above issues, a simple but effective Dynamic Adjust‐Head (SiamDAH) model is proposed to extract features from different parts of an object. In addition, an improved pixelwise cross‐correlation model (PWCC) is designed to enhance the naive cross‐correlation operation to produce multiple similarity maps associated with different parts of the target. Experiments on serval challenging benchmarks including OTB‐100, GOT‐10k, LaSOT, and TrackingNet demonstrate that the proposed SiamDAH outperforms many state‐of‐the‐art trackers and achieves leading performance.

Siamese network-based object tracking algorithms have recently gained popularity due to continuous improvement in tracking accuracy and robustness. However, these trackers are often limited by inefficient initialization and unstable online update. The model drift accumulates as more frames are processed. To address the above issues, this paper introduced a method that combines both deep learning and conventional regression algorithms (SVR, Support Vector Regression). Specifically, we propose a feature projection algorithm, which can effectively reduce the dimension of the features extracted from deep neural networks and improve the distinguishability of them at the same time. To make the features more robust for SVR training, we further propose two feature aggregation methods at both the channel level and the spatial level. We update the SVR model online instead of the deep neural network to make the tracking process more robust. Extensive experiments on four challenging benchmarks indicate that our proposed tracker is superior to baseline methods both qualitatively and quantitatively.