Deep learning-based feature matching has showcased great superiority for point cloud registration. While coarse-to-fine matching architectures are prevalent, they typically perform sparse and geometrically inconsistent coarse matching. This forces the subsequent fine matching to rely on computationally expensive optimal transport and hypothesis-and-selection procedures to resolve inconsistencies, leading to inefficiency and poor scalability for large-scale real-time applications. In this paper, we design a consistency-aware spot-guided Transformer (CAST) to enhance the coarse matching by explicitly utilizing geometric consistency via two key sparse attention mechanisms. First, our consistency-aware self-attention selectively computes intra-point-cloud attention to a sparse subset of points with globally consistent correspondences, enabling other points to derive discriminative features through their relationships with these anchors while propagating global consistency for robust correspondence reasoning. Second, our spot-guided cross-attention restricts cross-point-cloud attention to dynamically defined "spots"-the union of correspondence neighborhoods of a query's neighbors in the other point cloud, which are most likely to cover the true correspondence of the query ensured by local consistency, eliminating interference from similar but irrelevant regions. Furthermore, we design a lightweight local attention-based fine matching module to precisely predict dense correspondences and estimate the transformation. Extensive experiments on both outdoor LiDAR datasets and indoor RGB-D camera datasets demonstrate that our method achieves state-of-the-art accuracy, efficiency, and robustness. Besides, our method showcases superior generalization ability on our newly constructed challenging relocalization and loop closing benchmarks in unseen domains.

In robotic navigation, maintaining precise pose estimation and navigation in complex and dynamic environments is crucial. However, environmental challenges such as smoke, tunnels, and adverse weather can significantly degrade the performance of single-sensor systems like LiDAR or GPS, compromising the overall stability and safety of autonomous robots. To address these challenges, we propose AF-RLIO: an adaptive fusion approach that integrates 4D millimeterwave radar, LiDAR, inertial measurement unit (IMU), and GPS to leverage the complementary strengths of these sensors for robust odometry estimation in complex environments. Our method consists of three key modules. Firstly, the pre-processing module utilizes radar data to assist LiDAR in removing dynamic points and determining when environmental conditions are degraded for LiDAR. Secondly, the dynamic-aware multimodal odometry selects appropriate point cloud data for scan-tomap matching and tightly couples it with the IMU using the Iterative Error State Kalman Filter. Lastly, the factor graph optimization module balances weights between odometry and GPS data, constructing a pose graph for optimization. The proposed approach has been evaluated on datasets and tested in real-world robotic environments, demonstrating its effectiveness and advantages over existing methods in challenging conditions such as smoke and tunnels. Furthermore, we open source our code at https://github.com/NeSC-IV/AF-RLIO.git to benefit the research community.

Multi-robot exploration in unknown environments is a fundamental task for multi-robot systems, which requires the coordination of the robots to avoid collisions and conflicts while performing task allocation. Existing exploration strategies improve the efficiency of multi-robot exploration by modeling the multi-robot task allocation problem as a variant of the multiple traveling salesman problem. However, this is computationally intensive and difficult to deploy on physical platforms. Hence, this paper develops a hybrid strategy for range-sensing multi-robot exploration with effective team coordination, enabling a larger team dispersion degree and higher exploration efficiency. In addition, we present a novel multi-robot exploration point detection method suitable for narrow and dynamic environments, effectively reducing exploration failure and incompleteness. The Gazebo simulations demonstrate better exploration efficiency and the least time cost of our exploration framework compared with state-of-the-art methods, and real-world experiments also validate the effectiveness. The code is released at https://github.com/NeSC-IV/DHC_ME.

Robotic and autonomous driving platforms necessitate efficient 3D Multi-Object Tracking (MOT) that harmonizes geometric precision, motion robustness, and computational efficiency. Traditional 3D MOT approaches face critical challenges: geometric similarity metrics (e.g., IoU-based) degrade at long ranges with high computational costs, while distance-based methods fail to capture object orientation and shape; the effects of occlusion and the intricate relative ego-object motion degrade tracking performance in dynamic scenes. To this end, we propose PB-MOT, an online framework integrating two key innovations: ego-motion-compensated state estimation that decouples dynamic interactions; and a rotated ellipse association algorithm unifying pose and shape-aware matching with adaptive distance constraints. Evaluations on the KITTI benchmark show that our PB-MOT achieves state-of-the-art performance with a HOTA score of 81.94%, while running at an impressive 2,402.76 FPS on CPU. This enables real-time, high-fidelity perception and tracking for resource-constrained robotic systems.

This work presents a visual odometry (VO) system that leverages image edge features. Edges are spatially expressive cues commonly present across diverse environments, offering rich textural and structural information. However, existing edge-based VO methods often fail to fully exploit this potential. To this end, we introduce a novel feature representation termedorganized edges, which transforms disjoint edge pixels into sequentialized clusters, enabling more effective retention and utilization of the underlying textural and structural information. Another nice property of this formulation is that organized edges can perform edge-level association across multiple frames, enabling the establishment of a co-visibility graph. To achieve precise and efficient pose estimation, we propose a range of particularly designed tracking and joint optimization methods based on the characteristics of organized edges. For tracking, we formulate edge-wise rather than pixel-wise residuals to achieve robust and accurate inter-frame registration. For joint optimization, we introduce a novel shape-preserving edge-fitting method and an organized edge-based Bundle Adjustment (BA) approach, which decomposes the traditional BA problem into fitting and registration to preserve the structural integrity. Based on these novel techniques, we develop a complete VO system that exclusively employs organized edge features, achieving efficient tracking and precise local mapping. Extensive experiments demonstrate its accuracy and robustness in indoor environments, outperforming or achieving comparable performance to state-of-the-art methods.

Robust and accurate pose estimation in unknown environments is an essential part of robotic applications. We focus on LiDAR-based point-to-point ICP combined with effective semantic information. This paper proposes a novel semantic information-assisted ICP method named SAGE-ICP, which leverages semantics in odometry. The semantic information for the whole scan is timely and efficiently extracted by a 3D convolution network, and these point-wise labels are deeply involved in every part of the registration, including semantic voxel downsampling, data association, adaptive local map, and dynamic vehicle removal. Unlike previous semantic-aided approaches, the proposed method can improve localization accuracy in large-scale scenes even if the semantic information has certain errors. Experimental evaluations on KITTI and KITTI-360 show that our method outperforms the baseline methods, and improves accuracy while maintaining real-time performance, i.e., runs faster than the sensor frame rate.

Sparse keypoint matching based on distinct 3D feature representations can improve the efficiency and robustness of point cloud registration. Existing learning-based 3D descriptors and keypoint detectors are either independent or loosely coupled, so they cannot fully adapt to each other. In this work, we propose a tightly coupled keypoint detector and descriptor (TCKDD) based on a multi-task fully convolutional network with a probabilistic detection loss. In particular, this self-supervised detection loss fully adapts the keypoint detector to any jointly learned descriptors and benefits the self-supervised learning of descriptors. Extensive experiments on both indoor and outdoor datasets show that our TCKDD achieves state-of- the-art performance in point cloud registration. Furthermore, we design a keypoint detector and descriptors-assisted LiDAR odometry and mapping framework (KDD-LOAM), whose real-time odometry relies on keypoint descriptor matching-based RANSAC. The sparse keypoints are further used for efficient scan-to-map registration and mapping. Experiments on KITTI dataset demonstrate that KDD-LOAM significantly surpasses LOAM and shows competitive performance in odometry.

Existing methods for place recognition in 3D point clouds either ignore partial structure information by converting 3D scans to 2D images or construct constrained bag-of-words (BoW) representations reliant on specific feature extraction algorithms. In this paper, we propose a novel method based on incremental and general bag of words. Incorporating an adaptable keypoint and 3D local feature extraction method, we employ an incremental BoW model that is updated regularly. This enables a coarse-to-fine candidate selection from the database. And a revisit can be identified following geometric verification. In addition, we propose a new supplementary metric that addresses the leaving-out issue of the conventional metric, enhancing the identification of true loops. Employing a state-of-the-art (SOTA) keypoint and feature extraction algorithm, we evaluate our method as well as SOTA place recognition methods using diverse datasets with varying qualities. Experimental results demonstrate that our method outperforms the baselines across all three datasets, showcasing robust performance and notable generalization capabilities.

Precise and long-term stable localization is essential in parking lots for tasks like autonomous driving or autonomous valet parking, etc. Existing methods rely on a fixed and memory-inefficient map, which lacks robust data association approaches. And it is not suitable for precise localization or long-term map maintenance. In this paper, we propose a novel mapping, localization, and map update system based on ground semantic features, utilizing low-cost cameras. We present a precise and lightweight parameterization method to establish improved data association and achieve accurate localization at centimeter-level. Furthermore, we propose a novel map update approach by implementing high-quality data association for parameterized semantic features, allowing continuous map update and refinement during re-localization, while maintaining centimeter-level accuracy. We validate the performance of the proposed method in real-world experiments and compare it against state-of-the-art algorithms. The proposed method achieves an average accuracy improvement of 5cm during the registration process. The generated maps consume only a compact size of 450 KB/km and remain adaptable to evolving environments through continuous update.

Point set registration, crucial in computer vision and robotics applications, encounters challenges, such as noise, outliers, and misalignment. Current methods often struggle with these issues, leading to suboptimal registration accuracy. This article proposes a novel graph correspondence-based algorithm to address these challenges in rigid point set registration. We model point sets as graphs, transforming the registration problem into a graph isomorphism problem. This approach is enhanced with probabilistic linear programming heuristics to efficiently establish correspondences between point sets. Our method significantly improves robustness against common registration errors and does not require initial pose estimation, a notable advantage over existing algorithms. Extensive experiments on various datasets, including applications in intelligent vehicle mapping and localization, demonstrate superior performance in correspondence establishment and registration accuracy compared to state-of-the-art methods, particularly under conditions of noise, outliers, and misalignment.

We propose an endoscopic image mosaicking algorithm that is robust to light conditioning changes, specular reflections, and feature-less scenes. These conditions are especially common in minimally invasive surgery where the light source moves with the camera to dynamically illuminate close range scenes. This makes it difficult for a single image registration method to robustly track camera motion and then generate consistent mosaics of the expanded surgical scene across different and heterogeneous environments. Instead of relying on one specialised feature extractor or image registration method, we propose to fuse different image registration algorithms according to their uncertainties, formulating the problem as affine pose graph optimisation. This allows to combine landmarks, dense intensity registration, and learning-based approaches in a single framework. To demonstrate our application we consider deep learning-based optical flow, hand-crafted features, and intensity-based registration, however, the framework is general and could take as input other sources of motion estimation, including other sensor modalities. We validate the performance of our approach on three datasets with very different characteristics to highlighting its generalisability, demonstrating the advantages of our proposed fusion framework. While each individual registration algorithm eventually fails drastically on certain surgical scenes, the fusion approach flexibly determines which algorithms to use and in which proportion to more robustly obtain consistent mosaics.

Localization in the complex urban environment is an open problem for current methods. The occlusion from dynamic objects, such as vehicles and pedestrians, degenerates the precision of the localization result. This article proposes a pole-like feature-based localization framework to solve this problem. Pole-like objects, such as posts of lamps or traffic sign and tree trunks, widely exist in the urban environment and are robust to occlusion, as they are usually higher than the objects on the road. First, this type of feature is extracted from the point cloud by a robust clustering algorithm. Then, the features from different frames of data are stitched to generate a feature map. For online localization, a Monte Carlo localization (MCL) framework is used to fuse the vehicle motion data and the map-matching result. An improved version of iterative closest point (ICP) that is specifically designed for the pole-like feature association is used for map matching based on the state of every particle. With the MCL scheme, localization is robust to the local minimum or robot kidnapping problem. Experimental results in the real urban environment demonstrate the precision and robustness of the proposed method, with mean absolute errors less than 0.20 m and 0.5°. The results also show that the proposed method outperforms some state-of-the-art localization methods in the complex urban environment.Note to Practitioners—There are some works using features from the 3-D point cloud, e.g., corners, planes, and reflectance, for robot localization. Instead of the abstract features, this article presents an object-feature-based localization scheme. We propose a novel pole-like object extraction algorithm based on the spatial distribution of the 3-D points. This algorithm can extract most types of pole-like objects in the urban environment. As these objects are highly distinct from other types of objects in their surroundings, localization is achieved by associating the pole-like features in the map and the features detected in real time through maximizing the likelihood. The whole system is verified with data collected in the real world, which indicates that its accuracy can fulfill the requirements of autonomous driving. The limitation of the proposed method is that it highly depends on one specific type of feature, which may not work well in the rural environment. In future research, we will address this problem by incorporating more types of semantic features for localization.

Point cloud registration is widely used in the research of intelligent vehicles, typical problems include map matching, visual odometer, pose estimation,etc. This paper proposes a deep learning-based registration method that can input point clouds directly, thereby preventing information loss of preprocessing needed by alternative deep-learning approaches. Our network, named DPFNet (Direct Point Feature Net), gradually downsamples the point cloud and aggregates points around determined reference points to formulate local features automatically. This is facilitated by a novel convolution-like operator and a novel loss function. The points in the point cloud are mapped to a high dimensional embedding through the designed deep neural network, where every embedding reflects the local feature of a specific spatial area. Based on the embedding features, correspondences between points can be estimated robustly and the registration between the point clouds can be obtained using an external geometric optimization algorithm. Experimental results on open benchmarks validate the proposed method and show that its performance is favourable over several baseline methods. Specifically, we test the proposed algorithm on KITTI benchmark, which shows its potential in tasks of intelligent vehicles,e.g., map matching, visual or LiDAR odometer.

This article presents a pose fusion method that accounts for the possible correlations among measurements. The proposed method can handle data fusion problems whose uncertainty has both independent and dependent parts. Different from the existing methods, the uncertainties of the various states or measurements are modeled on the Lie algebra and projected to the manifold through the exponential map, which is more precise than that modeled in the vector space. The correlation is based on the theory of covariance intersection, where the independent and dependent parts are split to yield a more consistent result. In this article, we provide a novel method for the correlated pose fusion algorithm on the manifold. Theoretical derivation and analysis are detailed first, and then, the experimental results are presented to support the proposed theory. The main contributions are threefold: First, we provide a theoretical foundation for the split covariance intersection filter performed on the manifold, where the uncertainty is associated with the Lie algebra. Second, the proposed method gives an explicit fusion formalism on$ \text{SE}(3)$and$ \text{SE}(2)$, which covers the most use cases in the field of robotics. Third, we present a localization framework that can work for both single-robot and multirobot systems, where not only the fusion with possible correlation is derived on the manifold but also the state evolution and relative pose computation are performed on the manifold. The experimental results validate the advantage of this approach over state-of-the-art methods.

Point set registration is a problem with a long history in many pattern recognition tasks. This paper presents a robust point set registration algorithm based on optimizing the distance between two probability distributions. A major problem in point to point algorithms is defining the correspondence between two point sets. This paper follows the idea of some probability-based point set registration methods by representing the point sets as Gaussian mixture models (GMMs). By optimizing the distance between the two GMMs, rigid transformations (rotation and translation) between two point sets can be obtained without having to find a correspondence. Previous studies have used L2, Kullback Leibler, etc. distance to measure similarity between two GMMs; however, these methods have problems with robustness to noise and outliers, especially when the covariance matrix is large, or a local minimum exists. Therefore, in this paper, the signature quadratic form distance is derived to measure the distribution similarity. The contribution of this paper lies in adopting the signature quadratic form distance for the point set registration algorithm. The experimental results show the precision and robustness of this algorithm and demonstrate that it outperforms other state-of-the-art point set registration algorithms regarding factors, such as noise, outliers, missing partial structures, and initial misalignment.

This paper presents a precise and robust localization framework for autonomous vehicles. In contrast to simultaneous localization and mapping, localization and mapping in our method are separate (i.e., first mapping and then localization within this map). The map used in this paper is a 3-D occupancy map that is generated from a stitched point cloud. For localization, a hybrid filtering framework is proposed to match the live data with the prior map. In the upper layer, odometer data, IMU data, and the map matching result are fused by a cubature Kalman filter, which will limit the predictive pose within a reasonable bound. In the lower layer, the map matching problem is converted into a point set registration problem that is solved using a particle filter, which will make the matching result robust to local minima. This hybrid scheme makes localization more robust to convergence to a local minimum, which is often encountered in the localization task for autonomous vehicles. This method can also guarantee decimeter-level precision in industrial environments. Experiments demonstrate the validity of this method and also show that it outperforms some state-of-the-art methods.

Point set registration is a basic but still open problem in numerous computer vision tasks. In general, there are more than one type of error sources for registration, for example, noise, outliers and false initialization may exist simultaneously. These errors could influence the registration independently and dependently. Previous works usually test performance under one of the two types of errors at one time, or they do not perform well under some extreme situations with both of the error sources. This work presents a robust point set registration algorithm under a filtering framework, which aims to be robust under various types of errors simultaneously. The point set registration problem can be cast into a non-linear state space model. We use a split covariance intersection filter (SCIF) to capture the correlation between the state transition and the observation (moving point set). The two above-mentioned types of errors can be represented as dependent and independent parts in the SCIF. The covariance of the two types of errors will be updated every iteration. Meanwhile, the non-linearity of the observation model is approximated by a cubature transformation. First, the recursive cubature split covariance intersection filter is derived based on the non-linear state space model. Then, we use this algorithm to solve the point set registration problem. This algorithm can approximate non-linearity by a third-order term and consider correlations between the process model and the observation model. Compared to other filtering-based methods, this algorithm is more robust and precise. Tests on both public datasets and experiments validate the precision and robustness of this algorithm to outliers and noise. Comparison experiments show that this algorithm outperforms state-of-the-art point set registration algorithms in certain respects.

Point set registration is a key problem in intelligent vehicle localization and mapping. This paper presents a rigid point set registration algorithm based on the cubature Kalman filter (CKF). First, the point set registration problem is cast into the state space model assuming that the correspondence between these two point sets is previously unknown. Then, CKF is used to solve this nonlinear filtering problem. At every iterative step, all the points in the moving point set will be considered, and the corresponding points in the model point set will be updated accordingly. In the time update, the scale of the free space that can be explored is significant for this registration algorithm. Herein, continuous simulated annealing (CSA) is adopted to gradually optimize the covariance of the model noise to speed up convergence. Tests on public data sets show that the CKF-based point set registration algorithm is robust to outliers, noise, and initialization misalignment. Then, the application of this algorithm on intelligent vehicles and intelligent transportation systems is demonstrated through mapping and localization experiments. The precision and robustness are both validated compared with the traditional ICP, NDT, and CPD based point set registrations in the localization experiments. Thus, the contributions of this paper are threefold. First, a CKF scheme is utilized in the point set registration problem. Second, CSA serves as the local optimizer to speed up the convergence process and make it more accurate. Third, this filtering-based method is adopted in intelligent vehicles and SLAM applications.

Point set registration is a long addressed problem in lots of pattern recognition tasks. This paper presents a robust point set registration algorithm based on optimization of distance between two probability distributions. A major problem encountered in the point to point algorithms is the definition of correspondence between two point sets. This paper follows the idea of some probability based point set registration methods and the point set is represented as Gaussian Mixture Models (GMMs). Through optimizing distance between the two GMMs, the rigid transformation (rotation and translation) between two point sets will be obtained while averting the trouble of finding correspondence. Previous studies used L2 distance, KL distance, etc. to measure similarity between two GMMs, the problem therein is the robustness to noise and outliers, especially when the covariance matrix is large or there exists local minimum. So in this work, the signature quadratic form distance is derived for the distribution similarity measurement. The contribution of this paper is as follows. First, we derive the signature quadratic form distance for GMMs similarity measurement. Second, the signature quadratic form distance is adopted to the point set registration algorithm. And performance of the proposed method compared with some existing widely used point set registration algorithms is also presented. Experimental results show precision and robustness of this algorithm. The results also demonstrate this algorithm outperforms some state-of-the-art point set registration algorithms in terms of noise, outliers, partial structures and misalignment initialization, etc.

High-precision and reliable localization is current research focus in the area of autonomous vehicles. Previous studies rely on either high-cost sensors or some specific characteristics, which means that the methods are limited to only a bit given situations. In this paper, a road DNA based localization method is proposed. It could afford high-precision result and does not have the shortcomings of previous methods at the same time. The scenery on both sides of the roads are used to generate the prior-map. The map is presented as grid map by the joint probability of occupation and reflectivity. With this type of map, different environments show different properties, which means that this method is not limited to specific environments and is effective in most cases. It costs much less memory than the previous maps. The map and live road scene flatting are both generated by data collected by low-cost LIDAR. Normalized Information Distance is utilized to align the live road scene flatting with the road DNA. Experiments show the validation and precision of this method.