Accurate localization is essential for enabling modern full self-driving services. These services heavily rely on map-based traffic information to reduce uncertainties in recognizing lane shapes, traffic light locations, and traffic signs. Achieving this level of reliance on map information requires centimeter-level localization accuracy, which is currently only achievable with LiDAR sensors. However, LiDAR is known to be vulnerable to spoofing attacks that emit malicious lasers against LiDAR to overwrite its measurements. Once localization is compromised, the attack could lead the victim off roads or make them ignore traffic lights. Motivated by these serious safety implications, we design SLAMSpoof, the first practical LiDAR spoofing attack on localization systems for self-driving to assess the actual attack significance on autonomous vehicles. SLAMSpoof can effectively find the effective attack location based on our scan matching vulnerability score (SMVS), a point-wise metric representing the potential vulnerability to spoofing attacks. To evaluate the effectiveness of the attack, we conduct real-world experiments on ground vehicles and confirm its high capability in real-world scenarios, inducing position errors of$\geq 4.2$meters (more than typical lane width) for all 3 popular LiDAR-based localization algorithms. We finally discuss the potential countermeasures of this attack. Code is available at https://github.com/Keio-CSG/slamspoof.

This paper presents range-based 6-DoF Monte Carlo SLAM with a gradient-guided particle update strategy. While non-parametric state estimation methods, such as particle filters, are robust in situations with high ambiguity, they are known to be unsuitable for high-dimensional problems due to the curse of dimensionality. To address this issue, we propose a particle update strategy that improves the sampling efficiency by using the gradient information of the likelihood function to guide particles toward its mode. Additionally, we introduce a keyframe-based map representation that represents the global map as a set of past frames (i.e., keyframes) to mitigate memory consumption. The keyframe poses for each particle are corrected using a simple loop closure method to maintain trajectory consistency. The combination of gradient information and keyframe-based map representation significantly enhances sampling efficiency and reduces memory usage compared to traditional RBPF approaches. To process a large number of particles (e.g., 100,000 particles) in real-time, the proposed framework is designed to fully exploit GPU parallel processing. Experimental results demonstrate that the proposed method exhibits extreme robustness to state ambiguity and can even deal with kidnapping situations, such as when the sensor moves to different floors via an elevator, with minimal heuristics.

In this paper, we propose a non-parametric method for estimating the posterior distribution of global positioning satellite systems (GNSS) integer ambiguity. It is difficult to estimate the posterior probability of discrete integer ambiguities directly from carrier phase observations due to the unclear domain definition. We thus introduce a positional likelihood field that accumulates the ambiguity function method values in the position space and then estimate the integer ambiguity distributions by marginalizing the likelihood over the entire position. Defining the positional likelihood field in the position space facilitates carrier phase likelihood accumulation. To correctly estimate the posterior distribution, however, a sufficient density of samples is required, which results in a large computational cost. The proposed method enables large-scale sampling by taking advantage of GPU parallel processing. Experimental results demonstrate that the proposed method enables accurate and robust estimation of integer ambiguity distributions, contributing to improved centimeter-level position estimation accuracy. In addition, the histograms provide quantitative evidence of events in urban environments where integer ambiguity is not uniquely determined.

This paper presents an accurate and fast 3D global localization method, 3D-BBS, that extends the existing branchand-bound (BnB)-based 2D scan matching (BBS) algorithm. To reduce memory consumption, we utilize a sparse hash table for storing hierarchical 3D voxel maps. To improve the processing cost of BBS in 3D space, we propose an efficient roto-translational space branching. Furthermore, we devise a batched BnB algorithm to fully leverage GPU parallel processing. Through experiments in simulated and real environments, we demonstrated that the 3D-BBS enabled accurate global localization with only a 3D LiDAR scan roughly aligned in the gravity direction and a 3D pre-built map. This method required only 878 msec on average to perform global localization and outperformed state-of-the-art global registration methods in terms of accuracy and processing speed.

This paper presents a 6-DoF range-based Monte Carlo localization method with a GPU-accelerated Stein particle filter. To update a massive amount of particles, we propose a Gauss-Newton-based Stein variational gradient descent (SVGD) with iterative neighbor particle search. This method uses SVGD to collectively update particle states with gradient and neighborhood information, which provides efficient particle sampling. For an efficient neighbor particle search, it uses locality sensitive hashing and iteratively updates the neighbor list of each particle over time. The neighbor list is then used to propagate the posterior probabilities of particles over the neighbor particle graph. The proposed method is capable of evaluating one million particles in real-time on a single GPU and enables robust pose initialization and re-localization without an initial pose estimate. In experiments, the proposed method showed an extreme robustness to complete sensor occlusion (i.e., kidnapping), and enabled pinpoint sensor localization without any prior information.

This paper presents a range inertial localization algorithm for a 3D prior map. The proposed algorithm tightly couples scan-to-scan and scan-to-map point cloud registration factors along with IMU factors on a sliding window factor graph. The tight coupling of the scan-to-scan and scan-to-map registration factors enables a smooth fusion of sensor ego-motion estimation and map-based trajectory correction that results in robust tracking of the sensor pose under severe point cloud degeneration and defective regions in a map. We also propose an initial sensor state estimation algorithm that robustly estimates the gravity direction and IMU state and helps perform global localization in 3- or 4-DoF for system initialization without prior position information. Experimental results show that the proposed method outperforms existing state-of-the-art methods in extremely severe situations where the point cloud data becomes degenerate, there are momentary sensor interruptions, or the sensor moves along the map boundary or into unmapped regions.

This paper presents an open source LiDAR-camera calibration toolbox that is general to LiDAR and cam-era projection models, requires only one pairing of LiDAR and camera data without a calibration target, and is fully automatic. For automatic initial guess estimation, we employ the Super-Glue image matching pipeline to find 2D-3D correspondences between LiDAR and camera data and estimate the LiDAR-camera transformation via RANSAC. Given the initial guess, we refine the transformation estimate with direct LiDAR-camera registration based on the normalized information distance, a mutual information-based cross-modal distance metric. For a handy calibration process, we also present several assistance capabilities (e.g., dynamic LiDAR data integration and user interface for making 2D-3D correspondence manually). The experimental results show that the proposed toolbox enables calibration of any combination of spinning and non-repetitive scan LiDARs and pinhole and omnidirectional cameras, and shows better calibration accuracy and robustness than those of the state-of-the-art edge-alignment-based calibration method.

This study presents a framework, L-C*, for resilient and lightweight direct visual localization, employing a loosely coupled fusion of visual and inertial data. Unlike indirect methods, direct visual localization facilitates accurate pose estimation on general color three-dimensional maps that are not tailored for visual localization. However, it suffers from temporal localization failures and high computational costs for real-time applications. For long-term and real-time visual localization, we developed an L-C* that incorporates direct visual localization C* in a visual-inertial loose coupling. By capturing ego-motion via visual-inertial odometry to interpolate global pose estimates, the framework allows for a significant reduction in the frequency of demanding global localization, thereby facilitating lightweight but reliable visual localization. In addition, forming a closed loop that feeds the latest pose estimate to the visual localization component as an initial guess for the next pose inference renders the system highly robust. A quantitative evaluation of a simulation dataset demonstrated the accuracy and efficiency of the proposed framework. Experiments using smartphone sensors also demonstrated the robustness and resiliency of L-C* in real-world situations.

This paper presents a point cloud downsampling algorithm for fast and accurate trajectory optimization based on global registration error minimization. The proposed algorithm selects a weighted subset of residuals of the input point cloud such that the subset yields exactly the same quadratic point cloud registration error function as that of the original point cloud at the evaluation point. This method accurately approximates the original registration error function with only a small subset of input points (29 residuals at a minimum). Experimental results using the KITTI dataset demonstrate that the proposed algorithm significantly reduces processing time (by 87%) and memory consumption (by 99%) for global registration error minimization while retaining accuracy.

This paper presents a real-time 3D mapping framework based on global matching cost minimization and LiDAR-IMU tight coupling. The proposed framework comprises a preprocessing module and three estimation modules: odometry estimation, local mapping, and global mapping, which are all based on the tight coupling of the GPU-accelerated voxelized GICP matching cost factor and the IMU preintegration factor. The odometry estimation module employs a keyframe-based fixed-lag smoothing approach for efficient and low-drift trajectory estimation, with a bounded computation cost. The global mapping module constructs a factor graph that minimizes the global registration error over the entire map with the support of IMU constraints, ensuring robust optimization in feature-less environments. The evaluation results on the Newer College dataset and KAIST urban dataset show that the proposed framework enables accurate and robust localization and mapping in challenging environments.

This paper presents an accurate and scalable method for fiducial tag localization on a 3D prior environmental map. The proposed method comprises three steps: 1) visual odometry-based landmark SLAM for estimating the relative poses between fiducial tags, 2) geometrical matching-based global tag-map registration via maximum clique finding, and 3) tag pose refinement based on direct camera-map alignment with normalized information distance. Through simulation-based evaluations, the proposed method achieved a 98 % global tag-map registration success rate and an average tag pose estimation accuracy of a few centimeters. Experimental results in a real environment demonstrated that it enables to localize over 110 fiducial tags placed in an environment in 25 minutes for data recording and post-processing.

LiDAR odometry algorithms are complex and involve a number of hyper-parameters. The choice of hyper-parameters can substantively affect the performance of odometry estimation, and it is necessary to carefully fine-tune the hyper-parameters depending on the sensor, environment, and algorithm to achieve the best estimation results. While odometry estimation algorithms are often tuned manually, this is time-consuming and may also result in a sub-optimal parameter set. This paper presents an automatic hyper-parameter tuning approach for LiDAR odometry estimation. By taking advantage of the sequential model-based optimization (SMBO) approach, we automatically optimize the hyper-parameter set of a black-box odometry estimation algorithm without detailed knowledge of the algorithm. In addition, a LiDAR data augmentation approach is also proposed to prevent overfitting. Through evaluation, we show that the combination of SMBO-based parameter exploration and data augmentation enables us to efficiently and robustly optimize the hyper-parameter set for several different odometry estimation algorithms. We also demonstrate that the optimized parameter set exhibits superior performance with respect to KITTI dataset and in a real use scenario.

This paper presents the voxelized generalized iterative closest point (VGICP) algorithm for fast and accurate three-dimensional point cloud registration. The proposed approach extends the generalized iterative closest point (GICP) approach with voxelization to avoid costly nearest neighbor search while retaining its accuracy. In contrast to the normal distributions transform (NDT), which calculates voxel distributions from point positions, we estimate voxel distributions by aggregating the distribution of each point in the voxel. The voxelization approach allows us to efficiently process the optimization in parallel, and the proposed algorithm can run at 30 Hz on a CPU and 120 Hz on a GPU. Through evaluations in simulated and real environments, we confirmed that the accuracy of the proposed algorithm is comparable to GICP, but is substantially faster than existing methods. This will enable the development of real-time 3D LIDAR applications that require extremely fast evaluations of the relative poses between LIDAR frames.

In this paper, a three-dimensional light detection and ranging simultaneous localization and mapping (SLAM) method is proposed that is available for tracking and mapping with 500–1000 Hz processing. The proposed method significantly reduces the number of points used for point cloud registration using a novel ICP metric to speed up the registration process while maintaining accuracy. Point cloud registration with ICP is less accurate when the number of points is reduced because ICP basically minimizes the distance between points. To avoid this problem, symmetric KL-divergence is introduced to the ICP cost that reflects the difference between two probabilistic distributions. The cost includes not only the distance between points but also differences between distribution shapes. The experimental results on the KITTI dataset indicate that the proposed method has high computational efficiency, strongly outperforms other methods, and has similar accuracy to the state-of-the-art SLAM method.

This study proposes an adaptive data-driven hy-perparameter tuning framework for black-box 3D LiDAR odometry algorithms. The proposed framework comprises offline parameter-error function modeling and online adaptive parameter selection. In the offline step, we run the odometry estimation algorithm for tuning with different parameters and environments and evaluate the accuracy of the estimated trajectories to build a surrogate function that predicts the trajectory estimation error for the given parameters and environments. Subsequently, we select the parameter set that is expected to result in good accuracy in the given environment based on trajectory error prediction with the surrogate function. The proposed framework does not require detailed information on the inner working of the algorithm to be tuned, and improves its accuracy by adaptively optimizing the parameter set. We first demonstrate the role of the proposed framework in improving the accuracy of odometry estimation across different environments with a simulation-based toy example. Further, an evaluation on the public dataset KITTI shows that the proposed framework can improve the accuracy of several odometry estimation algorithms in practical situations.

Autonomous driving of a personal mobility vehicle such as a wheelchair in a walking space is crucial as a means of transportation for the elderly and the physically handicapped. To realize this, accurate pedestrian detection is indispensable. As existing 3D object detection methods are trained with a roadway dataset, they are widely used for object detection in roadways. These methods have two major drawbacks as regards the detection of objects in walking spaces. The first is that they largely depend on the different LIDAR models. To eliminate this issue, we propose a 3D object detection method, CosPointPillars, that does not take the reflection intensities of the LIDAR point cloud, which causes a sensor model dependency, as input. The second drawback is that networks trained with a roadway dataset cannot sufficiently detect pedestrians (who are major traffic participants in walking spaces) located within a short distance; this is because the roadway dataset hardly includes nearby pedestrians. To solve this issue, we generated a new walking space dataset called SimDataset, which includes nearby pedestrians as a training dataset in the simulations. An experiment on a real walking space showed that SimDataset is suitable for use in pedestrian detection.

This paper describes a calibration method for RGB-D camera networks consisting of not only static overlapping, but also dynamic and non-overlapping cameras. The proposed method consists of two steps: online visual odometry-based calibration and depth image-based calibration refinement. It first estimates the transformations between overlapping cameras using fiducial tags, and bridges non-overlapping camera views through visual odometry that runs on a dynamic monocular camera. Parameters such as poses of the static cameras and tags, as well as dynamic camera trajectory, are estimated in the form of the pose graph-based online landmark SLAM. Then, depth-based ICP and floor constraints are added to the pose graph to compensate for the visual odometry error and refine the calibration result. The proposed method is validated through evaluation in simulated and real environments, and a person tracking experiment is conducted to demonstrate the data integration of static and dynamic cameras.

With the aim of contributing to the development of a robotic attendant system, this study proposes the concept of assessing the risk of collision using awareness estimation. The proposed approach enables an attendant robot to assess a person's risk of colliding with an obstacle by estimating whether he/she is aware of it based on behavior, and to take the requisite preventative action. To implement the proposed concept, we design a model that can simultaneously estimate a person's awareness of obstacles and predict his/her trajectory based on a convolutional neural network. When trained on a dataset of collision-related behaviors generated from people trajectory datasets, the model can detect objects of which the person is not aware and with which he/she at risk of colliding. The proposed method was evaluated in an empirical environment, and the results verified its effectiveness.

This paper proposes a 3D LiDAR simultaneous localization and mapping (SLAM) method that improves accuracy, robustness, and computational efficiency for an iterative closest point (ICP) algorithm employing a locally approximated geometry with clusters of normal distributions. In comparison with previous normal distribution-based ICP methods, such as normal distribution transformation and generalized ICP, our ICP method is simply stabilized with normalization of the cost function by the Frobenius norm and a regularized covariance matrix. The previous methods are stabilized with principal component analysis, whose computational cost is higher than that of our method. Moreover, our SLAM method can reduce the effect of incorrect loop closure constraints. The experimental results show that our SLAM method has advantages over open source state-of-the-art methods, including LOAM, LeGO-LOAM, and hdl_graph_slam.

This paper describes an estimation method of a person's awareness of an obstacle. We assume that the person's awareness influences the person's motion, and construct a model of the relationship between the awareness and the motion using HCRF. We extract a sequence of motion features from the person trajectory, and then classify whether the person is aware of the obstacle or not using the model. Awareness estimation experiments are conducted in order to validate the method and evaluate its performance. Since the method uses only the position and the velocity of the person, it can be applicable to mobile robots.

This paper describes a person identification method using a smartphone and laser range finders (LRFs) for a mobile service robot. The robot is equipped with LRFs and the target person holds a smartphone. The method first detects the foot strike timings of the target person using the smartphone and those of all people by using the LRFs. By finding the person whose foot strike timings captured by the LRFs are similar to those obtained by the smartphone, the robot can identify the target person. Person identification experiments and person following experiments are conducted in order to validate the method. Since the method only requires a person to simply hold a smartphone, it can be easily applied to daily situations.