We investigate a novel way to integrate visual SLAM and lidar SLAM. Instead of enhancing visual odometry via lidar depths or using visual odometry as the motion initial guess of lidar odometry, we propose tightly-coupled visual-lidar SLAM (TVL-SLAM), in which the visual and lidar frontend are run independently and which incorporates all of the visual and lidar measurements in the backend optimizations. To achieve large-scale bundle adjustments in TVL-SLAM, we focus on accurate and efficient lidar residual compression. The visual-lidar SLAM system implemented in this work is based on the open-source ORB-SLAM2 and a lidar SLAM method with average performance, whereas the resulting visual-lidar SLAM clearly outperforms existing visual/lidar SLAM approaches, achieving 0.52% error on KITTI training sequences and 0.56% error on testing sequences.

We propose a new way of fusing visual and lidar measurements for an effective odometry. When to combine visual measurements with lidar measurements, it is typical to assign the depth information from lidar measurements to seemingly corresponding, visual features or to utilize the results of visual odometry as the motion initial guess for a lidar odometry. Instead, our method 1) builds two maps in different modalities: a lidar voxel map and a visual map with map-points and 2) use them together when to solve the residuals for an odometry. And the results of the odometry residual optimization are used to keep the lidar voxel map to be globally consistent with the operating environment. By using visual-lidar measurements in such a tightly-coupled way, our method estimates the odometry more accurately as incorporating more geometric constraints into the odometry residuals, and removes any chances of errors in assigning the depths of lidars to non-corresponding visual features. Experimental results showed that our method outperforms the visual (i.e., the front-end of the ORB-SLAM2), and lidar odometry, and a depth-enhanced visual odometry.

A local bundle adjustment is an important proce- dure to improve the accuracy of a visual odometry solution. However, it is computationally very expensive as it jointly optimize all the poses of cameras and locations of map points. To reduce the computational complexity of a local bundle adjustment, the state-of-the-art algorithms [1]–[3] were proposed to manipulate the map point variables, using extra matrix operations, from their linearized optimization solutions. Instead of relying on complex matrix manipulations, this paper proposes a novel way of addressing this complexity issue-we represent a map point as a function of two camera poses, and uses the triangulated location of the map point when needed. Our method is more efficient than ones in the full-SLAM formulation in solving the visual odometry problem in that 1) the complexity of our solution is lower than those of the state-of-the-art methods, 2) no extra matrix operations required to eliminate map points, 3) no need guesses on map points' initial locations. Experiemental results, through simulated experiments and experiments with the KITTI dataset, demonstrated that our results are more accurate than those of a full-SLAM approach with lower runtime complexities.

This paper presents a complete perception system of moving point detection and target tracking for robustly following target humans in an unknown indoor dynamic environment. To detect moving points under grid-based formulation, a modified inverse observation model is proposed to overcome several frequently happened detection limitations. Next, related human-extraction techniques are proposed to filter out less possible clusters for detecting potential human target from these moving points. Finally, the multiple hypothesis tracking algorithm is implemented to deal with the data association problem for enhancing the reliability and robustness of the human tracking when measurements are noisy. Different levels of experiments have been performed to evaluate the effectiveness of the proposed algorithm framework.

This work aims to accurately match two scene images under large viewpoint changes, which is the key issue in appearance-based localization tasks. In this paper, two key ideas are proposed to solve the challenging problem. First, to detect extreme small overlapping regions between two images, a new approach is developed to estimate the camera motion using only two pairs of matched features, while the state-of-art needs at least five. Second, proper prior knowledge to the environmental structures is utilized to strengthen the outlier rejection. The proposed 2-point approach is tested on challenging scenes and shows good robustness to the drastic occlusion and scaling caused by viewpoint changes.