The scene flow estimation methods make significant progress by estimating pixel-wise 3D motion on implicitly learning a motion embedding using an end-to-end differentiable optimization framework. However, the motion embedding learned implicitly is insufficient for grouping pixels into rigid object in challenging regions, such as occlusion and inconsistent multi-view geometric properties. To address this issue, we propose a novel method for estimating scene flow called OAMaskFlow, which has three novelties. Firstly, we propose the concept of occlusion-aware motion (OAM) mask and generate the ground truth annotation through the photo-metric and geometry consistency. Secondly, we propose to supervise the motion embedding with the OAM mask to learn informative and reliable motion representation of the scene. Finally, a 3D motion propagation module is proposed to propagate high-quality 3D motion from reliable pixels to the challenging occluded regions. Experiments show that our proposed OAMaskFlow has reduced the EPE3D metric by 21.0% on the FlyingThings3D dataset and decreased SF-all metric by 24.3% on the KITTI scene flow benchmark than the baseline method RAFT-3D. Furthermore, we apply our proposed OAM mask in simultaneous localization and mapping (SLAM) to improve a state-of-the-art method DROID-SLAM. In comparison, the ATE metric has decreased by 65.7% and 58.3% on the TartanAir monocular and stereo datasets respectively.

Recent deep learning based visual simultaneous localization and mapping (SLAM) methods have made significant progress. However, how to make full use of visual information as well as better integrate with inertial measurement unit (IMU) in visual SLAM has potential research value. This paper proposes a novel deep SLAM network with dual visual factors. The basic idea is to integrate both photometric factor and re-projection factor into the end-to-end differentiable structure through multi-factor data association module. We show that the proposed network dynamically learns and adjusts the confidence maps of both visual factors and it can be further extended to include the IMU factors as well. Extensive experiments validate that our proposed method significantly outperforms the state-of-the-art methods on several public datasets, including TartanAir, EuRoC and ETH3D-SLAM. Specifically, when dynamically fusing the three factors together, the absolute trajectory error for both monocular and stereo configurations on EuRoC dataset has reduced by 45.3% and 36.2% respectively.

A loop closure module plays an important role in visual SLAM systems, which can reduce the accumulat-ed drift. This task faces the challenges of large viewpoint changes and expensive computational costs when optimizing the global map. This paper proposes DH-LC, a novel accurate and robust loop closure method that consists of hierarchical spatial feature matching (HSFM) and hybrid bundle adjustment (HBA). HSFM estimates a reliable relative pose between the query image and the retrieval image in a coarse-to-fine way. Specifically, 3D points are firstly triangulated and then clus-tered according to the spatial distribution. The cluster centers estimate coarse cube-level matching pairs in a larger perception field which can tolerate large viewpoint changes. HBA optimizes the global map efficiently by adaptively selecting incremental bundle adjustment or full bundle adjustment according to the accumulated drift and relative pose verification in the temporal window. Experimental results demonstrate that our proposed method easily detects loops in large viewpoint changes and efficiently optimizes the global map. When compared with the state-of-the-art methods, our method increases loop closure recall and improves SLAM localization accuracy with reducing the accumulated drift.

Most of the state-of-the-art visual inertial SLAM methods pay less attention to 2D-2D and 3D-2D matching with more reliable features in a long time span, which easily results in continuous estimation drift. In this paper, we propose an efficient drift-free visual-inertial SLAM method by a pose guided feature matching method to re-identify existing features from a spatial-temporal sensitive sub-global map. The re-identified features serve as augmented visual measurements to anchor the current frame and gradually decrease the accumulated error in the long run. When incorporating the measurements into the optimization module, it benefits to build a drift-free global map in the system. Extensive experiments show that our feature re-identification method is both effective and efficient. Specifically, when combining the feature re-identification with the state-of-the-art SLAM method [1], our method achieves 67.3% and 87.5% absolute trajectory error reduction with only a small additional computational cost on two public SLAM benchmark DBs: EuRoC and TUM-VI respectively.

Most of the state-of-the-art visual-inertial SLAM methods pay less attention to the scene structure of man-made environments. In this paper, based on the assumption of multiple local Manhattan worlds (MWs), we propose a Manhattan frame (MF) re-identification method to build relative rotation constraints between MF matching pairs and tightly couple these constraints into global bundle adjust module. Specifically, a coarse-to-fine vanishing point (VP) estimation method and pose guided MF temporal consistency verification method are firstly proposed to improve the accuracy and robustness of MF estimation. Then unreliable MF matching pairs are filtered out by a spatial temporal consistency check. Finally, the relative rotation constraints of the remaining MF matching pairs are combined into global bundle adjustment energy function for further optimization. We have validated our proposed method on both synthetic and real-world datasets. When comparing with the baseline method [1], the real-time absolute trajectory error (ATE) of our proposed method has decreased by 29.1%, 19.8% on TartanAir hospital and EuRoC datasets respectively. Our method also exceeds existing state-of-the-art algorithms on both synthetic and real-world datasets.