The recovery of structure from motion in real time over extended areas demands methods that mitigate the effects of computational complexity and arithmetical inconsistency. In this paper, we develop SCISM, an algorithm based on relative frame bundle adjustment, which splits the recovered map of 3D landmarks and keyframes poses so that the camera can continue to grow and explore a local map in real time while, at the same time, a bulk map is optimized in the background. By temporarily excluding certain measurements, it ensures that both maps are consistent, and by using the relative frame representation, new results from the bulk process can update the local process without disturbance. The paper first shows how to apply this representation to the parallel tracking and mapping (PTAM) method, a real-time bundle adjuster, and compares results obtained using global and relative frames. It then explains the relative representation's use in SCISM and describes an implementation using PTAM. The paper provides evidence of the algorithm's real-time operation in outdoor scenes, and includes comparison with a more conventional submapping approach.

A successful approach in the recovery of video- rate structure from motion is to allow the camera to keep track of its position in every frame assuming the recovered set of scene landmarks is fixed in 3D, and then to use the poses in a subset of separated frames, or keyframes, to initialise further landmark structure. The landmark structure and keyframe poses are optimised in a bundle adjustment. Unfortunately this monolithic bundle adjustment has cubic complexity. This paper shows how representing landmarks and camera poses in relative frames, and by temporarily removing certain measurements, introduces a conditional indepedence which allows the bundle adjustment to be split into two parts. One "local" part involves the most recent keyframes and associated landmarks, and runs in constant time. The other "global" part deals with older keyframes and structure, and runs, as ever, in cubic time. Three important outcomes are: (i) the fixed- time local adjustment allows exploratory map-building to keep pace with camera pose tracking; (ii) it produces statistically consistent results; and (iii) referencing to relative frames means that any update in positions from the global adjustment are immediately incorporated in the local fixed-time adjustment. The relative frame approach is applied to the parallel tracking and mapping method for structure from motion, and its results shown to be identical, and the exploratory map building phase shown to maintain fixed time performance.

This paper develops a Square Root Unscented Kalman Filter (SRUKF) for performing video-rate visual simultaneous localization and mapping (SLAM) using a single camera. The conventional UKF has been proposed previously for SLAM, improving the handling of nonlinearities compared with the more widely used Extended Kalman Filter (EKF). However, no account was taken of the comparative complexity of the algorithms: In SLAM, the UKF scales as O(N;{3}) in the state length, compared to the EKF's O(N;{2}), making it unsuitable for video-rate applications with other than unrealistically few scene points. Here, it is shown that the SRUKF provides the same results as the UKF to within machine accuracy and that it can be reposed with complexity O(N;{2}) for state estimation in visual SLAM. This paper presents results from video-rate experiments on live imagery. Trials using synthesized data show that the consistency of the SRUKF is routinely better than that of the EKF, but that its overall cost settles at an order of magnitude greater than the EKF for large scenes.

This paper introduces a square root unscented Kalman filter (SRUKF) solution to the problem of performing visual simultaneous localization and mapping (SLAM) using a single camera. Several authors have proposed the conventional UKF for SLAM to improve the handling of non-linearities compared with the more widely used EKF, but at the expense increasing computational complexity from O(N2) to O(N3) in the map size, making it unattractive for video-rate application. Van der Merwe and Wan's general SRUKF delivers identical results to a general UKF along with computational savings, but remains O(N3) overall. This paper shows how the SRUKF for the SLAM problem can be re-posed with O(N2) complexity, matching that of the EKF. The paper also shows how the method of inverse depth feature initialization developed by Montiel et al. for the EKF can be reformulated to work with the SRUKF. Experimental results confirm that the SRUKF and the UKF produce identical estimates, and that the SRUKF is more consistent than the EKF. Although the complexity is the same, the SRUKF remains more expensive to compute.