This paper proposes a multi-map based visual localization method for image sequences. Given multiple single-map based localization results, we combine them with SLAM to estimate robust and accurate camera poses under challenging conditions. Our method comprises three modules connected in a sequence. First, we reconstruct multiple reference maps using the Structure-from-Motion technique, one map for each reference sequence. A single-image-based localization pipeline is performed to estimate 6-DoF camera poses for each query image, one for each map. Second, a consensus set maximization module is proposed to select the best camera poses from multi-map poses, estimating one 6-DoF camera pose for each query image. Finally, a robust pose refinement module is proposed to optimize 6-DoF camera poses of query images, combining map-based localization and local SLAM information. Experiments show that the proposed pipeline achieves state-of-the-art performance on challenging map-based localization benchmarks. Demonstrating the broad applicability of our method, we obtained first place in the challenge of Map-Based Localization for Autonomous Driving at ECCV2022.

Aging degradation dominated by bias temperature instability (BTI) effect is one of the important considerations in system on chip (SOC) design margin. Research on path delay with BTI recovery effect which mitigates degradation is meaningful to set a reasonable aging margin. Since BTI recovery effect known in transistor level occurs very fast, it is a challenge to sample aged path delay in a short interval. In this paper, we propose an aging monitor to investigate the impact of BTI recovery effect on path delay degradation (Δdelay) in nanosecond intervals. The results show that the power function of recovery time accurately fits the trend of Δdelay after the removal of stress. The higher the stress voltage, the faster the absolute value of Δdelay recovers. Increasing stress time obviously reduces the recovery speed of Δdelay. It’s note that BTI recovery effect occurs not only after the removal of stress but also during AC stress. Therefore, the Δdelay is so dependent on stress type that the Δdelay with BTI recovery is 0.2 times of that without BTI recovery. The silicon data also contributes to aging model’s calibration by the introduction of BTI recovery coefficient, which has a ~4% design margin benefit in a 1GHz SOC design.

Camera–inertial measurement unit (IMU) sensor fusion has been extensively studied in recent decades. Numerous observability analysis and fusion schemes for motion estimation with self-calibration have been presented. However, it has been uncertain whether the intrinsic parameters of both the camera and the IMU are observable under general motion. To answer this question, by using the Lie derivatives, we first prove that for a rolling shutter (RS) camera–IMU system, all the intrinsic and extrinsic parameters, camera time offset, and readout time of the RS camera are observable with an unknown landmark. To our knowledge, we are the first to present such a proof. Next, to validate this analysis and to solve the drift issue of a structureless filter during standstills, we develop a keyframe-based sliding window filter (KSWF) for odometry and self-calibration, which works with a monocular RS camera or stereo RS cameras. Though the keyframe concept is widely used in vision-based sensor fusion, to our knowledge, the KSWF is the first of its kind to support self-calibration. Our simulation and real data tests have validated that it is possible to fully calibrate the camera–IMU system using the observations of opportunistic landmarks under diverse motion. Real data tests confirmed previous allusions that keeping landmarks in the state vector can remedy the drift in standstill and showed that the keyframe-based scheme is an alternative solution.

State estimation problems without absolute position measurements routinely arise in navigation of unmanned aerial vehicles, autonomous ground vehicles, etc., whose proper operation relies on accurate state estimates and reliable covariances. Unaware of absolute positions, these problems have immanent unobservable directions. Traditional causal estimators, however, usually gain spurious information on the unobservable directions, leading to over-confident covariance inconsistent with actual estimator errors. The consistency problem of fixed-lag smoothers (FLSs) has only been attacked by the first estimate Jacobian (FEJ) technique because of the complexity to analyze their observability property. But the FEJ has several drawbacks hampering its wide adoption. To ensure the consistency of a FLS, this paper introduces the right invariant error formulation into the FLS framework. To our knowledge, we are the first to analyze the observability of a FLS with the right invariant error. Our main contributions are twofold. As the first novelty, to bypass the complexity of analysis with the classic observability matrix, we show that observability analysis of FLSs can be done equivalently on the linearized system. Second, we prove that the inconsistency issue in the traditional FLS can be elegantly solved by the right invariant error formulation without artificially correcting Jacobians. By applying the proposed FLS to the monocular visual inertial simultaneous localization and mapping (SLAM) problem, we confirm that the method consistently estimates covariance similarly to a batch smoother in simulation and that our method achieved comparable accuracy as traditional FLSs on real data.