Precise position information plays a crucial role in industries such as intelligent unmanned systems and autonomous driving. Combining the advantages of traditional precise point positioning (PPP) and real-time kinematic (RTK) technologies, the emerging PPP-RTK technique is able to provide real-time centimeter-level positioning in open environments. However, its stability is often insufficient in complex environments due to frequent interference of satellite signals. To enhance the continuity and robustness of positioning results in such challenging environments, inertial navigation system (INS), which can perceive the motion information of the carrier, is commonly integrated with global navigation satellite system (GNSS). In this paper, an optimization-based tightly-coupled multi-GNSS PPP-RTK/INS system is proposed to achieve continuous and reliable positioning in urban environments. In the proposed algorithm, an estimator based on Maximum A Posteriori (MAP) estimation is utilized to integrate multi-GNSS raw observations, inertial measurements and precise atmospheric information. Moreover, the outlier detection algorithm based on standardized residuals is applied in the proposed method to reject gross errors introduced by GNSS observations, making the positioning results more consistent with observations to get reliable ambiguity-float solutions. Then, a cascaded ambiguity resolution is conducted to obtain final precise ambiguity-fixed solutions. The experimental results show that the proposed method achieves centimeter-level horizontal position estimates in urban environments, with an accuracy improvement of over 80% compared to PPP-RTK. Besides, the proposed optimization-based algorithm represents an accuracy improvement of approximately 30% compared to the traditional extended Kalman filter-based method.

Continuous and reliable estimation of navigation states is of paramount importance in ensuring the safe operation of intelligent vehicles. The conventional global navigation satellite system (GNSS)-Inertial-Visual navigation systems have demonstrated the capability to achieve locally accurate and globally drift-free pose estimation. However, challenges such as frequent satellite signal interference, limited visual features, and sudden sensor failures can severely degrade performance or even lead to complete system collapse when utilizing minimal sensor configurations. For this reason, we propose a novel and globally drift-free tightly coupled (TC) system that can integrate any number of GNSS, inertial measurement units (IMUs), and cameras to enable accurate and robust vehicle navigation. Specifically, a stacked state estimator centered on the IMU is designed to fuse information from all sensors at the raw measurement level. The pseudorange and carrier phase measurements from all GNSS terminals are directly correlated with the core IMU, ensuring accurate and fast positioning of the system in a global frame. The feature measurements from multiple independent cameras are also used to update the states by exploiting inter-epoch geometric constraints. In addition, the multiple homogeneous IMUs can not only further improve the state estimation of the system by imposing rigid constraints on the core IMU, but also switch over in time for smooth state estimation when the core IMU are faulty. We comprehensively evaluate the state estimation accuracy and robustness of the proposed approach through a series of in-vehicle experiments and simulation experiments in real urban scenarios. The results indicate that the proposed system can achieve 93.2% availability with a horizontal position error less than 0.5 m and 97.8% availability with a heading error less than 0.2 deg in typical urban environments, significantly outperforming both conventional and state-of-the-art approaches. Note to Practitioners—This study focuses on the tight integration of multiple homogeneous and heterogeneous sensors with the goal of addressing frequent interference and degradation challenges in wide-area vehicle navigation applications. We propose a general-purpose GNSS-Inertial-Visual tight coupled framework capable of integrating any number of GNSS, IMUs, and cameras at the raw measurement level. It maximizes the use of as much sensor information as possible to achieve accurate and robust state estimation and is resilient to anomalous measurements and sensor unavailability. This solution holds practical and effective for autonomous vehicles that are now commonly equipped with multiple sensors.

Most existing LiDAR-Inertial odometry (LIO) systems heavily depend on the assumption of a static environment, which hinders their ability to effectively utilize dynamic objects in enhancing localization. Furthermore, accurate tracking of surrounding objects is crucial for emerging applications, such as autonomous driving and multi-robot collaboration. To this end, we present LIO-LOT, a tightly coupled LiDAR-inertial and multi-object tracking (MOT) system. In our system, each movable object is represented by a 3-D bounding box, derived from an object detector. Subsequently, these objects are associated with historical trajectories through a hybrid information matching strategy that integrates detections and geometric feature information. Considering tracking continuity and motion consistency, the associated objects are further categorized into high-confidence ones and low-confidence ones. Building upon this foundation, these two categories of objects are flexibly coupled with inertial measurement unit (IMU) pre-integration measurements and static scene structures to optimize the poses of both tracked objects and ego-vehicle under a factor graph optimization framework. For implementation, the aggregated information of high- and low-confidence objects are fully leveraged to form the object-related factors, which are hierarchically optimized with LIO, enhancing system stability in cases of poor object detections. The system’s performance is extensively evaluated using the KITTI and nuScenes datasets. Experimental results explicitly demonstrate that the proposed system outperforms other state-of-the-art methods in terms of ego-localization and multi-object tracking, particularly in highly dynamic scenarios with situations of occlusion and truncation.

The integrated inertial system, typically integrating an IMU and an exteroceptive sensor, such as radar, light detection and ranging (LiDAR), and camera, has been widely accepted and applied in modern robotic applications for ego-motion estimation, motion control, or autonomous exploration. To improve system accuracy, robustness, and further usability, both multiple and various sensors are generally resiliently integrated, which benefits the system performance regarding failure tolerance, perception capability, and environment compatibility. For such systems, accurate and consistent spatiotemporal calibration is required to maintain a unique spatiotemporal framework for multisensor fusion. Considering that most existing calibration methods first, are generally oriented to specific integrated inertial systems, second, often focus on spatial-only determination, and third, usually require artificial targets, lacking convenience and usability, we proposeiKalibr:a unified targetless spatiotemporal calibration framework for resilient integrated inertial systems, which overcomes the above issues, and enables both accurate and consistent calibration. Altogether four commonly employed sensors are supported iniKalibrcurrently, namely, IMU, radar, LiDAR, and camera. The proposed method starts with a rigorous and efficient dynamic initialization, where all parameters in the estimator would be accurately recovered. Subsequently, several continuous-time batch optimizations are conducted to refine the initialized parameters toward better states. Sufficient real-world experiments were conducted to verify the feasibility and evaluate the calibration performance ofiKalibr. The results demonstrate thatiKalibrcan achieve accurate resilient spatiotemporal calibration.

For advanced autonomous systems, precise and consistent ego-motion estimation is critical to ensure system safety and performant closed-loop control. Unfortunately, traditional discrete-time approaches generally rely on interpolation to fuse asynchronous measurements, which would inevitably lose accuracy in high-speed motions. To this end, we propose a tightly coupled LiDAR/IMU/Radar odometry tailored for accurate and smooth state estimation using continuous-time trajectory optimization. Specifically, a novel coarse-to-fine dynamic initialization method is first proposed to obtain system states, IMU biases, and optional intersensor spatiotemporal parameters without requiring specific artificial targets and movements. After performing data association for LiDAR scans and outlier removal for Radar scans, LiDAR point-to-plane factor, angular velocity factor, linear acceleration factor, and Radar Doppler velocity factor are jointly minimized in a factor graph framework to derive optimal estimates. A series of dedicated experiments on the real-world data sets were conducted to evaluate the proposed method. The results demonstrate that the proposed method exhibits superior navigation performance in complex environments than the state-of-the-art methods in terms of position, velocity, and attitude.

Nowadays, LiDAR-IMU systems have progressively prevailed in mobile robotic applications due to their excellent complementary characteristics. With the steady decline and shrinking in cost and size of these sensors, it has become feasible and even imperative to further leverage multiple sensor units for better accuracy and robustness. In such a fusion-based system, accurate spatiotemporal calibration is a fundamental prerequisite. However, existing calibration methods generally necessitate artificial targets or an overlapping field-of-view (FoV) between LiDARs. To this end, we propose an accurate and easy-to-use and spatiotemporal calibration approach tailored to multi-LiDAR multi-IMU systems based on continuous-time batch estimation, which supports both mechanical spinning LiDARs and small FoV solid-state LiDARs. Inspired by classical hand–eye calibration, a stepwise multistage nonlinear optimization problem is first built to recover the rotational extrinsic and poses of control points in the spline without using any artificial targets, special movements, or manual intervention. Meanwhile, a virtual IMU and LiDAR are constructed to bridge all IMUs and LiDARs based on the centralized principles in both temporal and spatial domains. Subsequently, the point-to-plane factors associated by different LiDARs, gyroscope, and accelerometer factors are jointly minimized to optimize all spatiotemporal parameters over multiple batches. Extensive simulation tests and real-world experiments were conducted to quantitatively evaluate the feasibility of the proposed method. Meanwhile, the proposed method is compared with the state-of-the-art methods and shows superior calibration performance.

With significant advances in manufacturing technology, multiple sensors such as global navigation satellite system (GNSS) devices, inertial measurement units (IMU) and cameras have become affordable in intelligent vehicle systems and are generally integrated for motion estimation and environmental perception. In such integrated systems, accurate and flexible sensor calibration is a basic foundation for optimal information fusion. However, traditional calibration methods are generally labor-intensive or only applicable to handheld platforms. To address this issue, we propose an accurate and plug-and-play self-calibration method for in-vehicle GNSS/inertial navigation system (INS)/Vision system. Without relying on specially-designed targets and initial guess of sensor setups, all involved sensing parameters, including GNSS-IMU and Camera-IMU spatial-temporal extrinsics, are properly initialized and estimated online via a centralized Extended Kalman Filter (EKF). Besides, an observability constrained module is also developed to address degradation cases through detailed observability analysis. The results in both Monte-Carlo simulations and real-world experiments indicate that the proposed method enables accurate online calibration of all spatiotemporal parameters involved in GNSS/INS/Vision systems and shows superior calibration performance over the state-of-the-art methods. Furthermore, for the first time, we prove the superiority of using raw GNSS observations to enhance camera-IMU calibration than using processed GNSS positions or not using any GNSS information.

Accurate position, velocity and orientation are essential for the autonomous navigation of unmanned vehicles. The integration of GNSS and INS that can deliver continuous navigation states is widely used for intelligent vehicle systems. In this paper, we propose a tightly coupled GNSS/INS positioning framework with carrier-phase ambiguity resolution based on factor graph optimization (FGO). In this approach, a sliding window optimizer is employed to fuse the multi-GNSS pseudorange and carrier-phase observations with inertial measurements. The same ambiguity within the window is considered as a state node, and the constraint on the ambiguity is continuously preserved by marginalization. To further improve the accuracy and reliability of precise positioning, the carrier-phase ambiguity resolution is introduced to the FGO-based GNSS/INS framework. When the vehicle is detected to be stationary, a zero-velocity constraint and an attitude invariant constraint will be imposed. Several experimental results indicate that the proposed method can accomplish the centimeter-level position estimation performance with beyond 90% positioning availability (horizontal$<$10 cm and vertical$<$10 cm) and outperforms the current state-of-the-art filter-based tightly coupled method.

Monocular visual-inertial odometry (VIO) is a low-cost solution to provide high-accuracy, low-drifting pose estimation. However, it encounters challenges in vehicular scenarios, as the restricted motion of a ground vehicle could lead to degraded observability, and a lack of stable features might occur in dynamic road environments. In this paper, we propose Ground-VIO, which utilizes ground features and the specific camera-ground geometry to enhance monocular VIO performance in realistic road environments. In the method, the camera-ground geometry is modeled with vehicle-centered parameters and integrated into an optimization-based VIO framework. These parameters could be calibrated online and simultaneously improve the odometry accuracy by providing stable scale-awareness. Besides, a specially designed visual front-end is developed to stably extract and track ground features via the inverse perspective mapping (IPM) technique. Both real-world experiments and tests on public datasets are conducted to verify the effectiveness of the proposed method. The results show that our implementation could dramatically improve monocular VIO accuracy in vehicular scenarios, achieving comparable performance to state-of-art stereo VIO solutions and showing good robustness in challenging conditions. The system can also be used for the auto-calibration of IPM which is widely used in vehicle perception. A toolkit for ground feature processing, together with the experimental datasets, has been made open-source.

High-precision position, velocity, and orientation information is essential for vehicles to achieve autonomous driving. In this contribution, we propose a multi-sensor integration system for high-precision vehicle navigation, based on the fusion of GNSS PPP-RTK, MEMS IMU, and wheel odometer. Meanwhile, to fully use the physical characteristics of vehicle-specific motion, vehicle motion constraints (VMC) are used in conjunction with the wheel odometer. In the proposed system, the data from all sensors and vehicle motion constraints are tightly integrated into Kalman Filter. To validate the effectiveness of the proposed system, a series of real urban vehicle-borne experiments including different scenarios were conducted. Results indicate that using the proposed system, the position error RMS is (0.02 m, 0.02 m, 0.06 m) in the east, north, and vertical directions, with 95.2% availability of high-precision positioning (horizontal$<$10 cm, vertical$<$20 cm). The continuous and stable high-precision positioning could be maintained in the typical scenarios of GNSS degradation, such as boulevards, viaducts and tunnels. In addition, tests with simulated GNSS outages statistically demonstrate that this proposed system is capable of achieving 5.3dead reckoning error with a maximum position error of 0.86 m in the case of simulated 30 s outages.

The emerging Internet of Things (IoT) applications, such as driverless cars, have a growing demand for high-precision positioning and navigation. Nowadays, the global navigation satellite system (GNSS) is recognized as an important approach for worldwide positioning services. However, its application is limited in urban areas due to severe signal attenuation, reflections, and blockages. Inertial navigation system (INS) can provide high-precision navigation outputs within a short period, but its accuracy suffers from error accumulation, especially when equipped with the low-cost microelectromechanical system (MEMS) inertial measurement units (IMUs). In addition, light detection and ranging (LiDAR) is becoming more common as an option in vehicles, which can detect rich geometric information in the environment for ego-motion estimation. Aiming at taking advantage of the complementary characteristics of these onboard technologies to navigate in urban environments, a tightly coupled multi-GNSS precise point positioning (PPP)/INS/LiDAR integrated system is proposed. We also develop an LiDAR sliding-window plane-feature tracking method to further improve navigation accuracy and computational efficiency. The performance of the proposed integrated system was evaluated in vehicular experiments with different GNSS observation conditions. Results indicate that our proposed GNSS/INS/LiDAR integration can maintain submeter level horizontal positioning accuracy in GNSS-challenging environments, with improvements of (73.3%, 59.7%, and 64.2%) compared to traditional GNSS/INS integration. Moreover, the plane-feature tracking method is proved to outperform traditional point-to-line and point-to-plane scan matching in terms of accuracy and efficiency.

The integration of inertial navigation system (INS) and global navigation satellite system (GNSS) with a single antenna is widely applied in consumer-level vehicle navigation. However, it is challenging for a low-cost single-antenna GNSS/INS integrated system to provide reliable, consistent state estimation under low-dynamic scenarios (e.g., low-speed robotic applications) as the limited dynamics could lead to degradation of the system observability. We propose a novel moving-antenna GNSS/INS integration method, in which the antenna is designed to perform specific motions on the platform during vehicle maneuver, thus to improve the system observability in degraded conditions, bringing about the advantage of multi-antenna GNSS using a single antenna. It turns out that a moving antenna with a maximum speed of just 0.15 m/s would improve the system performance dramatically by applying raw GNSS carrier phase observations into the integration. In the proposed method, firstly, a moving-antenna fast initial alignment algorithm is developed, which enables the system to perform instantaneous initial alignment in static. Secondly, for GNSS/INS integration filter, the moving-antenna scheme improves the heading angle estimation accuracy by more than 50% under typical dynamic scenarios, achieving comparable performance to dual-antenna GNSS/INS integration. This work shows that it is possible and even beneficial to continuously change the inter-sensor transformation in an integrated navigation system.