Accurate and real-time global pose estimation relative to a global prior map is indispensable in many applications, such as logistics with micro aerial vehicles and Augmented Reality. Supposed that a pure sparse 3D point map can provide a structureless representation of the environment, then generating a point-plane prior map can further model the environment topology and offer global constraints for an accurate localization. To implement this, we propose a filter-based, large-scale visual-inertial odometry system, termed PPM-VIO, which utilizes a point-plane map to correct the cumulative drift. Our system, detecting coplanar information from sparse point clouds with semantic information, achieves accurate online plane matching via geometric constraints, semantic constraints, and descriptor constraints. To improve the localization performance, we effectively integrate and formulate the global planar measurements and points measurements in a filter-based estimator. The effectiveness of the proposed method is extensively validated on real-world datasets collected in different scenarios. Experimental results demonstrate that, rather than using the point map alone, leveraging the plane information in the prior map can yield better trajectory estimates and broaden the effective scope of the prior map in different scenes.

Bundle adjustment (BA) is a well-studied fundamental problem in the robotics and vision community. In man-made environments, coplanar points and lines are ubiquitous. However, the number of works on bundle adjustment with coplanar points and lines is relatively small. This paper focuses on this special BA problem, referred to as$\pi-\mathbf{BA}$. For a point or a line on a plane, we derive a new constraint to describe the relationship among two poses and the plane, called$\pi$-constraint. We distribute$\pi$-constraints into different groups. Each group is called a$\pi$-factor. We prove that, with some simple preprocessing, the computational complexity associated with a$\pi$-factor in the Levenberg-Marquardt (LM) algorithm is$O(1)$, independent of the number of$\pi$-constraints packed into the$\pi$-factor. In$\pi-\mathbf{BA}, \pi$-factors replace original reprojection errors. One problem is how to divide$\pi$-constraints into$\pi$-factors. Different strategies may result in different numbers of$\pi$-factors, which in turn affects the efficiency. It is difficult to get the optimal division. We present a greedy algorithm to overcome this problem. Experimental results verify that our algorithm can significantly accelerate the computation.