We introduce the first learning-based dense matching algorithm, termed Equirectangular Projection-Oriented Dense Kernelized Feature Matching (EDM), specifically designed for omnidirectional images. Equirectangular projection (ERP) images, with their large fields of view, are particularly suited for dense matching techniques that aim to establish comprehensive correspondences across images. However, ERP images are subject to significant distortions, which we address by leveraging the spherical camera model and geodesic flow refinement in the dense matching method. To further mitigate these distortions, we propose spherical positional embeddings based on 3D Cartesian coordinates of the feature grid. Additionally, our method incorporates bidirectional transformations between spherical and Cartesian coordinate systems during refinement, utilizing a unit sphere to improve matching performance. We demonstrate that our proposed method achieves notable performance enhancements, with improvements of +26.72 and +42.62 in AUC@5° on the Matterport3D and Stanford2D3D datasets. Project Page: https://jdk9405.github.io/EDM

This paper tackles a localization problem in large-scale indoor environments with wayfinding maps. A wayfinding map abstractly portrays the environment, and humans can localize themselves based on the map. However, when it comes to using it for robot localization, large geometrical discrepancies between the wayfinding map and the real world make it hard to use conventional localization methods. Our objective is to estimate a robot pose within a wayfinding map, utilizing RGB images from perspective cameras. We introduce two different imagination modules which are inspired by how humans can comprehend and interpret their surroundings for localization purposes. These modules jointly learn how to effectively observe the first-person-view (FPV) world to interpret bird-eye-view (BEV) maps. Providing explicit guidance to the two imagination modules significantly improves the precision of the localization system. We demonstrate the effectiveness of the proposed approach using real-world datasets, which are collected from various large-scale crowded indoor environments. The experimental results show that, in 85% of scenarios, the proposed localization system can estimate its pose within 3m in large indoor spaces. Project Site: https://rllab-snu.github.io/projects/WayIL/

Global registration using 3D point clouds is a crucial technology for mobile platforms to achieve localization or manage loop-closing situations. In recent years, numerous researchers have proposed global registration methods to address a large number of outlier correspondences. Unfortunately, the degeneracy problem, which represents the phenomenon in which the number of estimated inliers becomes lower than three, is still potentially inevitable. To tackle the problem, a degeneracy-robust decoupling-based global registration method is proposed, called Quatro. In particular, our method employs quasi-SO(3) estimation by leveraging the Atlanta world assumption in urban environments to avoid degeneracy in rotation estimation. Thus, the minimum degree of freedom (DoF) of our method is reduced from three to one. As verified in indoor and outdoor 3D LiDAR datasets, our proposed method yields robust global registration performance compared with other global registration methods, even for distant point cloud pairs. Furthermore, the experimental results confirm the applicability of our method as a coarse alignment. Our code is available: https://github.com/url-kaist/quatro

Estimating the precise location of a camera using visual localization enables interesting applications such as augmented reality or robot navigation. This is particularly useful in indoor environments where other localization technologies, such as GNSS, fail. Indoor spaces impose interesting challenges on visual localization algorithms: occlusions due to people, textureless surfaces, large viewpoint changes, low light, repetitive textures, etc. Existing indoor datasets are either comparably small or do only cover a subset of the mentioned challenges. In this paper, we introduce 5 new indoor datasets for visual localization in challenging real-world environments. They were captured in a large shopping mall and a large metro station in Seoul, South Korea, using a dedicated mapping platform consisting of 10 cameras and 2 laser scanners. In order to obtain accurate ground truth camera poses, we developed a robust LiDAR SLAM which provides initial poses that are then refined using a novel structure-from-motion based optimization. We present a benchmark of modern visual localization algorithms on these challenging datasets showing superior performance of structure-based methods using robust image features. The datasets are available at: https://naverlabs.com/datasets

With its essential role in achieving full autonomy of robot navigation, place recognition has been widely studied with various approaches. Recently, numerous point cloud-based methods with deep learning implementation have been proposed with promising results for their application in outdoor environments. However, their performances are not as promising in indoor spaces because of the high level of occlusion caused by structures and moving objects. In this paper, we propose a point cloud-based place recognition method for crowded indoor spaces. The method consists of voxelizing point clouds in spherical coordinates and defining the occupancy of each voxel in ternary values. We also present SpoxelNet, a neural network architecture that encodes input voxels into global descriptor vectors by extracting the structural features in both fine and coarse scales. It also reinforces its performance in occluded places by concatenating feature vectors from multiple directions. Our method is evaluated in various indoor datasets and outperforms existing methods with a large margin.

In indoor spaces, plane features are general geometric primitives that are robust to noise. To estimate the relative displacement and orientation between two consecutive 3D poses, at least three-pair correspondences of planes with different normals are needed at both poses. Otherwise, the estimated state falls into degeneracy, which is usually caused by the narrow field-of-view (FoV) of sensors as well as environmental structures. In the case where there are changes in degeneracy, with the exception of rotation, the state can be resolved using existing methods. To compensate for rotation in degeneracy for one-pair correspondences, the main approach is to project the initial prediction of an inertial measurement unit's (IMU) orientation to the normal of the corresponding plane such that it satisfies the shortest arc length. We also propose an algorithm that detects various degeneracy cases through which a seamless implementation becomes feasible, regardless of the number of plane pairs. We show the usefulness of the proposed method by performing a 3D mapping experiment where several degenerate cases arise due to limited FoV.