Dense image correspondence is central to many applications, such as visual odometry, 3D reconstruction, object association, and re-identification. Historically, dense correspondence has been tackled separately for wide-baseline scenarios and optical flow estimation, despite the common goal of matching content between two images. In this paper, we develop a Unified Flow \& Matching model (UFM), which is trained on unified data for pixels that are co-visible in both source and target images. UFM uses a simple, generic transformer architecture that directly regresses the $(u,v)$ flow. It is easier to train and more accurate for large flows compared to the typical coarse-to-fine cost volumes in prior work. UFM is 28\% more accurate than state-of-the-art flow methods (Unimatch), while also having 62\% less error and 6.7x faster than dense wide-baseline matchers (RoMa). UFM is the first to demonstrate that unified training can outperform specialized approaches across both domains. This result enables fast, general-purpose correspondence and opens new directions for multi-modal, long-range, and real-time correspondence tasks.

The 360-depth estimation with spherical images and LiDAR data has recently become increasingly popular in autonomous driving and scene reconstruction. Compared with perspective images, spherical images have omnidirectional FoV, which exceedingly matches LiDAR data. However, the spherical distortion makes the 360-depth estimation a great challenge. To address this problem, we propose a self-supervised 360 depth estimation network in this paper. The network consists of a spherical convolution branch to extract panoramic image features and a ResNet branch to extract LiDAR features. Then an attention-based decoder is designed to estimate the depth. The reprojection error is used to self-supervise the network training. Experiments on the KITTI-360 dataset demonstrate the effectiveness of the proposed method.

Global visual localization is an important task in geoscience with a plethora of applications such as SLAM and autonomous navigation. Current place recognition approaches restrict the modality of the query data which relies on the database data modality. However, real-world robots are equipped with different sensors in different application scenarios and it is difficult for data from a single fixed modality to accommodate all challenging environments. To overcome this limitation, we propose to build a generalized model that allows spherical images and point clouds to be retrieved under any single-modal query. Our 2D-3D dataset is created based on the KITTI360 dataset with spherical images and corresponding point clouds for training and evaluation. Extensive experimental results demonstrate the effectiveness of our proposed approach.