Videos consisting of thousands of high resolution frames are challenging for existing structure from motion (SfM) and simultaneous-localization and mapping (SLAM) techniques. We present a new approach for simultaneously computing extrinsic camera poses and 3D scene structure that is capable of handling such large volumes of image data. The key insight behind this paper is to effectively exploit coherence in densely sampled video input. Our technical contributions include robust tracking and selection of confident video frames, a novel window bundle adjustment, frame-to-structure verification for globally consistent reconstructions with multi-loop closing, and utilizing efficient global linear camera pose estimation in order to link both consecutive and distant bundle adjustment windows. To our knowledge we describe the first system that is capable of handling high resolution, high frame-rate video data with close to real-time performance. In addition, our approach can robustly integrate data from different video sequences, allowing multiple video streams to be simultaneously calibrated in an efficient and globally optimal way. We demonstrate high quality alignment on large scale challenging datasets, e.g., 2-20 megapixel resolution at frame rates of 25-120 Hz with thousands of frames.

The matching of oriented local image feature descriptors like SIFT, SURF or ORB often includes the refinement and filtering of matches based on the relative orientation of the features. This is important since the computational cost for subsequent tasks like camera pose estimation or object detection increases dramatically with the number of outliers. Simple 2D orientation descriptions are unsuitable for Omni directional images because of image distortions and non-monotonic mapping from camera rotations to image rotations. In this work we introduce 3D orientation descriptors which, unlike 2D descriptors, are suitable for match refinement on Omni directional images and improve matching results on images from cameras and camera rigs with a wide field of view. We evaluate different match refinement strategies based on 2D and 3D orientations and show the fundamental advantages of our approach.