In this paper, we consider the problem of estimating the spatiotemporal alignment between N unsynchronized video sequences of the same dynamic 3D scene, captured from distinct viewpoints. Unlike most existing methods, which work for N = 2 and rely on a computationally intensive search in the space of temporal alignments, we present a novel approach that reduces the problem for general N to the robust estimation of a single line in IR(N). This line captures all temporal relations between the sequences and can be computed without any prior knowledge of these relations. Considering that the spatial alignment is captured by the parameters of fundamental matrices, an iterative algorithm is used to refine simultaneously the parameters representing the temporal and spatial relations between the sequences. Experimental results with real-world and synthetic sequences show that our method can accurately align the videos even when they have large misalignments (e.g., hundreds of frames), when the problem is seemingly ambiguous (e.g., scenes with roughly periodic motion), and when accurate manual alignment is difficult (e.g., due to slow-moving objects).

This paper introduces a practical approach to register large-scale GIS imagery to a database of road vectors automatically. The proposed approach breaks the global alignment problem into a set of localized domains (tiles). Within each tile, the displacement between imagery and vectors is approximated by a translation. Finally, a global thin-plate-spline warp based on these local approximations is applied to register the imagery to the vector data.

The wide availability of GPS sensors is changing the landscape in the applications of structure from motion techniques for localization. In this paper, we study the problem of estimating camera orientations from multiple views, given the positions of the viewpoints in a world coordinate system and a set of point correspondences across the views. Given three or more views, the above problem has a finite number of solutions for three or more point correspondences. Given six or more views, the problem has a finite number of solutions for just two or more points. In the three-view case, we show the necessary and sufficient conditions for the three essential matrices to be consistent with a set of known baselines. We also introduce a method to recover the absolute orientations of three views in world coordinates from their essential matrices. To refine these estimates we perform a least-squares minimization on the group cross product SO(3) × SO(3) × SO(3). We report experiments on synthetic data and on data from the ICCV2005 Computer Vision Contest.

This paper introduces a method to recover photometric parameters of a set of 3D surfaces from a single image with significant global-illumination effects such as inter-reflections and transparencies. Since this problem is ambiguous for arbitrary unknown scenes, our formulation assumes that the scene consists of a small set of photometrically homogeneous surfaces with known 3D shapes, illuminated by known light sources. We show that under these conditions, the system of nonlinear equations that defines how the image is formed may be factorized into a vector composed only of products of some photometric parameters, and a matrix, whose elements depend non-linearly on both the known illumination, the known 3D shapes and the remaining photometric parameters. This factorization leads to an efficient optimization-based algorithm to compute all unknown photometric parameters from a single input image. Experiments with real data show that this algorithm is more stable and efficient than simpler alternatives

In this paper we study the problem of recovering the 3D shape, reflectance, and non-rigid motion of a dynamic 3D scene. Because these properties are completely unknown, our approach uses multiple views to build a piecewise continuous geometric and radiometric representation of the scene's trace in space-time. Basic primitive of this representation is the dynamic surfel, which (1) encodes the instantaneous local shape, reflectance, and motion of a small region in the scene, and (2) enables accurate prediction of the region's dynamic appearance under known illumination conditions. We show that complete surfel-based reconstructions can be created by repeatedly applying an algorithm called surfel sampling that combines sampling and parameter estimation to fit a single surfel to a small, bounded region of space-time. Experimental results with the Phong reflectance model and complex real scenes (clothing, skin, shiny objects) illustrate our method's ability to explain pixels and pixel variations in terms of their physical causes-shape, reflectance, motion, illumination, and visibility.

We introduce a framework for recovering the 3D shape and motion of unknown, arbitrarily-moving curves from two or more image sequences acquired simultaneously from distinct points in space. We use this framework to (1) identify ambiguities in the multi-view recovery of (rigid or nonrigid) 3D motion for arbitrary curves, and (2) identify a novel spatio-temporal constraint that couples the problems of 3D shape and 3D motion recovery in the multi-view case. We show that this constraint leads to a simple hypothesize-and-test algorithm for estimating 3D curve shape and motion simultaneously. Experiments performed with synthetic data suggest that, in addition to recovering 3D curve motion, our approach yields shape estimates of higher accuracy than those obtained when stereo analysis alone is applied to a multi-view sequence.

In this paper we consider the problem of recovering the 3D motion and shape of an arbitrarily-moving, arbitrarily-shaped curve from multiple synchronized video streams acquired from distinct and known points in space. By studying the 3D motion and shape constraints provided by the input video streams, we show that (1) shape and motion recovery is equivalent to the problem of recovering the differential properties of the spatio-temporal curve manifold that describes the curve's trace in space-time, and (2) a local analytical description of this manifold can be computed directly from the spatio-temporal volumes defined by the input video streams. Our experimental results suggest that this manifold-based approach to joint shape and motion estimation yields shape estimates of higher accuracy that those obtained from stereo alone, allows accurate recovery of 3D curve motion, and provides significant robustness against image noise and camera calibration errors.

Both the original version of David Lowe's influential and classic algorithm for tracking known objects and a reformulation of it implemented by Ishiiet al. rely on (different) approximated imaging models. Removing their simplifying assumptions yields a fully projective solution with significantly improved accuracy and convergence, and arguably better computation-time properties.