The photorealistic modeling of large-scale scenes, such as urban structures, requires a fusion of range sensing technology and traditional digital photography. This paper presents a system that integrates multiview geometry and automated 3D registration techniques for texture mapping 2D images onto 3D range data. The 3D range scans and the 2D photographs are respectively used to generate a pair of 3D models of the scene. The first model consists of a dense 3D point cloud, produced by using a 3D-to-3D registration method that matches 3D lines in the range images. The second model consists of a sparse 3D point cloud, produced by applying a multiview geometry (structure-from-motion) algorithm directly on a sequence of 2D photographs. This paper introduces a novel algorithm for automatically recovering the rotation, scale, and translation that best aligns the dense and sparse models. This alignment is necessary to enable the photographs to be optimally texture mapped onto the dense model. The contribution of this work is that it merges the benefits of multiview geometry with automated registration of 3D range scans to produce photorealistic models with minimal human interaction. We present results from experiments in large-scale urban scenes.

This paper describes a novel technique to recover large similarity transformations (rotation/scale/translation) and moderate perspective deformations among image pairs. We introduce a hybrid algorithm that features log-polar mappings and nonlinear least squares optimization. The use of log-polar techniques in the spatial domain is introduced as a preprocessing module to recover large scale changes (e.g., at least four-fold) and arbitrary rotations. Although log-polar techniques are used in the Fourier-Mellin transform to accommodate rotation and scale in the frequency domain, its use in registering images subjected to very large scale changes has not yet been exploited in the spatial domain. In this paper, we demonstrate the superior performance of the log-polar transform in featureless image registration in the spatial domain. We achieve subpixel accuracy through the use of nonlinear least squares optimization. The registration process yields the eight parameters of the perspective transformation that best aligns the two input images. Extensive testing was performed on uncalibrated real images and an array of 10,000 image pairs with known transformations derived from the Corel Stock Photo Library of royalty-free photographic images.

This paper introduces a "diminished reality" technique for removing an object or collection of objects and replacing it with an appropriate background image. Diminished reality can be considered an important part of many mixed and augmented reality applications. Our target application is the use of augmented reality (AR) to revamp procedures in industrial plants. An object or a region of interest is delineated on a single reference image. A paraperspective projection model is used to generate the correct background from multiple calibrated views of the scene. We propose methods to deal with approximately planar backgrounds with different orientations. We also propose a multi-resolution approach to deal with non-planar backgrounds. Different sets of experimental results demonstrate the success and limits of the algorithms. Results on real data from water treatment and power plants show the usefulness of this method for industrial applications.

This paper describes a hierarchical image registration algorithm for affine motion recovery. The algorithm estimates the affine transformation parameters necessary to register any two digital images misaligned due to rotation, scale, shear, and translation. The parameters are computed iteratively in a coarse-to-fine hierarchical framework using a variation of the Levenberg-Marquadt nonlinear least squares optimization method. This approach yields a robust solution that precisely registers images with subpixel accuracy. A log-polar registration module is introduced to accommodate arbitrary rotation angles and a wide range of scale changes. This serves to furnish a good initial estimate for the optimization-based affine registration stage. We demonstrate the hybrid algorithm on pairs of digital images subjected to large affine motion.