We introduce a methodology for radiometric reconstruction (i.e. the simultaneous recovery of multiple illuminants and surface albedoes from multiple views), assuming that the geometry of the scene and of the cameras is known. We formulate a linear theory of multiple illuminants and show its similarity to the theory of geometric recovery of multiple views. Linear and nonlinear implementations are proposed, simulation results are discussed and, finally, results on real images are presented.

A method for reliably detecting change in the 3-D shape of objects that are well-modeled as single-value functions Z=f(x,y) is presented. It uses an estimate of the accuracy of the 3-D models derived from a set of images taken simultaneously. This accuracy estimate is used to distinguish between significant and insignificant changes in 3-D models derived from different image sets. The accuracy of the 3-D model is estimated using a general methodology, called self-consistency, for estimating the accuracy of computer vision algorithms, which does not require prior establishment of "ground truth". A novel image-matching measure based on Minimum Description Length (MDL) theory allows us to estimate the accuracy of individual elements of the 3-D model. Experiments to demonstrate the utility of the procedure are presented.

We present a multiview method for the computation of object shape and reflectance characteristics based on the integration of shape from shading (SFS) and stereo, for nonconstant albedo and non-uniformly Lambertian surfaces. First we perform stereo fitting on the input stereo pairs or image sequences. When the images are uncalibrated, we recover the camera parameters using bundle adjustment. Eased on the stereo result, we can automatically segment the albedo map (which is taken to be piece-wise constant) using a minimum description length (MDL) based metric, to identify areas suitable for SFS (typically smooth textureless areas) and to derive illumination information. The shape and the illumination parameter estimates are refined using a deformable model SFS algorithm, which iterates between computing shape and illumination parameters. Our method takes into account the viewing angle dependent for shortening and specularity effects, and compensates as much as possible by utilizing information from more than one images. We demonstrate that we can extend the applicability of SFS algorithms to real world situations when some of its traditional assumptions are violated. We demonstrate our method by applying it to face shape reconstruction. Experimental results indicate a significant improvement over SFS-only or stereo-only based reconstruction. Model accuracy and detail are improved, especially in areas of low texture detail. Albedo information is retrieved and can be used to accurately re-render the model under different illumination conditions.

We present a method for registering images of complex 3D surfaces that does not require explicit correspondences between features across the images. Our method relies on the use of a full 3D model of the surface to adjust the position and orientation of the camera by minimizing an objective function based on the projections of the images onto the model. This approach constrains the camera parameters strongly enough so that the models do not need, initially, to be accurate to yield good results. When registration has been achieved, the models can be refined and the fine details recovered. Our method is applicable to the calibration of stereo imagery, the precise registration of new images of a scene and the tracking of deformable objects. It can therefore lead to important applications in fields such as augmented reality in a medical context or data compression for transmission purposes. We demonstrate its applicability by using images of faces and of terrain.>

A method for recovering shape from shading that solves directly for the surface height is presented. By using a discrete formulation of the problem, it is possible to achieve good convergence behavior by employing numerical solution techniques more powerful than gradient descent methods derived from variational calculus. Because this method solves directly for height, it avoids the problem of finding an integrable surface maximally consistent with surface orientation. Furthermore, since additional constraints are not needed to make the problem well posed, a smoothness constraint is used only to drive the system towards a good solution; the weight of the smoothness term is eventually reduced to near zero. By solving directly for height, stereo processing may be used to provide initial and boundary conditions. The shape from shading technique, as well as its relation to stereo, is demonstrated on both synthetic and real imagery.>

The detailed structure of intensities in the local neighborhood of an edge can often indicate the nature of the physical event givinig rise to that edge. We argue that the limit, as we approach arbitrarily close to either side of an edge, of such image parameters as type of texture, texture gradient, color, appropriate directional derivatives of intensity, etc., is a key aspect of this structure. However, the general problem of capturing this local structure is surprisingly complex. Thus, we restrict ourselves in this paper to a relatively simple domain¿one-dimensional cuts through idealized images modeled by piecewise smooth (C1) functions corrupted by Gaussian noise. Within this domain, we define local structure to be the limit of the uncorrupted intensity and of its derivatives as we approach arbitrarily close to either side of a discontinuity. We develop a technique that captures this local structure while simultaneously locating the discontinuities, and demonstrate that these tasks are in fact inseparable. The technique is an extension, using estimation theory, of the classical definition of discontinuity. It handles, in a consistent fashion, both jump discontinuities in the function and jump discontinuities in its first derivative (so-called step-edges are a special case of the former and roof-edges of the latter). It also integrates, again in a consistent fashion, information derived from a number of different neighborhood sizes.

Relaxation labeling processes are a class of iterative algorithms for using contextual information to reduce local ambiguities. This paper introduces a new perspective toward relaxation-that of considering it as a process for reordering labels attached to nodes in a graph. This new perspective is used to establish the formal equivalence between relaxation and another widely used algorithm, local maxima selection. The equivalence specifies conditions under which a family of cooperative relaxation algorithms, which generalize the well-known ones, decompose into purely local ones. Since these conditions are also sufficient for guaranteeing the convergence of relaxation processes, they serve as stopping criteria. We feel that equivalences such as these are necessary for the proper application of relaxation and maxima selection in complex speech and vision understanding systems.