We introduce a method for segmenting surfaces of three-dimensional objects using two images of the object obtained from the same viewpoint under different illumination conditions. The method allows surface spectral reflectance to vary from point to point and requires only weak conditions on the uncalibrated illumination configuration. The algorithm is based on the local recovery of an illumination change matrix that depends on surface geometry but not on the spectral reflectance of the surface. We show that for typical sensor noise levels, this technique can be used for the reliable detection of surface orientation changes of a few degrees. This approach can be generalized using a calibrated setup to recover a dense set of surface orientation estimates from two images. We present a set of experiments demonstrating the capability of the algorithm for the segmentation of planar surfaces in the presence of spatially varying spectral reflectance.

We present an energy matrix representation for multiband images which captures spatial and spectral properties. Using a physical model for spectral reflectance, we derive a pseudoinverse method for the comparison of energy matrices which is invariant to the spectral properties of the scene illumination. At the same time, this method determines the illumination change matrix allowing direct comparison of images obtained under different illumination conditions. We demonstrate the performance of the method for both illumination-invariant recognition and illumination correction on a large set of multiband images. The energy matrices are generated using a small set of oriented steerable filters. We also demonstrate that a related set of rotationally symmetric filters can be used for recognition invariant to both illumination and rotation and that subsequent processing can be used to recover the rotation angle of a recognized object.

We develop a method for recognizing color texture independent of rotation, scale, and illumination. Color texture is modeled using spatial correlation functions defined within and between sensor bands. Using a linear model for surface spectral reflectance with the same number of parameters as the number of sensor classes, we show that illumination and geometry changes in the scene correspond to a linear transformation of the correlation functions and a linear transformation of their coordinates. A several step algorithm which includes scale estimation and correlation moment computation is used to achieve the invariance. The key to the method is the new result that illumination and geometry changes in the scene correspond to a specific transformation of correlation function Zernike moment matrices. These matrices can be estimated from a color image. This relationship is used to derive an efficient algorithm for recognition. The algorithm is substantiated using classification results on over 200 images of color textures obtained under various illumination conditions and geometric configurations.

We represent texture in a color image using spatial correlation functions defined within and between sensor bands. This representation has been shown to be useful for surface recognition, but the structure of spatial correlation functions depends on the spectral properties of the scene illumination. Using a linear model for surface spectral reflectance with the same number of parameters as the number of classes of photoreceptors, we show that illumination changes correspond to linear transformations of a surface correlation matrix. From this relationship, we derive a distance function for comparing sets of spatial correlation functions that can be used for illumination-invariant recognition. This distance function can be computed efficiently from estimated correlation functions. We demonstrate using a large body of experiments that this distance function can be used for accurate surface recognition in the presence of large changes in illumination spectral distribution.>