We present a new formalism for the treatment and understanding of multispectral images and multisensor imagery based on first-order contrast information. Although little attention has been paid to the utility of multispectral contrast, we develop a theory for multispectral contrast that enables us to produce an optimal grayscale visualization of the first-order contrast of an image with an arbitrary number of bands. We demonstrate how our technique can reveal significantly more interpretive information to an image analyst, who can use it in a number of image understanding algorithms. Existing grayscale visualization strategies are reviewed. A variety of experimental results are presented to support the performance of the new method.

A key problem for face recognition has been accurate identification under variable illumination conditions. Conventional video cameras sense reflected light so that image grayvalues are a product of both intrinsic skin reflectivity and external incident illumination, thus obfuscating the intrinsic reflectivity of skin. Thermal emission from skin, on the other hand, is an intrinsic measurement that can be isolated from external illumination. We examine the invariance of Long-Wave InfraRed (LWIR) imagery with respect to different illumination conditions from the viewpoint of performance comparisons of two well-known face recognition algorithms applied to LWIR and visible imagery. We develop rigourous data collection protocols that formalize face recognition analysis for computer vision in the thermal IR.

We present a new formalism for the treatment and understanding of multispectral images and multisensor imagery based on first order contrast information. Although little attention has been paid to the utility of multispectral contrast, we develop a theory for multispectral contrast that enables us to produce an optimal grayscale visualization of the first order contrast of an image with an arbitrary number of bands. We demonstrate how our technique can reveal significantly more interpretive information to an image analyst, who can use it in a number of image understanding algorithms. Existing grayscale visualization strategies are reviewed and a discussion is given as to why our algorithm is optimal and outperforms them. A variety of experimental results are presented.

Existing polarization-based image understanding techniques use information only from reflected light. Apart from incandescent bodies thermally emitted light radiation from elements of a scene in the visible spectrum is insignificant. However, at longer wavelengths such as in the infrared thermal emission is typically quite prevalent from a number of scene elements of interest. FLIR imagery of both indoor and outdoor scenes reveals that many objects thermally emit a significant amount of radiation. Polarization from thermally emitting objects has been observed as long as 170 years ago from incandescent objects but since then there have only been a limited number of empirical investigations into this phenomenon. This paper presents a comprehensive model for explaining polarization of thermal emission from both rough and smooth surfaces, in agreement with empirical data, that can significantly enhance the image understanding of FLIR imagery. In particular it is possible to discern metal from dielectric materials under certain conditions, and from an accurate model for thermally emitted polarization it is possible to predictively model polarization signatures from CAD models of importance to automatic target recognition.

We compute the sign of Gaussian curvature using a purely geometric definition. Consider a point p on a smooth surface S and a closed curve /spl gamma/ on S which encloses p. The image of /spl gamma/ on the unit normal Gaussian sphere is a new curve /spl beta/. The Gaussian curvature at p is defined as the ratio of the area enclosed by /spl gamma/ over the area enclosed by /spl beta/ as /spl gamma/ contracts to p. The sign of Gaussian curvature at p is determined by the relative orientations of the closed curves /spl gamma/ and /spl beta/. We directly compute the relative orientation of two such curves from intensity data. We employ three unknown illumination conditions to create a photometric scatter plot. This plot is in one-to-one correspondence with the subset of the unit Gaussian sphere containing the mutually illuminated surface normal. This permits direct computation of the sign of Gaussian curvature without the recovery of surface normals. Our method is albedo invariant. We assume diffuse reflectance, but the nature of the diffuse reflectance can be general and unknown. Error analysis on simulated images shows the accuracy of our technique. We also demonstrate the performance of this methodology on empirical data.

The authors compute the sign of Gaussian curvature using a purely geometric definition. Consider a point p on a smooth surface S and a closed curve /spl gamma/ on S which encloses p. The image of /spl gamma/ on the unit normal Gaussian sphere is a new curve /spl beta/. The Gaussian curvature at p is defined as the ratio of the area enclosed by /spl gamma/ over the area enclosed by /spl beta/ as /spl gamma/ contracts to p. The sign of Gaussian curvature at p is determined by the relative orientations of the closed curves /spl gamma/ and /spl beta/. They directly compare the relative orientation of two such curves from intensity data. They employ three unknown illumination conditions to create a photometric scatter plot. This plot is in one-to-one correspondence with the subset of the unit Gaussian sphere containing the mutually illuminated surface normals. This permits direct computation of the sign of Gaussian curvature without the recovery of surface normals. Their method is albedo invariant. They assume diffuse reflectance, but the nature of the diffuse reflectance can be general and unknown. Simulations, as well as empirical results, demonstrate the accuracy of the technique.

Gaussian curvature is an invariant local descriptor of smooth surfaces. We present an object signature which is a condensed representation of the distribution of Gaussian curvature information at visible object points. An invariant related to Gaussian curvature at a point is derived from the covariance matrix of the photometric values in a neighborhood about that point. In addition, we introduce an albedo-normalization method that is capable of cancelling albedo on Lambertian surfaces. We use three illumination conditions, two of which are unknown. The three-tuple of intensity values at a point is related via a one-to-one mapping to the surface normal at that point. The determinant of the covariance matrix of the local three-tuples is invariant to albedo, rotation and translation. The collection of determinants over mutually illuminated object points is combined into a signature distribution which is albedo, rotation, translation, and scale invariant. An object recognition methodology using these signatures is proposed.

Generalized cylinders (GCs) are a popular representational tool in computer vision. In medical imaging, the curved axis GC is particularly applicable to a number of elongated physical structures such as vasculature, bone and bronchi. In many of these instances, it is necessary to recover curved-axis GCs with arbitrary cross-sections. It is also vital that these structures, once recovered, can be analyzed and visualized with off-the-shelf algorithms and software packages. Such tools are usually designed to operate on the domain of polynomial or rational surfaces. Unfortunately most extant, suitably versatile GC representations do not admit rational parameterizations. We develop an entirely rational B-spline representation for generalized cylinders with curved axes and arbitrary cross-section functions. We demonstrate how our representation can be used as a deformable model by extracting a rational GC from pre-segmented spinal data using a discrete dynamic surface fit.

We present a multiple illumination technique that directly recovers the viewer-centered curvature matrix, up to a scalar factor, at each mutually illuminated point on a smooth object surface. This technique is completely independent of knowledge of incident illumination orientation, local surface orientation, or diffuse surface albedo. The cornerstore of this technique is the use of theintegrability constraintwhich is a fundamental mathematical property of smooth surfaces. The integrability constraint allows the derivation of an equation at each object point, which is linear in terms of quantities involving the initially unknown parameters of incident illumination orientation. These quantities, which we call thegradient ratio constants, can be simultaneously solved for five or more equations arising from the same number of object points. We show that deriving these gradient ratio constants provides just enough calibration information about incident illumination geometry to compute the viewer-centered curvature matrix at each object point, up to a scalar multiple. We demonstrate three important applications of this technique: (i) segmentation of the object surface by sign of Gaussian curvature, (ii) further segmentation of nonnegative Gaussian curvature into convexity and concavity, and (iii) reconstruction of the surface height, independent of knowledge of incident illumination orientation and diffuse surface albedo.

We extract topological and local structural information from X-ray-computed tomography (CT) volume images of the vascular tree of the lung. We then produce an abstract model of the topology and geometry of the tree which is suitable for generating physical models. Physiologists model the vascular tree as a connected network of tubes and, to date, the only accurate data available to produce such models has come from dissection and measurement. Modeling the dynamic behavior of the tree in a living organism has been, until now, impossible. Building models from CT scans will increase the volume and quality of information available for both the study of physiology and diagnosis. We present an efficient, accurate analysis technique for volume images of tube networks. Using local operations that link techniques based on morphology to differential geometry, we transform the volume into a vector field which resembles idealized, axis-parallel fluid flow through the tube network. This field provides information to condense the salient features of the image into an augmented Euclidean minimum spanning tree (EMST). This augmented EMST proves to be a convenient and logical abstract representation of the vascular tree.

Overviewed are recent results of a new general approach to image understanding and computer vision utilizing the sensing of polarization of light. Whereas human vision is oblivious to components of light polarization, polarization parameters of light are shown to provide an important visual extension to intensity and color significantly expanding the application potential for image understanding. A physical state of polarization can be visualized directly in human terms as a particular hue and saturation, and this paper utilizes such a scheme presenting images of ordinary scenes as never seen before by humans in the domain of polarization vision. Metaphorically, humans are ‘color blind’ with respect to the perception of polarization, and even though this does not appear to inhibit human visual performance, we show how polarization vision is a sensory augmentation that can significantly enhance both automated image understanding and even possibly improve human visual performance itself under certain conditions. Sensors, calledpolarization cameras, have been developed that automatically sense components of partial linear polarization and computationally process these components to produce polarization images. Prototypes of different polarization camera sensors have been presented in earlier literature. A recent advancement in the design of polarization cameras has made it possible to interface low-cost modular components with almost any existing imaging device converting it into an automatic polarization camera. This compatibility with small portable imaging devices is making polarization imaging for the first time accessible to a number of application areas outside the laboratory, both outdoors and underwater, revealing polarization vision as a vast new visually augmented domain with unique capabilities. This paper presents various results from three on-going field applications: natural object recognition, inspection of ship hulls for damage, and marine biology.

We present a fully automated system which unites CCD camera technology with liquid crystal technology to create a polarization camera capable of sensing the partial linear polarization of reflected light from objects at pixel resolution. As polarization sensing not only measures intensity but also additional physical parameters of light, it can therefore provide a richer set of descriptive physical constraints for the understanding of images. Previously it has been shown that polarization cues can be used to perform dielectric/metal material identification, specular and diffuse reflection component analysis, as well as complex image segmentations that would be significantly more complicated or even infeasible using intensity and color alone. Such analysis has so far been done with a linear polarizer mechanically rotated in front of a CCD camera. The full automation of resolving polarization components using liquid crystals not only affords an elegant application, but significantly speeds up the sensing of polarization components and reduces the amount of optical distortion present in the wobbling of a mechanically rotating polarizer. In our system two twisted nematic liquid crystals are placed in front of a fixed linear polarizer placed in front of a CCD camera. The application of a series of electrical pulses to the liquid crystals in synchronization with the CCD camera video frame rate produces a controlled sequence of polarization component images that are stored and processed on Datacube boards. We present a scheme for mapping a partial linear polarization state measured at a pixel into hue, saturation and intensity producing a representation for a partial linear polarization image. Our polarization camera currently senses partial linear polarization and outputs such a color representation image at 5 Hz. The unique vision understanding capabilities of our polarization camera system are demonstrated with experimental results showing polarization-based dielectric/metal material classification, specular reflection and occluding contour segmentations in a fairly complex scene, and surface orientation constraints.

No abstract available.

A robust and accurate polarization phase-based technique for material classification is presented. The novelty of this technique is three-fold In (i) its theoretical development, (ii) its application, and, (iii) its experimental implementation. The concept of phase of polarization of a light wave is introduced to computer vision for discrimination between materials according to their intrinsic electrical conductivity, such as distinguishing conducting metals, and poorly-conducting dielectrics. Previous work has used intensity, color and polarization component ratios. This new method is based on the physical principle that metals retard orthogonal components of light upon reflection while dielectrics do not. This method has significant complementary advantages with respect to existing techniques, is computationally efficient, and can be easily implemented with existing imaging technology. Experiments for real circuit board inspection, non-conductive and conductive glass, and, outdoor object recognition have been performed to demonstrate its accuracy and potential capabilities.

Recently, polarization vision has been shown to simplify some important image understanding tasks that can be more difficult to perform with intensity vision alone. This, together with the more general capabilities of polarization vision for image understanding, motivates the building of camera sensors that automatically sense and process polarization information. Described in this paper are a variety of designs for polarization camera sensors that have been built to automatically sense partial linearly polarized light, and computationally process this sensed polarization information at pixel resolution to produce a visualization of reflected polarization from a scene, and/or a visualization of physical information in a scene directly related to sensed polarization. The three designs for polarization camera sensors presented utilize (i) serial acquisition of polarization components using liquid crystals, (ii) parallel acquisition of polarization components using a stereo pair of cameras and a polarizing beamsplitter, and (iii) a prototype photosensing chip with three scanlines, each scanline coated with a particular orientation of polarizing material. As the sensory input to polarization camera sensors subsumes that of standard intensity cameras, they can potentially significantly expand the application potential of computer vision. A number of images taken with polarization cameras are presented, showing potential applications to image understanding, object recognition, circuit board inspection and marine biology.

We present a multiple illumination technique that directly recovers the viewer-centered curvature matrix up to a scalar factor, at each mutually illuminated point on a smooth object surface. This technique is completely independent of knowledge of incident illumination orientation, local surface orientation, or diffuse surface albedo. The cornerstone of this technique is the use of the integrability constraint which is a fundamental mathematical property of smooth surfaces. The integrability constraint allows the derivation of an equation at each object point, which is linear in terms of quantities involving the initially unknown parameters of incident illumination orientation. These quantities, which we call the gradient ratio constants, can be simultaneously solved for from five or more equations arising from the same number of object points. We show that deriving these gradient ratio constants provides just enough calibration information about incident illumination geometry to compute the viewer-centered curvature matrix at each object point, up to a scalar multiple. We demonstrate two important applications of this technique: segmentation of the object surface by sign of Gaussian curvature, and further segmentation of non-negative Gaussian curvature into convexity and concavity.>

Inhomogeneous dielectric surfaces exhibit both diffuse and specular reflection components. While various reflection models have been proposed for both of these components, the prediction of the relative strengths of these components in computer vision and computer graphics has so far not had a strong physical motivation. We propose in this paper a reflectance model for combined diffuse and specular reflection from dielectric materials which involves purely physical parameters (i.e., no ad hoc weighting of specular and diffuse components). This reflectance model is used to predict the relative strength of diffuse and specular reflection components in terms of imaging geometry, dielectric surface parameters, and, solid angular extent of incident light. We derive lower bounds on the contrast ratio between a specularity and surrounding diffuse reflecting regions. These can be used effectively to rule out highly contrasting diffuse reflecting regions being misidentified as specularities under a number of conditions which can significantly aid intensity-based specularity detection methods, and in turn image understanding. The theoretical developments in this paper can be used to predict the photometric dynamic range of illuminated objects which can be essential to inspection methods in machine vision. The developments in this paper can also be used in computer graphics for the physically precise rendering of the relative strengths of specular and diffuse reflection from inhomogeneous dielectrics.>

We show how the more general capabilities of polarization vision can be used to significantly enhance the understanding of imaged scenes particularly at regions containing specular and mirror reflections, and occluding contours. Intensity-based methods are easily confounded by specular reflection. Specular mirror reflection of an object is indistinguishable from the object itself when using intensity and color, but is shown to be distinguishable when reflected polarization is analyzed. Polarization-based methods also enhance the identification of occluding contour edges, and regions that are in the vicinity of occluding contour edges. In past work local polarization constraints have been shown to accurately label specularities and occluding contours in simple scenes. In more complicated scenes under general conditions local polarization constraints are usually not sufficient to produce an accurate labeling of these phenomena. The main new idea presented in this paper is to extract much more additional information by enforcing physical consistency between neighboring local polarization constraints and propagating these constraints across pixel edges and regions with significant partial polarization. Initial labeling of pixels is produced from local polarization constraints at edges. We present well-defined physically motivated rules for relabeling pixels where physical inconsistencies result between neighboring local polarization constraints. Propagation rules allow physically consistent labeling of regions of significant partial polarization, and a refinement rule allows some more detailed labeling of edges. We include the previously unstudied case of specular reflection at/near occluding contours. In all 10 edge and region labels are generated. Experimental results on a complex scene is presented demonstrating the vast amount of information that can be extracted from these new physical principles of polarization vision.>

Gaussian curvature is an intrinsic local shape characteristic of a smooth object surface that is invariant to orientation of the object in 3-D space and viewpoint. Accurate determination of the sign of Gaussian curvature at each point on a smooth object surface (i.e., the identification of hyperbolic, elliptical and parabolic points) can provide very important information for both recognition of objects in automated vision tasks and manipulation of objects by a robot. We present a multiple illumination technique that directly identifies elliptical, hyperbolic, and parabolic points from diffuse reflection from a smooth object surface. This technique is based upon a photometric invariant involving the behavior of the image intensity gradient under varying illumination. The nature of this photometric invariant allows direct segmentation of a smooth object surface according to the sign of Gaussian curvature independent of knowledge of local surface orientation, independent of diffuse surface albedo, and with only approximate knowledge of the geometry of multiple incident illumination. In comparison with photometric stereo, this new technique determines the sign of Gaussian curvature directly from image features without having to derive local surface orientation, is invariant to incident orientation errors of two of three light sources, and is invariant to the relative strength of incident radiance with respect to each of these light sources. We demonstrate how this segmentation technique works under conditions of simulated image noise, and actual experimental imaging results.>

The natural light field has long been known to be partly linearly polarized, and it has been suggested that various objects reflect light that is polarized at a specific orientation. Although humans use polarized light, our inability to see it limits our study and understanding of its distribution in nature and of the information it carries. By placing two twisted nematic liquid crystals and a fixed polarizing filter in series in front of a CCD camera, we constructed a portable polarimeter that analyses the linear polarization characteristics of a full image on a single pixel basis. Two configurations are presented: an autonomous sensor that uses a small camcorder for recording images that are analyzed at a later stage; and an online sensor that uses a digital camera connected to a personal computer which controls and analyses the information. Field measurements reveal possible usage for analyzing spatial orientation of objects or breaking color camouflage. Our new polarimeters provide an opportunity to inspect and understand an aspect of the visual world, currently obscure to our eyes.

A methodology for accurate determination of surface normals and light source location from depth and reflectance data is introduced. Estimation of local surface orientation using depth data alone from range finders with standard depth errors can produce significant error, while shape-from-shading using reflectance data alone produces approximate surface orientation results that are highly dependent on the correct initial surface orientation estimates and regularization parameters. Combining these two sources of information gives vastly more accurate surface orientation estimates under general conditions than either one alone, and can also provide better knowledge of local curvature. Novel iterative methods that enforce satisfaction of the image irradiance equation and surface integrability without using regularization are proposed. These iterative methods work when the light source is any finite distance from the object, producing variable incident light orientation over the object.>

An intensity reflectance model is proposed for diffuse reflection originating from subsurface multiple scattering within inhomogeneous dielectric materials. One of the most common assumptions in computer vision from inhomogeneous dielectrics is Lambertian. The proposed model utilizes the subsurface intensity distribution predicted by modeling subsurface multiple scattering based upon radiative transfer theory. For an optically smooth surface boundary this subsurface intensity distribution becomes altered by Fresnel attenuation upon refraction into air, making it become significantly non-Lambertian. This resulting intensity distribution serves as a reflection law for individual optically smooth microfacets that compose an opaque rough surface, producing a physical unification of diffuse reflection with the Torrance-Sparrow specular reflection model. Experimental results that verify the accuracy of the model are presented. A relatively simple formula that approximates closely the true behaviour of diffuse reflection and can be used in place of a Lambertian term is presented. The non-Lambertian nature of the diffuse component becomes most significant near occluding contours and sometimes produces a previously unnoticed intensity effect that can aid in their detection.>

Presents a fully automated system which unites CCD camera technology with liquid crystal technology to create a polarization camera capable of sensing the polarization of reflected light from objects at pixel resolution. As polarization affords a more general physical description of light than does intensity, it can therefore provide a richer set of descriptive physical constraints for the understanding of images. The authors present a scheme for mapping polarization states into hue, saturation and intensity which is a very convenient representation for a polarization image. The polarization camera outputs such a color image which can then be used in polarization-based vision methods. The unique vision understanding capabilities of the polarization camera system are demonstrated with experimental results showing polarization-based dielectric/metal material classification, specular reflection and occluding contour segmentations in a fairly complex scene, and surface orientation constraints for object recognition.>

The authors present a physically based approach, using polarization, to distinguish three types of image edges; limb edges, specular edges, and albedo/physical edges. Assuming general imaging conditions and smooth dielectric surfaces, a labeling scheme which enables one to distinguish among these edge types has been developed. The method is demonstrated on laboratory images.>

The authors present a polarization reflectance model that uses the Fresnel reflection coefficients. This reflectance model accurately predicts the magnitudes of polarization components of reflected light, and all the polarization-based methods presented follow from this model. The authors demonstrate the capability of polarization-based methods to segment material surfaces according to varying levels of relative electrical conductivity, in particular distinguishing dielectrics, which are nonconducting, and metals, which are highly conductive. Polarization-based methods can provide cues for distinguishing different intensity-edge types arising from intrinsic light-dark or color variations, intensity edges caused by specularities, and intensity edges caused by occluding contours where the viewing direction becomes nearly orthogonal to surface normals. Analysis of reflected polarization components is also shown to enable the separation of diffuse and specular components of reflection, unobscuring intrinsic surface detail saturated by specular glare. Polarization-based methods used for constraining surface normals are discussed.>

Using rigorous methods the authors have shown that the invariant direction of isophotes at parabolic points is the projection of the principal direction of zero curvature onto the image plane. They have gone beyond considering the limited case of pure orthographic projection and have shown under what conditions this photometric invariance property breaks down and is best approximated in perspective images, and for a nondistant light source. In addition to giving experimental evidence supporting their claim about how isophotes are parallel to the direction of zero curvature at parabolic points, the authors suggest possible applications of this geometric cue to determination of orientation and curvature at parabolic points. The best advantage taken from this geometric cue is when constraint information can be obtained, knowing absolutely nothing about the reflectance map. The true power of a physical invariant is the information it gives under the most generic conditions possible, in this case any type of smooth reflectance map.>

A computationally simple yet powerful method for distinguishing metal and dielectric material surfaces from the polarization characteristics of specularly reflected light is introduced. The method is completely passive, requiring only the sensing of transmitted radiance of reflected light through a polarizing filter positioned in multiple orientations in front of a camera sensor. Precise positioning of lighting is not required. An advantage of using a polarization-based method for material classification is its immunity to color variations, which so commonly exist on uniform material samples. A simple polarization-reflectance model, called the Fresnel reflectance model, is developed. The fundamental assumptions are that the diffuse component of reflection is completely unpolarized and that the polarization state of the specular component of reflection is dictated by the Fresnel reflection coefficients. The material classification method presented results axiomatically from the Fresnel reflectance model, by estimating the polarization Fresnel ratio. No assumptions are required about the functional form of the diffuse and specular components of reflection. The method is demonstrated on some common objects consisting of metal and dielectric parts.>

A novel idea for the analysis of shape from reflectance maps is introduced. It is shown that local surface orientation and curvature constraints can be obtained at points on a smooth surface by computing the instantaneous rate of change of reflected scene radiance caused by angular variations in illumination geometry. The resulting instantaneous changes in image irradiance values across an optic sensing array of pixels constitute what is termed a photometric flow field. There is no correspondence problem with respect to obtaining the instantaneous change in image irradiance values between successive image frames. This is because the object and camera remain static relative to one another as the illumination geometry changes. Simulations show how photometric flow fields quantitatively determine local surface orientation from a known incident orientation of an illuminator, as well as determining incident illuminator orientation from a known local surface orientation.>

A technique is presented which utilizes the polarization properties of reflected light to separate specular and diffuse components of reflection. This technique works for both dielectric and metal surfaces, regardless of the color of the illuminating light source or the color detail on the object surface. In addition to separating out diffuse and specular components of reflection, the technique can also identify whether certain image regions correspond to a dielectric or metal object surface. Extensive experimentation is presented for a variety of dielectric and metal surfaces, both polished and rough, using a point light source.>

The author analyzes a stereo method which determines lines in space from the intersection of projected planar sheets. Object descriptions are built from information about linear features instead of by points. It is shown that there are major advantages to accurately determining the orientation of object lines and surfaces using this stereo method. In the absence of errors (apart from the baseline translation error), the measurement of the orientation of lines and surfaces from this stereo method is translation-invariant in the sense that the orientation measurement is completely independent of knowledge of the baseline. Computer simulations of realistic imaging configurations show that, even in the presence of errors from other camera parameters, this stereo method is nearly translation-invariant and can far outperform stereo methods based on the absolute correspondence of points. Another advantage of determining the orientation of lines and surfaces from stereo using intersecting planes is that orientation errors do not grow rapidly as the object distance from the baseline increases.>

A technique for identifying the material properties of objects in an image using multiple images taken through a polarizing lens at various rotations in front of a stationary camera (only the filter moves). Using these images, it is possible to obtain the classification of material surfaces at all points on a spectacular highlight. The algorithm is demonstrated on laboratory images. The authors assume a point source, the theory can only be applied at points where specular reflection dominates. Extensions of the theory to deal with extended light sources, which greatly increase the portion of the image giving rise to specular reflection, are also considered.>