Summary In the presence of eyelids and eyelashes movement, pupil dilation, poor lighting, blur due to movement during iris image acquisition, and factors that collectively cause distortion in the iris image, 2D image‐based iris identification techniques become limited and might lead to iris misclassification due to the dependency on the image appearance (texture) on the possibly distorted image. To alleviate this problem, we introduce a new 3D iris model and reader based upon which iris identification is performed. Using a set of at least two 2D images taken from different views, a small set of reliable and corresponding salient fiducial points (corner points taken from crypts, corona, and serpentine rings of iris pattern) in the two images are extracted, from which a set of 3D iris salient points are obtained using triangulation. Corresponding salient points in the 2D images are found using the Random Sampling Consensus (RANSAC) algorithm, which is robust in identifying the inlier points that correspond to each other in different views of the iris. From this small and reliable salient point set, a denser (high‐resolution) set of extra salient feature points is constructed at minimum cost using a loop subdivision method that yields corresponding extra salient points in the two images from which a high‐resolution 3D iris model is obtained. The points associated with the 3D high‐resolution model are tessellated to form a high‐resolution triangular mesh with the appearance of a triangular patch in the image imported onto the iris personalized 3D model rendering it a 3D iris surface with appearance. This 3D model construction method allows for a 3D classification of a test iris to one of many possible irises stored in the database. The classification is based on a geometric 3D point cloud error. For cross‐validation, the geometric‐based classification can be augmented by considering a mean squared error (MSE) based on the appearance of corresponding points on the test and base models to further disambiguate between irises with close geometric error values.

In this paper we derive novel surface fiducial points that are computed from the differential geometry of the surface. The fiducial intrinsic points are intrinsic, local, and relative invariants, i.e., they are preserved under similarity, affine, and nonlinear transformations that are piecewise affine. As the fiducial points are computed from high order surface shape derivatives, their sensitivity to any noise either due to measurement error or local distortion is high. To reduce these effects, we use a B-Spline curve/surface representation that smoothes out the curve/surface prior to the computation of these intrinsic invariant points. The fiducial points are used in a non-iterative geometric-based method for 3D shape matching and registration of human faces. The matching is achieved by establishing correspondences between fiducial points after a sorting based on a set of absolute local affine invariants derived from them. The performance of the matching based on these fiducial points although shown for face alignment, the overall approach is generic to the alignment a variety of objects for which these intrinsic fiducial points exist and for scenarios where the classes of nonlinear transformations are piecewise affine.

We describe a novel approach for creating a three-dimensional (3-D) face structure from multiple image views of a human face taken at a priori unknown poses by appropriately morphing a generic 3-D face. A cubic explicit polynomial in 3-D is used to morph a generic face into the specific face structure. The 3-D face structure allows for accurate pose estimation as well as the synthesis of virtual images to be matched with a test image for face identification. The estimation of a 3-D person's face and pose estimation is achieved through the use of a distance map metric. This distance map residual error (geometric-based face classifier) and the image intensity residual error are fused in identifying a person in the database from one or more arbitrary image view(s). Experimental results are shown on simulated data in the presence of noise, as well as for images of real faces, and promising results are obtained.

We describe a novel approach for creating a 3D face structure from multiple image views of a human face taken at a priori unknown poses by appropriately morphing a generic 3D face. A 3D cubic explicit polynomial is used to morph a generic face into the specific face structure. This allows the creation of a database of 3D faces that is used in identifying a person (in the database) from one or more arbitrary image view(s). The estimation of a 3D person's face and its recognition from the database of faces is achieved through the use of a distance map metric. The use of this metric avoids either resorting to the formidable task of establishing feature point correspondences in the image views, or even more severely, relying on the extremely view-sensitive image intensity (texture). Experimental results are shown for images of real faces, and excellent results are obtained.

Surface representation is intrinsic to many applications in medical imaging, computer vision, and computer graphics. We present a method that is based on surface modeling by B-spline. The B-spline constructs a smooth surface that best fits a set of scattered unordered 3D range data points obtained from either a structured light system (a range finder), or from point coordinates on the external contours of a set of surface sections, as for example in histological coronal brain sections. B-spline stands as of one the most efficient surface representations. It possesses many properties such as boundedness, continuity, local shape controllability, and invariance to affine transformations that makes it very suitable and attractive for surface representation. Despite its attractive properties, however, B-spline has not been widely applied for representing a 3D scattered nonordered data set. This may be due to the problem in finding an ordering and a choice for the topological parameters of the B-spline that lead to a physically meaningful surface parameterization based on the scattered data set. The parameters needed for the B-spline surface construction, as well as finding the ordering of the data points, are calculated based on the geodesics of the surface extended Gaussian map. The set of control points is analytically calculated by solving a minimum mean square error problem for best surface fitting. For a noise immune modeling, we elect to use an approximating rather than an interpolating B-spline. We also examine ways of making the B-spline fitting technique robust to local deformation and noise.

This paper is concerned with the problem of full or partial alignment of surfaces in the presence of affine transformations, local deformation and noise. This work addresses many alignment problems in diverse areas such as face recognition, fusion of multi-modality (e.g., evoked potential and MRI) for in vivo alignment of metabolic with anatomical maps, and brain mapping. In this paper, we concentrate on brain mapping and consider the intra and inter-animal brain surface registration problems. Surface alignment is based on a set of local affine invariants derived from the set of ordered inflection point pairs that reside on the umbilical curves of the surface. These points are local intrinsic geometric surface landmarks that are sought after because they are preserved under the affine transformation. The method is illustrated for intra and inter-animal registration using 3D data sets obtained from a sequence of external contours of coronal sections.

A framework for deriving a class of new global affine invariants for both object matching and positioning based on a novel concept of cross-weighted moments with fractional weights is presented. The fractional weight factor allows for a more flexible range to balance between the capability to discriminate between objects that differ only in small shape details and the sensitivity of small shape details to the presence of the noise. Moreover, it makes it possible to arrive at low order (zero order) affine invariants that are more robust than those derived from higher order regular moments. The affine transformation parameters are recovered from the zero and the first order cross-weighted moments without requiring any feature point correspondence information. The equations used to find the affine transformation parameters are linear algebraic. The sensitivity of the cross-weighted moment invariants to noise, missing data, and perspective effects is shown on real images.

This paper is concerned with the problem of feature point registration and scene recognition from images under weak perspective transformations which are well approximated by affine transformations and under possible occlusion and/or appearance of new objects. It presents a set of local absolute affine invariants derived from the convex hull of scattered feature points (e.g., fiducial or marking points, corner points, inflection points, etc.) extracted from the image. The affine invariants are constructed from the areas of the triangles formed by connecting three vertices among a set of four consecutive vertices (quadruplets) of the convex hull, and hence do make direct use of the area invariance property associated with the affine transformation. Because they are locally constructed, they are very well suited to handle the occlusion and/or appearance of new objects. These invariants are used to establish the correspondences between the convex hull vertices of a test image with a reference image in order to undo the affine transformation between them. A point matching approach for recognition follows this. The time complexity for registering L feature points on the test image with N feature points of the reference image is of order O(N x L). The method has been tested on real indoor and outdoor images and performs well.

A new approach is proposed for registering a set of histological coronal two-dimensional images of a rat brain sectional material with coronal sections of a three-dimensional brain atlas, an intrinsic step and a significant challenge to current efforts in brain mapping and multimodal fusion of experimental data. The alignment problem is based on matching external contours of the brain sections, and operates in the presence of tissue distortion and tears which are routinely encountered, and possible scale, rotation, and shear changes (the affine and weak perspective groups). It is based on a novel set of local absolute affine invariants derived from the set of ordered inflection points on the external contour represented by a cubic B-spline curve. The inflection points are local intrinsic geometric features, which are preserved under both the affine and the weak perspective transformations. The invariants are constructed from the sequence of area patches bounded by the contour and the line connecting two consecutive inflection points, and hence do make direct use of the area (volume) invariance property associated with the affine transformation. These local absolute invariants are very well suited to handle the tissue distortion and tears (occlusion problem).

The article deals with the problem of matching and recognizing planar curves that are modeled by B-splines, independently of possible affine transformations to which the original curve has been subjected (for example, rotation, translation, scaling, orthographic, and semiperspective projections), and possible occlusion. It presents a fast algorithm for estimating the B-spline control points that is robust to nonuniform sampling, noise, and local deformations. Curve matching is achieved by using a similarity measure based on the B-spline knot points introduced by Cohen et al. (1991). This method, however, can neither handle the affine transformation between the curves nor the occlusion. Solutions to these two problems are presented through the use of a new class of weighted B-spline curve moments that are well defined for both open and closed curves. The method has been applied to classifying affine-transformed aircraft silhouettes, and appears to perform well.

This paper deals with the problem of detecting and estimating the scatterer spacing between the regularly-spaced resolvable coherent scatterers found in tissue. This parameter has been successfully used in classifying tissue structure, in differentiating between normal and cirrhotic liver, and in detecting diffuse liver disease. This paper presents a Wold decomposition of the radio frequency (RF) field into its diffused and coherent components from which maximum likelihood estimates (MLE) or minimum mean square error (MMSE) estimates of the scattering spacing is easily computed.

There have been many techniques for curve shape representation and analysis, ranging from Fourier descriptors, to moments, to implicit polynomials, to differential geometry features, to time series models, to B-splines, etc. The B-splines stand as one of the most efficient curve (surface) representations and possess very attractive properties such as spatial uniqueness, boundedness and continuity, local shape controllability, and invariance to affine transformations. These properties made them very attractive for curve representation, and consequently, they have been extensively used in computer-aided design and computer graphics. Very little work, however, has been devoted to them for recognition purposes. One possible reason might be due to the fact that the B-spline curve is not uniquely described by a single set of parameters (control points), which made the curve matching (recognition) process difficult when comparing the respective parameters of the curves to be matched. This paper is an attempt to find matching solutions despite this limitation, and as such, it deals the problem of using B-splines for shape recognition and identification from curves, with an emphasis on the following applications: affine invariant matching and classification of 2-D curves with applications in identification of aircraft types based on image silhouettes and writer-identification based on handwritten text.

This paper deals with the problem of matching and recognizing planar curves which are modeled as B-splines. The paper presents a fast algorithm for estimating the control points of the B-spline which is robust to nonuniform sampling and noise. Curve matching is achieved by using the similarity measure based on the knot points associated with the B-spline curves. This method, however, can not handle the affine transformation and/or occlusion between the curves. This paper presents solutions to these two problems through the use of a new class of weighted B-spline curve moments that results in (1) a closed-form solution for the affine transformation parameters; (2) the starting and ending points of the segment on the prototype curve that was affine transformed to yield the observed curve. The method is used for classifying affine transformed silhouette of aircrafts, and appears to perform well.>

This paper and its companion are concerned with the problems of 3-D object recognition and shape estimation from image curves using a 3-D object curve model that is invariant to affine transformation onto the image space, and a binocular stereo imaging system. The objects of interest here are the ones that have markings (e.g., characters, letters, special drawings and symbols, etc.) on their surfaces. The 3-D curves on the object are modeled as B-splines, which are characterized by a set of parameters (the control points) from which the 3-D curve can be totally generated. The B-splines are invariant under affine transformations. That means that the affine projected object curve onto the image space is a B-spline whose control points are related to the object control points through the affine transformation. Part I deals with issues relating to the curve modeling process. In particular, the authors address the problems of estimating the control points from the data curve, and of deciding on the "best" order B-spline and the "best" number of control points to be used to model the image or object curve(s). A minimum mean-square error (mmse) estimation technique which is invariant to affine transformations is presented as a noniterative, simple, and fast approach for control point estimation. The "best" B-spline is decided upon using a Bayesian selection rule. Finally, we present a matching algorithm that allocates a sample curve to one of p prototype curves when the sample curve is an a priori unknown affine transformation of one of the prototype curves stored in the data base. The approach is tried on a variety of images of real objects.>

For pt. I, see ibid., p.1-12 (1994). This paper is the second part of a 3-D object recognition and shape estimation system that identifies particular objects by recognizing the special markings (text, symbols, drawings, etc.) on their surfaces. The shape of the object is identified from the image curves using B-spline curve modeling as described in Part I, as well as a binocular stereo imaging system. This is achieved by first estimating the 3-D control points from the corresponding curves in each image in the stereo imaging system. From the 3-D control points, the 3-D object curves are generated, and these are subsequently used for estimating the 3-D surface parameters. A Bayesian framework is used for classifying the image into one of c possible surfaces based on the extracted 3-D object curves. This is complemented by a neural network (NN) that recognizes the surface as a particular object (e.g., a Pepsi can versus a peanut butter jar), by reading the text/markings on the surface. To reduce the amount of training the NN has to undergo for recognition, the object curves are "unwarped" into planar curves before the matching process. This eliminates the need for templates that are surface shape dependent and results in a planar curve that might be a rotated, translated, and scaled version of the template. Hence, for the matching process we need to use measures that are invariant to these transformations. One such measure is the Fourier descriptors (FD) derived from the control points associated with the unwarped parent curves. The approach is tried on a variety of images of real objects and appears to hold great promise.>

The problem of tracking a moving object having markings on its surface in front of a static camera is addressed. In this case, the motion analysis reduces to the problem of tracking these markings from frame to frame. The markings are modeled by B-splines. The problem of establishing the correspondence between the image markings points in the various frames is bypassed by relating the B-spline parameters of the same marking viewed at different frames. Implicit in this relationship are the motion parameters. The complexity of the problem is significantly reduced by first acquiring the 3-D object curve structure before the usual motion estimation.>

The problem of extracting the local shape information of a 3-D texture surface from a single 2-D image by tracking the perceived systematic deformations the texture undergoes by virtue of being present on a 3-D surface and by virtue of being imaged is examined. The surfaces of interest are planar and developable surfaces. The textured objects are viewed as originating by laying a rubber planar sheet with a homogeneous parent texture on it onto the objects. The homogeneous planar parent texture is modeled by a stationary Gaussian Markov random field (GMRF). A probability distribution function for the texture data obtained by projecting the planar parent texture under a linear camera model is derived, which is an explicit function of the parent GMRF parameters, the surface shape parameters. and the camera geometry. The surface shape parameter estimation is posed as a maximum likelihood estimation problem. A stereo-windows concept is introduced to obtain a unique and consistent parent texture from the image data that, under appropriate transformations, yields the observed texture in the image. The theory is substantiated by experiments on synthesized as well as real images of textured surfaces.>

The problem of recognizing and estimating the shape of objects with special markings (text, symbols, drawings, etc.) on their surfaces using B-spline curve modeling and a pair of binocular images is considered. As a direct consequence of the invariance of the B-splines to affine transformations the computation of the 3-D coordinates of the object curve points from a pair of stereo images becomes a straightforward task, and is obtained using minimum-mean-square-error estimation. The 3-D curve is recovered from the estimated object curve control points, and is subsequently used for the object shape estimation and classification.>

The problem of extracting the local shape information of a 3D textured surface from a single 2D image is addressed. The textured objects of interest are planar and developable surfaces that are viewed as originating by laying down a rubber planar sheet with a homogeneous parent texture on it onto the objects. The homogeneous planar parent texture is modeled by a stationary Gaussian Markov random field (GMRF). The probability density function of the projected planar parent texture is an explicit function of the parent GMRF parameters, the surface shape parameters, and the camera geometry. The surface shape parameter estimation is posed as a maximum-likelihood estimation problem. A stereo-windows concept is introduced to obtain a unique and consistent parent texture from the image data.>

The B-spline stands as one of the most efficient curve (surface) representation, and possesses very attractive properties such as spatial uniqueness, boundedness and continuity, local shape controllability, and invariance to affine transformations. These properties made them very attractive for curve representation in computer aided design and computer graphics. Very little work, however, has been devoted to them for recognition purpose. One possible reason might be due to the fact that the B-spline curve is not uniquely described by a single set of control points, which make the curve matching (recognition) process not a simple comparison between the respective parameters of the curves to be matched. The paper is an attempt to find matching solutions despite this limitation and addresses the problems of invariant matching and classification of 2D closed curves with application in identification of aircraft types based on image silhouettes, and writer-identification based on hand written text.>

The authors discuss the problem of textile fabric inspection using the visual textural properties of the fabric. The problem is to detect and locate the various kinds of defects that might be present in a given fabric sample based on an image of the fabric. Stochastic models are used to model the visual fabric texture. The authors use the Gaussian Markov random field to model the texture image of nondefective fabric. The inspection problem is cast as a statistical hypothesis testing problem on statistics derived from the model. The image of the fabric patch to be inspected is partitioned into nonoverlapping windows of size N*N where each window is classified as defective or nondefective based on a likelihood ratio test of size alpha . The test is recast in terms of the sufficient statistics associated with the model parameters. The sufficient statistics are easily computable for any sample. The authors generalize the test when the model parameters of the fabric are assumed to be unknown.>

Consideration is given to the problem of classifying a test textured image that is obtained from one of C possible parent texture classes, after possibly applying unknown rotation and scale changes to the parent texture. The training texture images (parent classes) are modeled by Gaussian Markov random fields (GMRFs). To classify a rotated and scaled test texture, the rotation and scale changes are incorporated in the texture model through an appropriate transformation of the power spectral density of the GMRF. For the rotated and scaled image, a bona fide likelihood function that shows the explicit dependence of the likelihood function on the GMRF parameters, as well as on the rotation and scale parameters, is derived. Although, in general, the scaled and/or rotated texture does not correspond to a finite-order GMRF, it is possible nonetheless to write down a likelihood function for the image data. The likelihood function of the discrete Fourier transform of the image data corresponds to that of a white nonstationary Gaussian random field, with the variance at each pixel (i,j) being a known function of the rotation, the scale, the GMRF model parameters, and (i,j). The variance is an explicit function of the appropriately sampled power spectral density of the GMRF. The estimation of the rotation and scale parameters is performed in the frequency domain by maximizing the likelihood function associated with the discrete Fourier transform of the image data. Cramer-Rao error bounds on the scale and rotation estimates are easily computed. A modified Bayes decision rule is used to classify a given test image into one of C possible texture classes. The classification power of the method is demonstrated through experimental results on natural textures from the Brodatz album.>

A unifying decision-theoretic model-based approach is presented for solving a broad range of vision problems. These include 3-D scene (outdoor and indoor) segmentation of a 2-D image, 3-D surface recognition and shape and position estimation from one or more images, and tracking of a moving camera from a sequence of images of fixed scenes. The image associated with a 3-D surface patch is locally approximated by either a homogeneous Markov random field (MRF) texture model, which is specified by a few parameters having unknown values, by parameterized contour curves having a few unknown parameters, or by other simply parameterized models. The least structured model considered consists of the expectation at each pixel of a single image treated as a completely arbitrary a priori unknown parameter, thus modeling every possible image but requiring a huge number of parameters. 3-D surfaces are modeled as functions described by a priori unknown parameters, ranging from a few to many. To provide a direct link between the image data and the 3-D surface that generates it, the 3-D surface parameters, the camera geometry, the scene lighting, and the image model parameters are related. Because of this linking, 3-D shape recognition, location and orientation estimation, and scene segmentation are possible and can be easily formulated as optimal detection and estimation problems.>

The authors are concerned with the problems of detecting the presence and tracking the unknown, time-varying instantaneous frequencies of nonoverlapping linear or nonlinear FM chirp signals embedded in noise and overlapping linear FM chirp signals embedded in noise. No prior knowledge is assumed about the signal parameters, or when the signal changes its parameters in time, or the number of signals present. For both the overlapping and nonoverlapping cases, the authors analyze the Wigner distribution (WD) of the received signals s(t). The WD of many FM chirp signals is highly concentrated above a 2-D curve in the time-frequency plane that corresponds with the signal's instantaneous frequency. The contours that are produced by properly thresholding the WD are generalized lines in the ( omega , t, t/sup 2/) space. Hence, the tracking problem for both cases reduces to the simpler problem of tracking generalized lines, and is done using unsupervised weighted maximum-likelihood clustering, and minimum-mean-square estimation.>

The problem of classifying a textured image which might be subject to unknown rotation and magnification scale changes into one of C possible texture classes is discussed. The texture classes are modeled by Gaussian Markov random fields. A Bayes decision rule based on the generalized likelihood function is used to classify a given test sample. A maximum-likelihood estimate for the scale and rotation parameters for each of the C texture classes is computed under the assumption that the observed texture came from a particular unrotated and unscaled texture model. The test texture is allocated to the class with the highest generalized likelihood function. The classification power of the method is demonstrated through extensive experimental results on natural texture from the Brodatz album as well as for the problem of fabric inspection.>

The problem of segmenting a range of image into homogeneous regions, in each of which the range data corresponds to a different surface, is discussed. A maximum-likelihood (ML) segmentation is sought. As most manufactured parts are well approximated by patches of planes, cylinders, and spheres, these three simple surfaces are considered to be the only surfaces in the image. The basic approach to segmentation is to divide the range image into windows, classify each window as a particular surface primitive, and group like windows into surface regions. Grouping windows of the same surface types is cast as a weighted ML clustering problem. Mixed windows are segmented using a ML hierarchical segmentation algorithm. The resulting regions and their associated ML surface parameter and boundary estimates can then be used to perform ML object position estimation and matching. A similar approach is taken for segmenting visible-light images of Lambertian objects illuminated by a point source at infinity.>

The problem of segmenting a range image into homogeneous regions in each of which the range data correspond to a different surface is considered. The segmentation sought is a maximum-likelihood (ML) segmentation. Only planes, cylinders, and spheres are considered as presented in the image. The basic approach to segmentation is to divide the range image into windows, classify each window as a particular surface primitive, and group like windows into surface regions. Mixed windows are detected by testing the hypothesis that a window is homogeneous. Homogeneous windows are classified according to a generalized likelihood ratio test which is computationally simple and incorporates information from adjacent windows. Grouping windows of the same surface types is cast as a weighted ML clustering problem. Finally, mixed windows are segmented using an ML hierarchical segmentation algorithm. A similar approach is taken for segmenting visible-light images of Lambertian objects illuminated by a point source at infinity.>

The modeling and segmentation of images by MRF's (Markov random fields) is treated. These are two-dimensional noncausal Markovian stochastic processes. Two conceptually new algorithms are presented for segmenting textured images into regions in each of which the data are modeled as one of C MRF's. The algorithms are designed to operate in real time when implemented on new parallel computer architectures that can be built with present technology. A doubly stochastic representation is used in image modeling. Here, a Gaussian MRF is used to model textures in visible light and infrared images, and an autobinary (or autoternary, etc.) MRF to model a priori information about the local geometry of textured image regions. For image segmentation, the true texture class regions are treated either as a priori completely unknown or as a realization of a binary (or ternary, etc.) MRF. In the former case, image segmentation is realized as true maximum likelihood estimation. In the latter case, it is realized as true maximum a posteriori likelihood segmentation. In addition to providing a mathematically correct means for introducing geometric structure, the autobinary (or ternary, etc.) MRF can be used in a generative mode to generate image geometries and artificial images, and such simulations constitute a very powerful tool for studying the effects of these models and the appropriate choice of model parameters. The first segmentation algorithm is hierarchical and uses a pyramid-like structure in new ways that exploit the mutual dependencies among disjoint pieces of a textured region.

A new approach to segmentation of range image data into quadric surfaces is treated here. The segmentation consists of three stages. First, an initial segmentation produces coarse surface regions classed as plane, cylinder, or sphere. The algorithm operates on contours, or 2D slices, of the data rather than the 3D range image data. Second, constrained surface fitting is used to obtain surface parameter estimates for each segmented region. And third, a high accuracy segmentation results in both a high resolution boundary between surface regions and accurate quadric surface parameters. The algorithm is hierarchical and works in stages from low to high image resolution while updating surface parameter estimates at each stage. Experimental results with synthetic and real range images are given.

This paper presents a suboptimal boundary estimation algorithm for noisy images which is based upon an optimal maximum likelihood problem formulation. Both the maximum likelihood formulation and the resulting algorithm are described in detail, and computational results are given. In addition, the potential power of the likelihood formulation is demonstrated through the presentation of three simple but insightful analyses of algorithm performance. These analyses are based on a technique we have developed for comparing the accuracies of different boundary finding algorithms. This technique also helps in understanding the interplay of object shape and data models in the relative performances of boundary finders. Some of the algorithm design considerations resulting from the use of our analysis technique are new and, at first, surprising. Our technique appears to be the only one developed for comparing the accuracies of different boundary finding algorithms.

This paper formulates the problem of object boundary estimation in noisy black and white images as a state sequence estimation problem for a discrete-time Markov process. Based upon this formulation and the resulting likelihood function a suboptimal estimation algorithm is developed for elliptically shaped boundaries. Appropriate transition probabilities are calculated by developing a dynamic model for boundary generation and implementing a Kalman filter.