This paper formulates shape registration as an optimal coding problem. It employs a set of landmarks to establish the correspondence between shapes, and assumes that the best correspondence can be achieved when the polygons formed by the landmarks optimally code all the shape contours, i.e., obtain their minimum description length (MDL). This is different from previous MDL-based shape registration methods, which code the landmark locations. In this paper, each contour is discretized to be a set of points to make the coding feasible, and a number of strategies are adopted to tackle the difficult optimization problem involved. The resulting algorithm, called CAP, is able to yield statistical shape model with better quality in terms of model generalization error, which is demonstrated on both synthetic and biomedical shapes.

This paper presents a novel and simple method of analyzing the motion of a large image sequence captured by a calibrated outward-looking video camera moving on a circular trajectory for large-scale environment applications. Previous circular motion algorithms mainly focus on inward-looking turntable-like setups. They are not suitable for outward-looking motion where the conic trajectory of corresponding points degenerates to straight lines. The circular motion of a calibrated camera essentially has only one unknown rotation angle for each frame. The motion recovery for the entire sequence computes only one fundamental matrix of a pair of frames to extract the angular motion of the pair using Laguerre's formula and then propagates the computation of the unknown rotation angles to the other frames by tracking one point over at least three frames. Finally, a maximum-likelihood estimation is developed for the optimization of the whole sequence. Extensive experiments demonstrate the validity of the method and the feasibility of the application in image-based rendering.

This paper presents a novel texture and shape priors based method for kidney segmentation in ultrasound (US) images. Texture features are extracted by applying a bank of Gabor filters on test images through a two-sided convolution strategy. The texture model is constructed via estimating the parameters of a set of mixtures of half-planed Gaussians using the expectation-maximization method. Through this texture model, the texture similarities of areas around the segmenting curve are measured in the inside and outside regions, respectively. We also present an iterative segmentation framework to combine the texture measures into the parametric shape model proposed by Leventon and Faugeras. Segmentation is implemented by calculating the parameters of the shape model to minimize a novel energy function. The goal of this energy function is to partition the test image into two regions, the inside one with high texture similarity and low texture variance, and the outside one with high texture variance. The effectiveness of this method is demonstrated through experimental results on both natural images and US data compared with other image segmentation methods and manual segmentation.

An autonomous agent response learning (AARL) algorithm is presented in this paper. We proposed to decompose the award function into a set of local award functions. By optimizing this objective function set, the response function with maximum award can be determined. To tackle the optimization problem, a modified particle swarm optimization (PSO) called "multi-species PSO (MS-PSO)" is introduced by considering each objective function as a specie swarm. Two sets of experiments are provided to illustrate the performance of MS-PSO. The results show that it returns a more accurate response set within shorter duration by comparing with other PSO methods.

This paper presents a real-time framework to achieve accurate deformation compensation using intraoperative ultrasound (US) with preoperative magnetic resonance imaging (MRI). It includes a shape prior based segmentation process and a warping computation. Given two different module templates of a region of interest (ROI), the method can extract the same ROI from new real-time images based on the shape prior model in the level set space. Then the segmentation result and the matching transformation are used to simulate deformation on MR images. Through experimental results, we show that this novel segmentation method can find the shape of ROI accurately. It can provide a good assistance for real-time simulation of organ deformation.

In robot applications, the performance of a robot agent is measured by the award received from its response. A lot of literature defines the response as either a state diagram or a neural network. Due to the absence of a desired response, neither is applicable to an unstructured environment. In this paper, a novel response knowledge learning algorithm is proposed to handle this domain. By using a set of experiences, the algorithm can extract the contributed experiences to construct the response function. Two sets of environments are provided to illustrate the performance of the proposed algorithm. The results show that it can effectively construct a response function that receives an award which is very close to the true maximum.

A robust image enlargement algorithm is presented in this paper. We formulate the image enlargement process as an edge information estimation process. In order to achieve a higher resolution, we first perform pixel duplication [W.K. Pratt, 1991] on the target image to form an initial high resolution image. Then the edge details of the enlarged image are estimated by using a novel neural network called "agent swarm regression network ASRN", which is trained by a set of low resolution (LR) / high resolution (HR) image patch pairs. Two benchmark images were used to verify the performance of the proposed algorithm. The results show that the enlarged images by the proposed algorithm are sharper than those by the conventional methods.

Circular motion or single axis motion is widely used in computer vision and graphics for 3D model acquisition. This paper describes a new and simple method for recovering the geometry of uncalibrated circular motion from a minimal set of only two points in four images. This problem has been previously solved using nonminimal data either by computing the fundamental matrix and trifocal tensor in three images or by fitting conics to tracked points in five or more images. It is first established that two sets of tracked points in different images under circular motion for two distinct space points are related by a homography. Then, we compute a plane homography from a minimal two points in four images. After that, we show that the unique pair of complex conjugate eigenvectors of this homography are the image of the circular points of the parallel planes of the circular motion. Subsequently, all other motion and structure parameters are computed from this homography in a straighforward manner. The experiments on real image sequences demonstrate the simplicity, accuracy, and robustness of the new method.

We describe a new and simple method of recovering the geometry of uncalibrated circular motion or single axis motion using a minimal data set of 2 points in 4 images. This problem has been solved using nonminimal data either by computing the fundamental matrix and trifocal tensor in 3 images, or by fitting conics to tracked points in 5 images. Our new method first computes a planar homography from a minimum of 2 points in 4 images. It is shown that two eigenvectors of this homography are the images of the circular points. Then, other fixed image entities and rotation angles can be straightforwardly computed. The crux of the method lies in relating this planar homography from two different points to a homology naturally induced by corresponding points on different conic loci from a circular motion. The experiments on real image sequences demonstrate the simplicity, accuracy and robustness of the new method.

Previous algorithms for recovering 3D geometry from an uncalibrated image sequence of a single axis motion of unknown rotation angles are mainly based on the computation of two-view fundamental matrices and three-view trifocal tensors. We propose three new methods that are based on fitting a conic locus to corresponding image points over multiple views. The main advantage is that determining only five parameters of a conic from one corresponding point over at least five views is simpler and more robust than determining a fundamental matrix from two views or a trifocal tensor from three views. It is shown that the geometry of single axis motion can be recovered either by computing one conic locus and one fundamental matrix or by computing at least two conic loci. A maximum likelihood solution based on this parametrization of the single axis motion is also described for optimal estimation using three or more loci. The experiments on real image sequences demonstrate the simplicity, accuracy, and robustness of the new methods.

An automatic algorithm to extract accurate eyebrow contours in a human face image is proposed. Based on the extracted eye corners, upper eye boundaries and the position points of the eyebrows, a feature region is limited to a local searching area. Separating the local searching area into several sub-areas and performing K-means clustering in each sub-area, we can obtain a rough region for each eyebrow. The rough contour obtained by tracking the rough eyebrow region is defined as an initial snake. Then the accurate eyebrow contour can be extracted by our improved snake method. Some prior knowledge about the shape of the object is used in both clustering and the snake method. Lastly, an application using the extracted eyebrow contours for face recognition is presented. The experimental results show that our method can detect the eyebrow contours with high accuracy and the eyebrow contour is useful for face recognition.

Image transfer is a method for projecting a 3D scene from two or more reference images. Typically, the correspondences of target points to be transferred and the reference points must be known over the reference images. We present two new transfer methods that eliminate the target point correspondence requirement. We show that five reference points matched across two reference images are sufficient to linearly resolve transfer under affine projection using two views instead of three views as needed by other techniques. Furthermore, given the correspondences of any four of the five reference points in any other view, we can transfer a target point to a third view from any one of the two original reference views. To improve the robustness of the affine projection method, we incorporate an orthographic camera model. A factorization method is applied to the reference points matched over two reference views. Experiments with real image sequences demonstrate the application of both methods for motion tracking.

In this paper, we propose a new approach for recovering 3D geometry from uncalibrated and unknown rotation angles of single axis motions. Unlike previous approaches, the computation of the trifocal tensor is not required. The new approach is based on fitting a conic to the corresponding points in different images of the same space point. It is then shown that the essential geometry of single axis motion is encoded by a conic. Since we are determining only 5 parameters from N views instead of 18 parameters from a subsequence of 3 views, our approach is much more simple and robust. The experiments on a real image sequence demonstrate the accuracy and robustness of the new algorithm.

Relief mosaics are collections of registered images that extend traditional mosaics by supporting motion parallax. A simple parallax interpolation algorithm based on computed correspondence information allows high quality blur-free and ghost-free mosaics to be created using images from moving hand-held cameras that would not be suitable for traditional mosaicing. The renderer can also display local parallax changes, giving a local but visually convincing illusion of depth. Moreover, relief mosaics can be used for approximate plenoptic modeling from hand-held cameras at lower spatial sampling rates than existing light-field methods. We present a fully automatic correspondence based construction system for relief mosaics, and show how they can be used in applications.

For 3D model based video conferencing, estimation of the rigid motions of the human head and the non-rigid motions of the face are usually required. In this paper, we focus on the image motions corresponding to the rigid motions of the human head on the video images. We discovered that the image motions in image i can be computed through estimating a non-rigid transformation matrix between the pose 1 and pose i. A novel two stage algorithm for estimating the transform is proposed. It assumes at least four 3D control points are estimated by stereo vision from the first pose, and the images of these 4 points are detected in image i. Then this transform can be estimated by minimizing the back projection errors of the 4 image points. This is more robust and accurate than methods based on estimating the rigid motions. To verify our method, a 3D wireframe model was used to model a human head for the simulated video image sequence.

Gesture, as a "natural" means, provides an alternative way for human-computer interaction. The recognition of continuous gestures suffers greatly from the existence of non-gesture hand motions. The given gestures can start at any moment in an input sequence. The Hidden Markov model (HMM) is used to tackle this problem. The paper proposes a method for the spotting and recognition of continuous spatio-temporal features. Without sliding the input temporal patterns past the trained models, the algorithm makes use of accumulation scores for evaluation. So it is an exhaustive evaluation method but only a sum operation is needed in each input frame. The method is demonstrated with real experiments on the recognition of some spatio-temporal trajectories. Results of the experiments show that the proposed method is very effective and fast in extracting given gestures from a continuous trajectory containing non-gestures.

We investigate the linear self-calibration method proposed by Newsam et al. (1996) for our project on 3D reconstruction of architectural buildings. This self-calibration method assumes that the principal point is known, the camera has square pixels and has no skew. It allows 3D shape to be reconstructed from two images while giving the camera the freedom to vary its focal length. In this paper, we evaluate the focal lengths obtained from their method with both synthetic data and real data. In real data where known 3D data are available, Tsai's (1987) calibration method is used for comparison. Our experimental results show that the focal lengths from the two methods differed by less than 5% and the reconstructed 3D shape was very good in that angles were well preserved.

In this paper, a novel image analysis system based on attributed relational graph matching is proposed, which is called accumulative Hopfield matching. We first divide the scene graph into many sub-graphs, and a modified Hopfield network is then constructed to obtain the sub-graph isomorphism between each sub-scene graph and model graph. The final result is deduced by accumulating the solutions of all small subnetworks. The proposed system was applied to automatic object recognition and modeling of articulated objects with good results.

A camera under general motions allows the fixed geometric entities: the absolute conic and the plane at infinity to be determined uniquely for self-calibration. For turntable motions, these two geometric entities cannot be determined, and will lead to ambiguities in 3D reconstruction. However, some fixed points and lines will arise as a result of turntable motions, and can be related to the calibration matrix. We have made two contributions for self-calibration under turntable motion. First, a detailed theoretical analysis of degeneracy is given, and the degrees of the resulting ambiguities are deduced. Second, from some constraints on the physical characteristics of the camera the ambiguities can be removed up to a single parameter family. Experiments on real turntable image sequences verified our findings which are used also to perform a 3D reconstruction.

A camera moving in a plane can often simplify a computer vision job. Camera self-calibration and robot self-location are good examples. We focus on the problem of camera planar motion and its application to the camera self-calibration method of Faugeras et al. (1998). We have made three new contributions to the camera planar motion detection. First, we prove that the trifocal lines in different views of the same planar motion must have the same line representation in the 2D retinal plane. This conclusion greatly simplifies the planar motion detection problem. Second, we distinguish the usage of three different cases of planar motion: ordinary planar motion, co-linear planar motion and rotation planar motion. Third, we propose the robust planar motion detection method and the method of estimation of trifocal lines in the uniform framework under the above three configurations. We have also purposed a method for eliminating the 2D image points whose 1D projection points are inaccurate and cause significant errors on the estimation of the 1D trifocal tensor. Experiments with our new techniques using simulated data and real images had obtained very good results, which are better than those reported in the above article.

We improve the robustness of a singular value decomposition method to compute the relative positions between two calibrated perspective cameras. The first one is an optimal step to constrain the essential matrix E to have two equal non-zero and one zero singular values in the presence of noise, which is the sufficient condition for E to be factored as a rotation matrix R and translation vector t. The other contribution is that we have found 4 new possible solutions of R and t to the relative positions of two cameras, which have not been reported in any other SVD methods. Furthermore, these 8 possible solutions are derived directly from the 8 feasible SVD decompositions. Based on the experiments on both simulation data and real images, this method performs very well and the estimation error of R and t are almost at the same level as the noise.

Existing fractal methods of texture analysis rely on the fractal dimension of textures as a function of scale for their discrimination and classification. We propose a method which is based on the possible multiscaling/multifractality of textures. A stochastic model is suggested to represent this multiscaling behaviour. We demonstrate the value of the method on a number of similar data sets (hence quite difficult for their discrimination) from the high-resolution Brodatz album.

The macro features, such as triangle, quadrilateral, polygon, play a very important role in many computer vision applications such as matching, visual inspection, object tracking. However, effective ways to extract such macro features from images are still not available in the literature up to now. Worse, there are few reports on the matter. The paper proposes a randomized Hough technique to directly extract general triangles (i.e., with unknown size and orientation) from images. Extensive simulations as well as experiments with real images show that the results are satisfactory.

The Hough transform has been a widely used technique for geometric primitive extraction. However, recently, a new family of techniques based on optimization, such as the genetic algorithm, the tabu search algorithm, the algorithm based on random samples of minimum subset, claimed their superiority over the Hough transform. In this paper, based on a reasonable criterion, namely the expected number of random samples of minimum subset for a single successful primitive extraction, the performance of the two families of technique is compared. We show that the Hough transform generally outperforms optimization based techniques. In particular, based on a large number of simulations and experiments with real images, we show that with a comparable performance, the randomized Hough transform (RHT), a representative of Hough techniques, is about twice as fast as the random sample consensus (RANSAC), a representative of optimization based techniques, in both line extraction and circle extraction.

An arbitrary object and its image in a plane mirror constitute a bilaterally symmetric structure. When the structure is viewed by a single camera, the correspondence between the object and its counterpart in the mirror is auto-epipolar in one image taken by the camera and can be computed in a way similar to that for stereo vision. The 3D structure of the object is therefore recoverable from the single image. In this paper, a 3D-reconstruction algorithm derived from this theory is verified using different objects in front of a mirror observed by a calibrated camera. Very good experimental results are presented. Remarks on tackling the limitation of the present scheme are also given.

In this paper two new feature tracking algorithms are proposed. In the first algorithm, a perspective camera model is used. Making use of the projective invariant of Barrett, and assuming the image feature points corresponding to 8 general points in space are tracked by a conventional method in the image sequence, the other feature points in the sequence can be tracked using a Hough technique. Correspondence between two reference images as required by the original Barrett's invariant is not necessary. In the second algorithm, an affine camera model is assumed and the image feature points corresponding to 4 non-coplanar points in space are assumed tracked in the image sequence using a conventional method. These image points form the basis of affine coordinates in each image. After the correspondence of a fifth point is established between the first two images, the affine coordinates of all image points in the first images existence can be computed. As far as we know this is the only algorithm which can transfer a point knowing only a single image. Experiments showed that both algorithms gave highly accurate tracking results.

In this paper a robust approach of shape recovery from one color image for non-Lambertian surfaces is presented. A dichromatic reflection model is assumed to describe the color hybrid surface. An extended light source model is used so that both specular and diffuse reflection components can be detected simultaneously in one image. The color of the light source is assumed different from that of the object. This is a very mild restriction which can almost be satisfied. Using color information, regions of the image with hybrid reflection can be segmented from regions with only diffuse reflection. A new reflectance map for non-Lambertian surface is developed so that surface shape can be inferred using the shape from shading technique. For regions with only diffuse reflection, the conventional reflectance map for Lambertian surface can be used. Some results based on real experiments and simulations are presented to show the validity of our method.

We propose a new method to recover the 3D shape and surface reflectance of objects by color image analysis. We introduce a new lighting system with an extended light source moving in a plane which contains the optical axis of the camera. An image sequence is acquired with each image taken at a different light source position. A dichromatic reflection model is assumed The diffuse and specular reflection components are separated by a pseudoinverse technique. A fast generic algorithm is then used for the shape and reflectance recovery from the disuse reflection component extracted from a pair of color image sequences. As a result, the normal of a surface patch nor in the light source moving plane can also be estimated robustly. Our method is verified by both real and simulated experiments.

This paper describes a new technique for determining the initial location and orientation of an indoor mobile robot by geometric hashing and model-based scene matching. The indoor scenes are first modelled off-line in hashing tables using geometric invariants of some special model edge points. Model-based scene matching is completed in three stages. (1) extraction of scene image features and derivation of the the geometric invariants from these features in robot-centered coordinates, (2) finding the best matched candidate and the corresponding transformation using geometric hashing and a weighted distance voting criterion, and (3) the verification of the match by back-projection of model features onto the intensity image of the scene. Matching efficiency is increased substantially by restricting the number of bases through the ordering of image features and the concurrence of features during geometric hashing. Experimental results show that the proposed method is quite efficient and reliable.

Scale-space filtering is the only known method which provides a hierarchic signal description method by extracting features across a continuum of scales. One of its important characteristics is that it demands the filtering involved does not create generic features as the scale increases. It has been shown that the Gaussian filter is unique in holding this remarkable property. This is in essence the so-called scaling theorem. In this paper, we propose two scaling theorems for band-limited signals. They are applicable to a broader class of signals and a bigger family of filtering kernels than in Babaud et al. (1986), Yuille et al. (1986) and Wu-Xie (1990). An in-depth discussion of our theorems and the previously published ones is also given.

A.P. Witkin (1981) described a method of shape-from-texture using tangent vectors of texture line segments by assuming directional isotropy of the vectors. However, the texture of natural scenes is often nonisotropic with detectable bias. The authors propose a maximum likelihood estimate for recovering surface orientation from surfaces with nonisotropic texture. A priori information of the bias is used to achieve the objective. Experiments using artificial and real data show that the method is far better than Witkin's method for nonisotropic textures.>