We propose an algorithm which can jointly estimate camera pose and point set registration. Given point sets from two views of a stationary scene, our algorithm registers the point sets while retaining internal scene structure. It simultaneously ensures that the resultant registration is consistent with that of a moving camera viewing a static scene (adheres to some epipolar constraint). Our statistical formulation can incorporate but does not necessarily require additional constraints such as brightness constancy and high dimensional point descriptors such as SIFT. We show that our algorithm is stable over a variety of scenes and offers a pose from edge system which handles currently difficult structure from motion scenes more robustly.

In this work, video segmentation is viewed as an efficient intra-frame grouping temporally reinforced by a strong inter-frame coherence. Traditional approaches simply regard pixel motions as another prior in the MRF-MAP framework. Since pixel pre-grouping is inefficiently performed on every frame, the strong correlation between inter-frame groupings is largely underutilized. We exploit the inter-frame correlation to propagate trustworthy groupings from the previous frame. A preceding graph is constructed and labeled for the previous frame. It is temporally propagated to the current frame and validated by similarity measures. All unlabeled subgraphs are spatially aggregated for the final grouping. Experimental results show that the proposed approach is highly efficient for spatio-temporal segmentation. It makes good use of temporal correlation and produces satisfactory grouping results.

Traditional building height estimation from a stereo pair of satellite images relies heavily on the selection of building primitives by the human operator. It is labor-intensive and highly inefficient. In this work, we propose a system to automate the estimation of 3-D building height from an IKONOS stereo pair. Minimum user intervention is required to outline the building top in the left image. Stereo correspondence is automatically established through a multi-scale feature (coarse-to-fine) matching, which is capable of achieving sub-pixel accuracy. The height of the building top is computed using the Rational Polynomial Coefficient (RPC) model. Experimental results show that the proposed method is efficient, robust and it produces accurate building height estimation.

This paper addresses the problem of extracting the road region in different driving environments with dynamic lighting changes. Previous approaches using Gaussian mixture models (GMM) have fixed number of models constructed from sample color data and could not keep models associated with shadows. As a result, although they work in some specific environments, they fail in other environments or in scenes with shadows. In this paper, we propose a new vision-based approach where flexible number of models are built from sample data. Those color samples are reliably collected from stereo-verified ground patches inside a pre-defined trapezoidal learning region. After model construction, models associated with shadows and highlights are detected and maintained. The advantages of this approach with respect to other techniques are that it gives more robust results and, in particular, recognizes shadows on road as drivable road surface instead of non-road.

This paper addresses the problem of extracting the road region in different driving environments with dynamic lighting changes, for driver-assistance applications. In this paper, we propose a stereo visual sensor system and a vision-based road extraction method in a new color space. The color space is designed such that it is representative of intrinsic reflectance of the road surface, and independent of illumination source. Our basic model of road color is a mixture of Gaussians in that color space, constructed from road sample pixels. Those color samples are reliably collected from stereo-verified ground patches inside a pre-defined trapezoidal learning region. The advantages of this system with respect to other systems are that it is more economical for driver-assistance applications while giving robust results and, in particular, recognizing shadows on road as drivable road surface instead of non-road.

An image segmentation system is proposed for the segmentation of color image based on neural networks. In order to measure the color difference properly, image colors are represented in a modified L*u* v* color space. The segmentation system comprises unsupervised segmentation and supervised segmentation. The unsupervised segmentation is achieved by a two-level approach, i.e., color reduction and color clustering. In color reduction, image colors are projected into a small set of prototypes using self-organizing map (SOM) learning. In color clustering, simulated annealing (SA) seeks the optimal clusters from SOM prototypes. This two-level approach takes the advantages of SOM and SA, which can achieve the near-optimal segmentation with a low computational cost. The supervised segmentation involves color learning and pixel classification. In color learning, color prototype is defined to represent a spherical region in color space. A procedure of hierarchical prototype learning (HPL) is used to generate the different sizes of color prototypes from the sample of object colors. These color prototypes provide a good estimate for object colors. The image pixels are classified by the matching of color prototypes. The experimental results show that the system has the desired ability for the segmentation of color image in a variety of vision tasks.

This paper describes a video surveillance system capable of recognizing human postures from video sequences. The system comprises of two key modules: human detection and posture classification. In the module of human detection, human blobs are extracted by the technique of background subtraction. An adaptive background model is employed to characterize the dynamics and complexity of outdoor scenes based on the mixture of Gaussians. In order to formulate the variations of human postures, pseudo 2D hidden Markov models (P2DHMM) is employed for representing and recognizing human postures based on its '2-D elastic matching' property. It is trained to differentiate human postures and tolerate the variations of the same human posture using embedded Viterbi and segmental K means algorithms. In the classification of human postures, observation sequence is extracted from current image frame. The probabilities of observation sequence corresponding to each P2DHMM model are computed by doubly embedded Viterbi optimization, and human blob is classified as the human posture with the highest likelihood.