Automatically extracting object models from images is a complex task. We describe research in extracting 3D models of buildings from aerial images. This work has resulted in several related systems including assisted extraction (minimal manual interaction to guide automatic processing), automatic extraction with limited imagery and limited building models, and automatic extraction with very good imagery and digital elevation models and more complex building models. Some results are provided for the assisted system and one of the automatic systems.

While two-dimensional maps exist for most urban areas, the descriptions may be incomplete or out of date, or of insufficient resolution for the given application and features such a roads are not described as 3-D objects. Most of the past work on road detection has concentrated on either low resolution, primarily rural roads (usually producing "spaghetti" roads with no notion of intersections), high resolution road following without the topological information of the intersections, or pixel classification where there is no sense of the road as an object. This paper address the problem of extracting a street grid in an urban environment while maintaining the topological information of the intersections. Starting from an initial seed intersection, which gives the size and orientation of the expected grid, this system uses a feature-based hypothesis and verify paradigm to extract a 3-D description of the street grid. The verification uses the context provided by an intersection model and by an extended street model and other available sensors.

The development of a general motion estimation system based on using three or more frames is reported. The relevant portion of the sequence is treated as a whole, thus simplifying the process of estimating the motion parameters, and making possible the computation of the motion parameters in terms of the natural center of motion. A general description of this motion estimation method is presented, with the derivation of motion equations for three cases (three points in three frames, two points in four frames, and one point in five frames). Results are given for real and synthetic images for the case of one point in five (and six) frames. On the average, the program converged quickly to the right answer for noiseless data. For noisy data, the answers were reasonable and their accuracy was directly a function of the amount of noise in the disparity vector.>

Many textures can be described structurally, in terms of the individual textural elements and their spatial relationships. This paper describes a system to generate useful descriptions of natural textures in these terms. The basic approach is to determine an initial, partial description of the elements using edge features. This description controls the extraction of the texture elements. The elements are grouped by type, and spatial relationships between elements are computed. The descriptions are shown to be useful for recognition of the textures, and for reconstruction of periodic textures.

Many different relaxation schemes have been proposed for image analysis tasks. We have developed a general matching procedure for comparing semantic network descriptions of images, and we have implemented a variety of relaxation techniques. An automatic segmentation and description system is used to produce the image representations so that the matching procedures must cope with variations in feature values, missing objects, and possible multiple matches. This environment is used to test different relaxation matching schemes under a variety of conditions. The best performance (of those we compared), in terms of the number of iterations and the number of errors, is for the gradient-based optimization approach of Faugeras and Price. The related optimization approach of Hummel and Zucker performed almost as well, with differences primarily in difficult matches (i.e., where much of the evidence is against the match, for instance, poor segmentations). The product combination rule proposed by Peleg was extremely fast, indeed, too fast to work when global context is needed. The classical Rosenfeld, Hummel, and Zucker method is included for historical comparisons and performed only adequately, producing fewer correct matches and taking more iterations.

A technique for locating desired structures utilizing user specified information about properties of these structures and their relationships with other more easily extracted objects is described. An edge-based and region-based technique is used for scene segmentation. Experimental results of the processing of aerial pictures are presented.

This paper discusses the application of stochastic labeling to a general symbolic image description problem. A method used to compute initial likelihoods and compatibilities is described. It was derived from an earlier symbolic matching procedure, but was modified to provide the data needed for application of the labeling method. This labeling procedure differs from simpler ones, in that it minimizes a global criterion at each iteration. This technique is compared to other matching methods, and results on two scenes are presented.