Research at the Digital Mapping Laboratory has focused on the automated analysis of aerial imagery for cartographic feature extraction. However, it has long been the authors' belief that optimal performance in cartographic feature extraction can be obtained only by the combination, or fusion, of feature extraction systems which use differing information sources and processing methods. This paper describes experiments on the pairwise fusion of cartographic feature extraction systems; surface material maps obtained from the classification of hyper-spectral imagery, digital elevation models derived from stereo panchromatic imagery, and three-dimensional (3D) building hypotheses generated from single panchromatic images. Fusion experiments were performed on three test areas and detailed evaluations conducted. The results showed that using surface material or stereo information to focus processing of the building extraction system led to significantly better overall performance and runtimes. Utilizing building hypotheses to refine material classification showed mixed results, due partially to residual registration errors.

We present a method to generate three-dimensional building hypotheses starting from sparse features, hypothesized building corners, extracted from multiple views of the scene. This technique relies on knowledge about the imaging geometry and acquisition parameters to provide rigorous geometric constraints for the matching process. The effectiveness of this approach is demonstrated using complex aerial imagery taken with highly oblique views containing buildings with flat or peaked roofs.>

A feature-based stereo matching system that is based on an algorithm for one-dimensional waveform matching is described. It is intended for use in automated cartography, to generate an accurate three-dimensional model of man-made structures and natural terrain. Each epipolar line in the stereo pair is represented as a one-dimensional intensity waveform. The waveform is described as a collection of features, such as peaks and valleys, and represented across a set of hierarchical levels, computed by approximation from the original waveform. These features are matched using an evaluation function that factors similarity of waveform shape, intensity, and symbolic feature description. Waveform matches at coarse resolution are used to constrain matches at finer levels. Intra/inter-scanline corrections are applied and the actual position of the stereo match is adjusted by using the gradient representation of the original waveform. Some representative results are presented for a complex urban scene.>

The authors examine how estimates of three-dimensional scene structure, as encoded in a scene disparity map, can be improved by the analysis of the original monocular imagery. They describe the utilization of surface illumination information provided by the segmentation of the monocular image into fine surface patches of nearly homogeneous intensity to remove mismatches generated during stereo matching. These patches are used to guide a statistical analysis of the disparity map based on the assumption that such patches correspond closely with physical surfaces in the scene. Such a technique is quite independent of whether the initial disparity map was generated by automated area-based or feature-based stereo matching. Refinement results on complex urban scenes containing various man-made and natural features are presented, and the improvements due to monocular fusion with a set of different region-based image segmentations are demonstrated.>

A technique for producing a triangulated irregular network (TIN) from a digital elevation model (DEM) is described. The overall goal is to produce an approximate terrain description that preserves the major topographic features using a greatly reduced set of points selected from the original DEM. The TIN generation process is iterative; at each iteration, areas in the DEM that lie outside of a user-supplied error tolerance in the TIN are identified, and points are chosen from the DEM to more accurately model these areas. Point selection involves the computation of the difference between the actual DEM and an approximate DEM. This approximate DEM is calculated by interpolating elevation points from the TIN.>

Three major areas in the development of competent 3-D scene interpretation system are discussed. First, the importance of accurate automatic scene registration and the difficulty in automated extraction and matching of scene reference points are described. Second, the authors describe two stereo matching algorithms, S1, which is an area-based matcher previously used in the SPAM system, and S2, which is a feature-based matching algorithm based on hierarchical waveform matching. Third, the authors introduce several performance evaluation metrics that made it possible to measure the quality of the overall scene recovery, the building disparity estimate, and the quality and sharpness of the building delineations. Such manually generated scene reference models are critical for understanding strengths and weaknesses of various matching algorithms and in the incremental development of improvements to existing algorithms. Experiments were performed on difficult examples of aerial imagery.>

The authors describe a comprehensive stereo analysis system developed to recover the 3D description of an urban area using high-resolution aerial imagery. Given an area of interest in terms of geographic coverage, the system can automatically find the appropriate stereo pair using a spatial database, select control points to register the two images so that epipolar geometry is satisfied, and recover disparity information using two complementary matching techniques. It is believed that stereo mapping research must explicitly address error and uncertainty in both scene registration and stereo matching. In the registration process, five different features can be automatically extracted to select control points for fine image registration. In the stereo matching process two techniques are utilized, an area-based and a feature-based stereo matcher, to generate a disparity map for a scene. Results are shown. An algorithm for stereo matching based upon 1D waveform matching is described. Techniques for generating a rigorous performance analysis to compare stereo matching algorithms on the basis of a manually derived 3D ground truth segmentation are discussed.>

Large production systems (rule-based systems) continue to suffer from extremely slow execution which limits their utility in practical applications as well as in research settings. Most investigations in speeding up these systems have focused on match (or knowledge-search) parallelism. Although good speed-ups have been achieved in this process, these investigations have revealed the limitations on the total speed-up available from this source. This limited speed-up is insufficient to alleviate the problem of slow execution in large-scale production system implementations. Such large-scale systems are expected to increase as researchers develop increasingly more competent production systems.

The authors describe computational techniques for utilizing the relationship between shadows and man-made structures to aid in the automatic extraction of man-made structures from aerial imagery. Four methods are described that perform the prediction of structure shape, grouping of related structures, verification of individual structures, and structure height estimation. In each method the relationship between the structure and the shadows cast is exploited in a unique fashion. Key issues involve the accurate localization of the structure/shadow boundary and the shadow edge and attribution of shadow segments to structure hypothesis. Several examples are presented that show how each method is used within the task of building detection, delineation, and height estimation.>

A description is given of research in digital mapping and image understanding in the area of automated feature extraction from aerial imagery. The authors discuss a system for road tracking, ARF (A Road Follower), that uses multiple cooperative methods for extracting information about road location and structure from complex aerial imagery. This system is a multilevel architecture for image analysis that follows for cooperation among low-level processes and aggregation of information by high-level analysis components. Two low-level road tracking methods have been implemented: road-surface texture correlation and road-edge following. Each works independently to establish a model of the centerline of the road, its width, and other local properties.>

In this paper, we describe the organization of a rule-based system, SPAM, that uses map and domain-specific knowledge to interpret airport scenes. This research investigates the use of a rule-based system for the control of image processing and interpretation of results with respect to a world model, as well as the representation of the world model within an image/map database. We present results on the interpretation of a high-resolution airport scene wvhere the image segmentation has been performed by a human, and by a region-based image segmentation program. The results of the system's analysis is characterized by the labeling of individual regions in the image and the collection of these regions into consistent interpretations of the major components of an airport model. These interpretations are ranked on the basis of their overall spatial and structural consistency. Some evaluations based on the results from three evolutionary versions of SPAM are presented.

This paper describes BROWSE, an interactive raster image display facility which is a major component of a larger integrated Map Assisted Photo-interpretation System (MAPS), being developed as a prototype interactive aid for photo-interpretation. Application areas for this research include image cartography, land use studies and reconnaissance, as well as image database organization, storage, and retrieval.