This article describes an investigation of the source of geospatial error in digital surface models (DSMs) constructed from multiple satellite images. In this study, the uncertainty in surface geometry is separated into two spatial components: global error that affects the absolute position of the surface and local error that varies from surface point to surface point. The global error component is caused by inaccuracy in the satellite imaging process, mainly due to uncertainty in the satellite position and orientation (pose) during image collection. The key sources of local error are lack of surface appearance texture, shadows, and occlusion. These conditions prevent successful matches between the corresponding points in the images of a stereo pair. A key result of the investigation is a new algorithm for determining the absolute geoposition of the DSM that reflects the pose covariance of each satellite during image collection. This covariance information is used to weigh the evidence from each image in the computation of the global position of the DSM. The use of covariance information significantly decreases the overall uncertainty in global position and results in a 3-D covariance matrix for the global accuracy of the DSM. This covariance matrix defines a confidence ellipsoid within which the actual error must reside. Moreover, the absolute geoposition of each image is refined to the reduced uncertainty derived from the weighted evidence from the entire image set. This article also describes an approach to the prediction of local error in the DSM surface. The observed variance in surface position within a single stereo surface reconstruction defines the local horizontal error. The variance in the fused set of elevations from multiple stereo pairs at a single DSM location defines the local vertical error. These accuracy predictions are compared to ground truth provided by light detection and ranging (LiDAR) scans of the same geographic region of interest. The prediction of global and local error is compared to the actual errors for several geographic locations and mixes of satellite type. The predicted error bounds contain the observed errors according to the allowed percentage of outliers.

This paper presents the innovative SatTel framework, designed to automatically access, collate, process, and exploit commercial satellite imagery from a wide variety of vendors. Established vendors such as DigitalGlobe provide high resolution imagery with limited coverage, while disruptive vendors such as Planet and BlackSky provide low resolution imagery with near global coverage. SatTel provides a single point of entry for exploitation of these contrasting and complementary vendor capabilities. The authors illustrate the value of the SatTel framework via demonstrative change detection capabilities. SatTel change detection from small satellite imagery based on comparison of image to image appearance achieves mean average precision (MAP) above 0.75 for many sites compared to ground truth analyst annotation. SatTel change detection from high resolution satellite imagery based on multidimensional geometric structures achieves an average precision of 0.84 for elevation changes above 3.0 meters compared to ground truth analyst annotation.

This paper presents a probabilistic volumetric framework for image based modeling of general dynamic 3-d scenes. The framework is targeted towards high quality modeling of complex scenes evolving over thousands of frames. Extensive storage and computational resources are required in processing large scale space-time (4-d) data. Existing methods typically store separate 3-d models at each time step and do not address such limitations. A novel 4-d representation is proposed that adaptively subdivides in space and time to explain the appearance of 3-d dynamic surfaces. This representation is shown to achieve compression of 4-d data and provide efficient spatio-temporal processing. The advances of the proposed framework is demonstrated on standard datasets using free-viewpoint video and 3-d tracking applications.

The JPEG2000 image coding standard provides many superior features compared to JPEG and other compression standards. However, the relatively slow performance of JPEG2000, especially in software implementations, is a critical drawback of the standard. Moreover, as image sizes rapidly grow in size, higher demands on performance for image coding and processing are introduced, making the slow performance of JPEG2000 even further pronounced. While much effort over the past decade has been devoted to accelerating the JPEG2000 encoder, there have been very few studies focusing on improving the performance of the JPEG2000 decoder, despite the fact that the performance of the decoder is just as critical as the encoder. This paper proposes a high-performance JPEG2000 decoder that efficiently exploits the recent improvements of modern parallel programming models and hardware architectures. Specifically, a parallel streaming decoder running on a GPGPU-CPU heterogeneous system is developed to fully exploit the flexibility of the high-performance multi-core CPUs and the massively parallel capability of GPGPUs. In addition, a new task scheduling strategy is developed that exploits the soft-heterogeneity in OpenCL and C/C++ at runtime in order to gain a significant performance boost. Running on a heterogeneous configuration of one Nvidia GTX 480 GPU and one Intel Core i7 CPU, the parallel streaming decoder gains more than 8X speedup in runtime compared to the JasPer JPEG2000 software implementation.

This paper presents the first performance evaluation of local shape descriptors in probabilistic volumetric models (PVM) that are learned from multi-view aerial imagery of large scale urban scenes. The PVM offers a dense solution to the multi-view stereo problem, handling in a probabilistic manner the ambiguities caused by highly reflective surfaces, varying illumination conditions, registration errors, and sensor noise. A GPUbased octree implementation guarantees scalability of the PVM to large urban models, encouraging its application to higher level 3-d computer vision tasks. Furthermore, local descriptors form the basis of many shape-based applications and their performance has been studied extensively for images-based applications. Local descriptors have also been popular for 3-d shape understanding, but most 3-d descriptors have been used within the context of point clouds data collected with range sensors or polygonal meshes of CAD models. This work presents an investigation of the performance of several local shape descriptors in the PVM, which is learned from image data and where surfaces ambiguities are common and explicitly modeled. Descriptors are evaluated on accuracy for object classification using Bag-of-Words models. This evaluation forms a step towards the characterization of a new 3-d probabilistic representation for 3-d scene understanding.

Nanoscale circuits operating at sub-threshold voltages are affected by growing impact of random telegraph signal (RTS) and thermal noise. Given the low operational voltages and subsequently lower noise margins, these noise phenomena are capable of changing the value of some of the nodes in the circuit, compromising the reliability of the computation. We propose a method for improving noise-tolerance by selectively applying feed-forward reinforcement to circuits based on use of existing invariant relationships. As reinforcement mechanism, we used a modification of the standard CMOS gates based on the Schmitt trigger circuit. SPICE simulations show our solution offers better noise immunity than both standard CMOS and fully reinforced circuits, with limited area and power overhead.

Given a set of high-resolution images of a scene, it is often desirable to predict the scene's appearance from viewpoints not present in the original data for purposes of change detection. When significant 3-D relief is present, a model of the scene geometry is necessary for accurate prediction to determine surface visibility relationships. In the absence of an a priori high-resolution model (such as those provided by LIDAR), scene geometry can be estimated from the imagery itself. These estimates, however, cannot, in general, be exact due to uncertainties and ambiguities present in image data. For this reason, probabilistic scene models and reconstruction algorithms are ideal due to their inherent ability to predict scene appearance while taking into account such uncertainties and ambiguities. Unfortunately, existing data structures used for probabilistic reconstruction do not scale well to large and complex scenes, primarily due to their dependence on large 3-D voxel arrays. The work presented in this paper generalizes previous probabilistic 3-D models in such a way that multiple orders of magnitude savings in storage are possible, making high-resolution change detection of large-scale scenes from high-resolution aerial and satellite imagery possible. Specifically, the inherent dependence on a discrete array of uniformly sized voxels is removed through the derivation of a probabilistic model which represents uncertain geometry as a density field, allowing implementations to efficiently sample the volume in a nonuniform fashion.

In automated surveillance, one is often interested in tracking road vehicles, measuring their shape in 3D world space, and determining vehicle classification. To address these tasks simultaneously, an effective approach is the constrained alignment of a prior model of 3D vehicle shape to images. Previous 3D vehicle models are either generic but overly simple or rigid and overly complex. Rigid models represent exactly one vehicle design, so a large collection is needed. A single generic model can deform to a wide variety of shapes, but those shapes have been far too primitive. This paper uses a generic 3D vehicle model that deforms to match a wide variety of passenger vehicles. It is adjustable in complexity between the two extremes. The model is aligned to images by predicting and matching image intensity edges. Novel algorithms are presented for fitting models to multiple still images and simultaneous tracking while estimating shape in video. Experiments compare the proposed model to simple generic models in accuracy and reliability of 3D shape recovery from images and tracking in video. Standard techniques for classification are also used to compare the models. The proposed model outperforms the existing simple models at each task.

Power consumption requirements drive CMOS scaling to ever lower supply voltages, reducing the stability margin with respect to thermal noise and raising the probability for thermally-induced soft errors. Given the long time scale of noise-induced soft errors, conventional Monte Carlo simulations cannot be used to predict error rates and alternative approaches are needed. In this paper, the analysis of thermal fluctuations in a CMOS flip-flop is performed using a 2D queue that maps the available configurations for the flip-flop in terms of electron populations on the two inverters, with the two stable logic states at the opposite corners of the 2D matrix. Trial simulations for model systems show that the thermally-induced logic transitions involve only a limited number of states immediately above and below the main diagonal of the full 2D queue. We present a numerical solution based on variable precision arithmetic for a truncated 2D queue consisting of a variable number of near-diagonal states. It is shown that increasing the width of the near-diagonal queue, an accurate solution for the error rate is asymptotically obtained without the need to consider the full 2D queue. Our approach is used to calculate the mean time to failure of flip-flops built in a 45-nm fully-depleted silicon-on-insulator (FD-SOI) technology modeled in the subthreshold regime, including parasitics. As a predictive tool, the framework can be used to investigate the thermal stability of devices built in future technologies and as a measure of device reliability in VLSI design.

Over the last several years, a new probabilistic representation for 3-d volumetric modeling has been developed. The main purpose of the model is to detect deviations from the normal appearance and geometry of the scene, i.e. change detection. In this paper, the model is utilized to characterize changes in the scene as vehicles. In the training stage, a compositional part hierarchy is learned to represent the geometry of Gaussian intensity extrema primitives exhibited by vehicles. In the test stage, the learned compositional model produces vehicle detections. Vehicle recognition performance is measured on low-resolution satellite imagery and detection accuracy is significantly improved over the initial change map given by the 3-d volumetric model. A PCA-based Bayesian recognition algorithm is implemented for comparison, which exhibits worse performance than the proposed method.

In traffic surveillance applications a good prior model of vehicle shape and appearance is becoming increasingly more important for tracking, shape recovery, and recognition from video. The usefulness of 2-d vehicle models is limited to a fixed viewing direction; 3-d models are nearly always more suitable. Existing 3-d vehicle models are either generic but far too simple to utilize high resolution imagery, or far too complex and limited to specific vehicle instances. This paper presents a deformable vehicle model that spans these two extremes. The model is constructed with a multi-resolution approach to fit various image resolutions. At each resolution, a small number of parameters controls the deformation to accurately represent a wide variety of passenger vehicles. The parameters control both 3-d shape and appearance of parts that deform in the 2-d manifold of the vehicle surface. These parts are regions representing windows, headlights, taillights, etc. The combination of part boundaries and surface occluding contours account for the most consistent edges observed in images of vehicles. It is shown that the model parameters can be recovered by fitting the deformable model to real images of vehicles.

Aerial video registration is traditionally performed using 2-d transforms in the image space. For scenes with large 3-d relief, this approach causes parallax motions which may be detrimental to image processing and vision algorithms further down the pipeline. A novel, automatic, and online video registration system is proposed which renders the scene from a fixed viewpoint, eliminating motion parallax from the registered video. The 3-d scene is represented with a probabilistic voxel model, and camera pose at each frame is estimated using an Extended Kalman Filter and a refinement procedure based on a popular visual servoing technique. 1

This paper presents a new normalcy model of a scene for change detection using images taken from multiple views and varying illumination conditions. Each coregistered pixel site is statistically modeled by a probability distribution conditioned on a set of pixels in a non-local neighborhood that are less likely to be affected by a change that happens at the pixel of interest. These ldquonon-compact neighborsrdquo are located using information theoretic approaches. The associated change detection algorithm is called non-compact Markovian Likelihood (NorMaL), which predicts normalcy of a scene based on non-compact neighborhoods using non-parametric conditional density estimation.

This paper introduces a 3-d representation of vehicles as a space of scale and orientation transformations that define the shape of individual vehicle instances. This shape space forms a group, where the similarity of different vehicle observations can be evaluated using a distance measure defined by Lie group theory. A generic class of vehicles (e.g. SUV) is represented by a set of curves on the Lie group manifold, called geodesics. The classification of any given vehicle instance is achieved by finding the class with the smallest Lie distance between the geodesics and the vehicle shape. Vehicle recognition is carried out on 3-d LIDAR point clouds. The performance of the Lie classifier is evaluated against two other approaches and found to provide superior recognition performance, particularly with respect to the ability to generalize from a small number of labeled prototypes.

This paper examines the problem of detecting changes in a 3-d scene from a sequence of images, taken by cameras with arbitrary but known pose. No prior knowledge of the state of normal appearance and geometry of object surfaces is assumed, and abnormal changes can occur in any image of the sequence. To the authors' knowledge, this paper is the first to address the change detection problem in such a general framework. Existing change detection algorithms that exploit multiple image viewpoints typically can detect only motion changes or assume a planar world geometry which cannot cope effectively with appearance changes due to occlusion and un-modeled 3-d scene geometry (ego-motion parallax). The approach presented here can manage the complications of unknown and sometimes changing world surfaces by maintaining a 3-d voxel-based model, where probability distributions for surface occupancy and image appearance are stored in each voxel. The probability distributions at each voxel are continuously updated as new images are received. The key question of convergence of this joint estimation problem is answered by a formal proof based on realistic assumptions about the nature of real world scenes. A series of experiments are presented that evaluate change detection accuracy under laboratory-controlled conditions as well as aerial reconnaissance scenarios.-

As CMOS technology downscales, higher noise levels, wider threshold variation, and low supply voltage will force designers to contend with high rates of soft logical errors and many defective devices. A probabilistic design framework based on Markov random fields (MRF) has been previously proposed to address dynamic fault and noise vulnerability of ultimate digital CMOS circuitry. The idea is to use additional transistors and feedback loops to achieve significant noise immunity and ensure correct logic operations at low VDD. However, the extra reliability achieved in previously published work came at a cost of high transistor counts. In this paper, the authors present techniques to reduce the transistor count of larger multilevel combinational circuits built within the MRF framework by using variable sharing, implied dependence and supergates. Using these techniques the authors show an average reduction of approximately 28% in transistor counts over a range of combinational benchmark circuits built within the MRF framework compared to the best previously published results

We propose an approach to model the background of images in a video sequence based on subpixel edge map. This work is motivated by the observation that intensity based background models are sensitive to changes in illumination and camera parameters, e.g., gain control. In addition, the false positive rate is higher due to accidental alignment of figure intensities with the background model. Background models of edge maps, however, are more localized and thus reduce the likelihood of accidental alignment. We argue that the discretization error in pixel-level background models is also responsible for some of the false positives and develop a method based on subpixel edges whose background is thus highly selective. This method models the edge position and orientation using a Mixture of Gaussians model. This approach has been tested on a wide range of videos and the resulting background models are a much more selective figure-ground segregation.

Vehicle tracking in video sequences is typically carried out by matching 2-d views (images or features) of the vehicle from one frame to the next. These views are adapted by gradual changes in 2-d image transforms between frames. This approach can work well for sequences where the vehicle projection is not highly perspective and 3-d vehicle orientation with respect to the camera varies slowly. In this paper, vehicle tracking is studied under conditions that violate these assumptions. Tracking is carried out on high-definition videos sequences of vehicles that pass near the camera, causing severe perspective distortion. Perspective effects are accounted for by tracking edge curves on the vehicle and reconstructing them in 3-d, using structure from motion on adjacent frames. The result is segmentation of the vehicle from the background and recovery of its geometry and motion in 3-d.

This paper presents a novel algorithm for simultaneous background appearance modeling and coarse-scale vehicle recognition in traffic surveillance applications. 3-d mesh models representing a small set of vehicle classes are used to the hypothesize image segmentations into background, shadow, and vehicle regions. The algorithm optimizes vehicle class and motion parameters to best agree with a Hidden Markov Model for the image appearance. The best hypothesis, combined with image data, is used to adapt the parameters of the appearance model. Experiments on real video show that an appearance model trained in this way performs almost as well as one trained using manually segmented images. 1

Shape is an important cue for generic object recognition but can be insufficient without other cues such as object appearance. We explore a number of ways in which the geometric aspects of an object can be augmented with its appearance. The main idea is to construct a dense correspondence between the interior regions of two shapes based on a shape-based correspondence so that the intensity and gradient distributions can be compared, e.g., using a mutual information paradigm. Three methods for regional alignment are suggested and compared here, based on: (i) propagation of correspondences from the silhouette to parallel curves in the interior, (ii) intersection of line segments anchored on corresponding points on the contour, and (iii) correspondence of shape skeletons. These methods have been implemented and applied to vehicle category recognition from aerial videos under known viewing and illumination conditions. We have constructed a photo-realistic synthetic video database to explore the performance of these methods under controlled conditions. We have also tested these algorithms on real video collected for this purpose from a balloon. Our findings indicate that (i) augmenting shape with appearance significantly increases recognition rate, and (ii) the region correspondence induced by the shape skeleton yields the highest performance.

As devices are scaled to the nanoscale regime, it is clear that future nanodevices will be plagued by higher soft error rates and reduced noise margins. Traditional implementations of error correcting codes (ECC) can add to the reliability of systems but can be ineffective in highly noisy operating conditions. This paper proposes an implementation of ECC based on the theory of Markov random fields (MRF). The MRF probabilistic model is mapped onto CMOS circuitry, using feedback between transistors to reinforce the correct joint probability of valid logical states. We show that our MRF approach provides superior noise immunity for memory systems that operate under highly noisy conditions

As CMOS devices and operating voltages are scaled down, noise and defective devices will impact the reliability of digital circuits. Probabilistic computing compatible with CMOS offers a possible solution. In this work, we present a new area and power efficient design methodology for the implementation of a probabilistic framework into CMOS technology based on Markov Random Fields (MRF). Using SPICE, we simulate elementary logic components and sample circuits from the MCNC'91 benchmark set and show the area and power benefits compared to older MRF mapping strategies. We also extend our area and power efficient approach to improving the design of a Hamming decoder based on MRF principles.

As Si CMOS devices are scaled down into the nanoscale regime, current computer architecture approaches are reaching their practical limits. Future nano-architectures will confront devices and interconnections with a large number of inherent defects, which motivates the search for new architectural paradigms. In this paper, we examine probabilistic-based design methodologies for nanoscale computer architectures based on Markov random fields (MRF). The MRF approach can express arbitrary logic circuits and the logic operation is achieved by maximizing the probability of correct state configurations in the logic network depending on the interaction of neighboring circuit nodes. The computation proceeds via probabilistic propagation of states through the circuit. Crucially, the MRF logic can be implemented in modified CMOS-based circuitry that trades off circuit area and operation speed for the crucial fault tolerance and noise immunity. This paper builds on the recent demonstration that significant immunity to faulty individual devices or dynamically occurring signal errors can be achieved by the propagation of state probabilities over an MRF network. In particular, we are interested in CMOS-based circuits that work reliably at very low supply voltages (VDD = 0.1–0.2 V), where standard CMOS would fail due to thermal and crosstalk noise, and transistor threshold variation. In this paper, we present results for simulated probabilistic test circuits for elementary logic components and well as small circuits taken from the MCNC91 benchmark suite and we show greatly improved noise immunity operating at very low VDD. The MRF framework extends to all levels of a design, where formally optimum probabilistic computation can be implemented as a natural element of the processing structure.

We present a complete set of perceptual observables that provides a unified image description for grouping, figure-ground separation, and texture analysis. Although much progress has been made recently in treating contours and texture simultaneously for image segmentation and grouping, current approaches rely on different models for contours, regions, and texture such as one-dimensional intensity discontinuities for contours and filter bank responses for texture. This results in expensive computation that arbitrates between these disparate representations at each pixel. In our approach, salient image content such as contours, regions, and texture are represented in a common, low-level framework of image observables. We model the image as a partition of surfaces bounded by intensity discontinuities and derive perceptual measures as relations between neighboring surfaces. This enables us to extend the traditional Gestalt measures based on local edge geometry and contrast to region-based measures that jointly exploit large scale image topology, photometry, and geometry. These measures provide a natural basis for grouping on multidimensional similarity criteria and texture is directly derived as relational properties on local region neighborhoods. The viability of our model is demonstrated by applying the common observables to texture recognition, figure-ground separation, and generic image segmentation.

The primary focus of work on perceptual grouping has been geometric constraints derived from projected object boundaries. Although boundaries can be robustly extracted under some conditions, much intensity information in the image is ignored. In this work, we incorporate image topology and intensity into grouping for object detection and recognition. Using boundary-based region segmentation, region intensity, geometry and topology are exploited in forming groups of regions that satisfy high-level constraints at arbitrary scales. It is demonstrated that regions provide a natural and computationally effective framework for discovering localized image structure, such as man-made objects in natural scenes.

Manufacturing flaws of all types, shapes, and sizes can be exhaustively detected as abnormal pixels, if process and noise variations can be learned at every pixel in the inspection area. This statistical template approach to automated visual inspection is extremely fast, effective, and flexible, while achieving false negative rate <10/sup -6/. Critical to this approach are the following novel features: 1) represent both geometry and process information in a model template; 2) align 3D surfaces with subpixel accuracy; compensate for local deformation and texture; 4) estimate bimodal distribution robustly. This novel paradigm was applied to the automatic screening of X-ray images of turbine blades. It has been validated with over 50,000 images and shown to outperform regular inspectors looking at high-pass filtered images.

A novel approach to grouping symmetrical planar curves under a projective transform is described. Symmetric curves are important as a generic model for object recognition where an object class is defined by the set of symmetries that any object in the class obeys. In this paper, a new algorithm is presented for grouping curves based on their correspondence under a plane projectivity. The correspondence between curves is established from an initial correspondence between two pairs of distinguished lines, such as lines tangent to inflection points. This initial correspondence leads to a reduced dimensional form for the projective mapping between the curves and a natural method for establishing correspondence between all points on the curves. A saliency measure is introduced which permits grouping results to be ordered in terms of the degree of symmetry supported by each curve pair. This saliency measure provides a basis for recognition in the case of approximate symmetry.

An important challenge in industrial metrology is to provide rapid measurement of critical 3D internal object geometry for either inspecting high volume parts or controlling a machining process. Existing metrological techniques are typically too slow to meet this need or can not measure small features with high precision. In this paper, we present a new method that achieves fast, accurate, internal 3D geometry measurement based on 3D reconstruction from a few X-ray views of a part. Our approach utilizes an accurate camera model for the X-ray sensor, calibration using in situ ground truth and geometry-guided X-ray feature extraction to achieve this goal and has been fully implemented in a prototype 3D measurement system. We describe a novel application of the system to CAD-based verification of drilled hole positioning. Experimental results are given to illustrate the precision of the system and 3D measurement on real industrial parts.

In any object recognition system a major and primary task is to associate those image features, within an image of a complex scene, that arise from an individual object. The key idea here is that a geometric class defined in 3D induces relationships in the image which must hold between points on the image outline (the perspective projection of the object). The resulting image constraints enable both identification and grouping of image features belonging to objects of that class. The classes include surfaces of revolution, canal surfaces (pipes) and polyhedra. Recognition proceeds by first recognising an object as belonging to one of the classes (for example a surface of revolution) and subsequently identifying the object (for example as a particular vase). This differs from conventional object recognition systems where recognition is generally targetted at particular objects. These classes also support the computation of 3D invariant descriptions including symmetry axes, canonical coordinate frames and projective signatures. The constraints and grouping methods are viewpoint invariant, and proceed with no information on object pose. We demonstrate the effectiveness of this class-based grouping on real, cluttered scenes using grouping algorithms developed for rotationally symmetric surfaces, canal-surfaces and polyhedra.>

This paper presents a new methodology for camera calibration of stereo X-ray projections. The image acquisition systems typically used in nondestructive evaluation result in views that are orthographic along one image axis and perspective along the other. A camera model for this sensing geometry, called the linear pushbroom model, is described. Four methods of calibration, which make different assumptions about what is known about the camera parameters, are presented and compared.

An important challenge in industrial metrology is to provide rapid measurement of critical 3D internal object geometry for either inspecting high volume parts or controlling a machining process. Existing metrological techniques are typically too slow to meet this need or can not measure small features with high precision. In this paper, we present an X-ray stereo system which aims to achieve fast 3D geometry measurement from a few X-ray views of a part. We describe the key algorithms in our system and a novel application of it to CAD-based verification of drilled hole positioning. Experimental results are given to illustrate the accuracy of the current system and inspection on a real part.

A recognition strategy consisting of a mixture of indexing on invariants and search, allows objects to be recognised up to a Euclidean ambiguity with an uncalibrated camera. The approach works by using projective invariants to determine all the possible projectively equivalent models for a particular imaged object; then a system of global consistency constraints is used to determine which of these projectively equivalent, but Euclidean distinct, models corresponds to the objects viewed. These constraints follow from properties of the imaging geometry. In particular, a recognition hypothesis is equivalent to an assertion about, among other things, viewing conditions and geometric relationships between objects, and these assertions must be consistent for hypotheses to be correct. The approach is demonstrated to work on images of real scenes consisting of polygonal objects and polyhedra.>

The Image Understanding Environment (IUE) project is a five year program, sponsored by ARPA, to develop a common object-oriented software environment for the development of algorithms and application systems. This paper reviews the design of this system and provides an overview of the distributed implementation effort currently underway at Amerinex AI, Advanced Decision Systems, Carnegie Mellon University, and Colorado State University. The ultimate goal of the project is to provide a software infrastructure of class hierarchies, user interface tools, and IU algorithms that are required to carry out state of the art research in image understanding.>

There is considerable current interest in deriving accurate dimensional measurements of the internal geometry of complex manufactured parts, particularly castings. This paper describes an approach to the reconstruction of 3D part geometry from multiple digital X-ray images. A novel method for radiographic stereo is described which takes into account the special imaging geometry of the digital X-ray sensor modeled by a linear moving array, or pushbroom, camera. The 3D reconstruction algorithm employs a nominal geometric model which is perturbed by X-ray image constraints. Manufacturing applications are discussed and illustrated by experimental results on synthetic phantoms and actual casting images.>

Over the last several years the concept of model-supported exploitation (MSE) has evolved to a point where relatively simple computer vision algorithms can extract significant intelligence information from aerial images in a robust and reliable manner. Information extraction is enabled by the use of detailed 3D site models which provide an extensive context for the application of image analysis algorithms. This paper reviews the basic MSE concept and illustrates the approach using three operational concepts taken from the RADIUS project, quick-look, detection and counting and focussed change detection.>

It is known that rotationally symmetric surfaces can be recognized from their outlines alone, using cross-ratios of bitangent intersections. A successful implementation of this technique is demonstrated using a novel bitangent finder which works on images of real scenes. The stability of the cross-ratios is reported and compared to affine invariants. The recognition technique is shown to extend to the case of straight homogeneous generalized cylinders.>

A novel approach to part tolerancing with measurement error based on Bayesian methods is presented. Parts are represented by parameterised constraint templates. A priori knowledge about expected part geometry is introduced through a prior distribution and template parameter distributions (rather than just nominal parameter values) estimated from data sets using Gibbs sampling. The case of a toleranced dimension and linear constraints is analyzed. An extension to nonlinear constraints is briefly described.>

A number of recent papers have argued that invariants do not exist for three-dimensional point sets in general position, which has often been misinterpreted to mean that invariants cannot be computed for any three-dimensional structure. It is proved by example that although the general statement is true, invariants do exist for structured three-dimensional point sets. Projective invariants are derived for two object classes: the first is for points that lie on the vertices of polyhedra, and the second for objects that are projectively equivalent to ones possessing a bilateral symmetry. The motivations for computing such invariants are twofold: they can be used for recognition, and they can be used to compute projective structure. Examples of invariants computed from real images are given.>

The history of the image understanding environment (IUE) project, a five-year program to develop a common software environment for the development of algorithms and application systems, is reviewed. An overview of some of the data structures that are currently evolving as a specification for the IUE is provided. The ultimate goal of the project is to provide the basic data structures and algorithms that are required to carry state-of-the-art research in image understanding.>

The concepts and design issues that provide the basis for the I/sup 2/F (image interpretation foundations) system are described. The I/sup 2/F system combines object-oriented design for machine vision software and constraint-based geometric modeling into a flexible and effective system for automatic template-guided visual inspection. Object-oriented design for 2-D geometry-based image analysis is discussed, and results from processing experimental X-ray data are presented.>

Projectively invariant shape descriptors allow fast indexing into model libraries without the need for pose computation or camera calibration. Progress in building a model-based vision system for plane objects that uses algebraic projective invariants is described. A brief account of these descriptors is given, and the recognition system is described, giving examples of the invariant techniques working on real images.>

A generic polyhedral model is represented as a network of nodes and constraints. Nodes are 3-D vectors representing the location and orientation of the geometric entities, or measure variables such as length or cosine. Constraints are polynomial equations in the node parameters. Modeling and recognition are viewed as solving for values of the node parameters such that all the constraint equations are satisfied and the mean square error between the model and the observed shape is minimized. Buchberger's Grobner basis algorithm and Ritt-Wu's triangulation algorithm can be used for eliminating dependent parameters as well as for detecting inconsistency among constraints. Numerical techniques are used to find the best-fit model subject to constraints.>

Invariant descriptors are shape descriptors that are unaffected by object pose, by perspective projection, or by the intrinsic parameters of the camera. These descriptors can be constructed using the methods of invariant theory, which are briefly surveyed. A range of applications of invariant descriptors in 3D model-based vision is demonstrated. First, a model-based vision system that recognizes curved plane objects irrespective of their pose is demonstrated. Curves are not reduced to polyhedral approximations but are handled as objects in their own right. Models are generated directly from image data. Once objects have been recognized, their pose can be computed. Invariant descriptors for 3D objects with plane faces are described. All these ideas are demonstrated using images of real scenes. The stability of a range of invariant descriptors to measurement error is treated in detail.>

Abstract The shape of objects seen in images depends on the viewpoint. This effect confounds recognition. We demonstrate a theoretical framework within which it is possible to construct descriptors for curves which do not vary with viewpoint. These descriptors are known as invariants. We use this framework to construct invariant shape descriptors for plane curves. These invariant shape descriptors make it possible to recognise plane curves, without explicitly determining the relationship between the curve reference frame and the camera coordinate system, and can be used to index quickly and efficiently into a large model base of curves. Many of these ideas are demonstrated by experiments on real image data.

Projectively invariant shape descriptors efficiently identify instances of object models in images without reference to object pose. These descriptions rely on frame independent representations of planar curves, using plane conies. We show that object pose can be determined from coplanar curves, given such a frame independent representation. This result is demonstrated for real image data. The shape of objects in images changes as the camera is moved around. This extremely simple observation represents the dominant problem in model based vision. Nielsen [4, 5] first suggested using projectively invariant labels as landmarks for navigation. Recent papers [1, 2] have shown that it is possible to compute shape descriptors of arbitrary plane objects that are unaffected by

It is shown that curved planar objects have shape descriptors that are unaffected by the position, orientation and intrinsic parameters of the camera. These shape descriptors can be used to index quickly and efficiently into a large model base of curved planar objects, because their value is independent of pose and unaffected by perspective. Thus, recognition can proceed independent of calculating pose. Object curves are represented using conics, attached with a fitting technique that commutes with projection. This means that the pose of an object can be determined by backprojecting known conics. The authors show examples of recognition and pose determination using real image data.>

A benchmark evaluation test of a model-based recognition system is discussed. The system was tested on a series of aerial reconnaissance images to evaluate recognition performance on the task of airfield monitoring. The effectiveness of the model pose constraint for recognition is discussed as well as an approach for selection of model features. The use of distance transforms for model hypothesis confirmation is also discussed.>

A geometrical theorem prover based on algebraic techniques has been used to prove and derive theorems in computer vision. The results show that theorems relevant to persepective viewing and the properties of shadows can be proven, including some which contain inequalities. Deriving theorems by stating incomplete hypotheses and letting the prover suggest the missing constraints can provide valuable new insights into vision probiems. However, there is no guarantee that the suggestions made by the proof algorithm are interpretable in terms of primitive geometric concepts.

The use of formal geometric reasoning is proposed to analyze algorithms for machine vision and robotics. This paper develops the basis for this approach and illustrates the method with several examples taken from perspective scene analysis. The application of recent developments in algebraic deduction methods is described.