In this paper, we present a simple and efficient way to add supervised information into Fisher vectors, which has become a popular image representation method for image classification and retrieval purposes in recent years. The basic idea of our approach is to improve the Fisher kernel in the training process by adding a discriminative label comparison matrix to it. The resulting new representations, which we call Label Consistent Fisher Vectors (LCFV), can be solved for both over determined and underdetermined cases. We show that LCFV has better classification performance than traditional Fisher vectors on three public datasets.

In traditional vision systems, high level information is usually inferred from images or videos captured by cameras, or depth images captured by depth sensors. These images, whether gray-level, RGB, or depth, have a human-readable 2D structure which describes the spatial distribution of the scene. In this paper, we explore the possibility to use distributed color sensors to infer high level information, such as room occupancy. Unlike a camera, the output of a color sensor has only a few variables. However, if the light in the room is color controllable, we can use the outputs of multiple color sensors under different lighting conditions to recover the light transport model (LTM) in the room. While the room occupancy changes, the LTM also changes accordingly, and we can use machine learning to establish the mapping from LTM to room occupancy.

We present a novel method to solve sign ambiguity for phase demodulation from a single interferometric image that possibly contains closed fringes. The problem is formulated in a binary pairwise energy minimization framework based on phase gradient orientation continuity. The objective function is non-submodular and therefore its minimization is an NP-hard problem, for which we devise a multigrid hierarchy of quadratic pseudoboolean optimization problems that can be improved iteratively to approximate the optimal solutions. Compared with traditional path-following phase demodulation methods, the new approach does not require any heuristic scanning strategy, it is not subject to the propagation of error, and the extension to three dimensional fringe patterns is straightforward. A set of experiments with synthetic data and real prelens tear film interferometric images of the human eye demonstrate the effectiveness and robustness of the proposed algorithm in comparison with existing state-of-the-art phase demodulation methods.

The automatic detection of lung nodules attached to other pulmonary structures is a useful yet challenging task in lung CAD systems. In this paper, we propose a stratified statistical learning approach to recognize whether a candidate nodule detected in CT images connects to any of three other major lung anatomies, namely vessel, fissure and lung wall, or is solitary with background parenchyma. First, we develop a fully automated voxel-by-voxel labeling/segmentation method of nodule, vessel, fissure, lung wall and parenchyma given a 3D lung image, via a unified feature set and classifier under conditional random field. Second, the generated Class Probability Response Maps (PRM) by voxel-level classifiers, are used to form the so-called pairwise Probability Co-occurrence Maps (PCM) which encode the spatial contextual correlations of the candidate nodule, in relation to other anatomical landmarks. Based on PCMs, higher level classifiers are trained to recognize whether the nodule touches other pulmonary structures, as a multi-label problem. We also present a new iterative fissure structure enhancement filter with superior performance. For experimental validation, we create an annotated database of 784 subvolumes with nodules of various sizes, shapes, densities and contextual anatomies, and from 239 patients. High accuracy of multi-class voxel labeling is achieved 89.3% ∼ 91.2%. The Area under ROC Curve (AUC) of vessel, fissure and lung wall connectivity classification reaches 0.8676, 0.8692 and 0.9275, respectively.

The computer aided diagnosis (CAD) problems of detecting potentially diseased structures from medical images are typically distinguished by the following challenging characteristics: extremely unbalanced data between negative and positive classes; stringent real-time requirement of online execution; multiple positive candidates generated for the same malignant structure that are highly correlated and spatially close to each other. To address all these problems, we propose a novel learning formulation to combine cascade classification and multiple instance learning (MIL) in a unified min-max framework, leading to a joint optimization problem which can be converted to a tractable quadratically constrained quadratic program and efficiently solved by block-coordinate optimization algorithms. We apply the proposed approach to the CAD problems of detecting pulmonary embolism and colon cancer from computed tomography images. Experimental results show that our approach significantly reduces the computational cost while yielding comparable detection accuracy to the current state-of-the-art MIL or cascaded classifiers. Although not specifically designed for balanced MIL problems, the proposed method achieves superior performance on balanced MIL benchmark data such as MUSK and image data sets.

We propose a dimension reduction technique named Resilient Subclass Discriminant Analysis (RSDA) for high dimensional classification problems. The technique iteratively estimates the subclass division by embedding the Fisher Discriminant Analysis (FDA) with Expectation-Maximization (EM) in Gaussian Mixture Models (GMM). The new method maintains the adaptability of SDA to a wide range of data distributions by approximating the distribution of each class as a mixture of Gaussians, and provides superior feature selection performance to SDA with modified EM clustering that estimates a posteriori probability of latent variables in lower-dimensional Fisher's discriminant space, which also improves the robustness in problems of small training datasets compared with conventional EM algorithm. Extensive experiments and comparison results against other well-known Discriminant Analysis (DA) methods are presented using synthetic data, benchmark datasets as well as a real computational vision problem.

In this paper, the development of a computer-aided system for the classification of grade of neuroblastic differentiation is presented. This automated process is carried out within a multi-resolution framework that follows a coarse-to-fine strategy. Additionally, a novel segmentation approach using the Fisher-Rao criterion, embedded in the generic expectation-maximization algorithm, is employed. Multiple decisions from a classifier group are aggregated using a two-step classifier combiner that consists of a majority voting process and a weighted sum rule using priori classifier accuracies. The developed system, when tested on 14,616 image tiles, had the best overall accuracy of 96.89%. Furthermore, multi-resolution scheme combined with automated feature selection process resulted in 34% savings in computational costs on average when compared to a previously developed single-resolution system. Therefore, the performance of this system shows good promise for the computer-aided pathological assessment of the neuroblastic differentiation in clinical practice.

We present a new deformable model technique following a snake-like approach and using a complex Fourier shape descriptors parameterization to efficiently formulate the forces that constrain contour deformation. The method was successfully applied to track the left ventricle’s (LV) endocardial and epicardial boundaries in sequences of shortaxis magnetic resonance images depicting complete cardiac cycles. The extracted shapes show the method’s robustness to weak contrast, noisy edge maps and to papillary muscle anatomy. Our second contribution is a statistical pattern recognition approach for the detection of asynchronous activation of the LV walls. We applied our deformable model method to provide spatio-temporal characterizations of complete cardiac cycles and then designed a linear classifier using the popular combination of Principal Component Analysis and Linear Discriminant Analysis. From a database comprising 33 patients, our approach provided a correct classification performance of 90.9% showing its potential in providing improved dyssynchrony characterization as an adjunct to current criteria for selecting patients for therapy, which provided a classification accuracy of just 62.5% on the same database.

Optical coherence tomography (OCT) uses retroreflected light to provide micrometer-resolution, cross-sectional scans of biological tissues. OCT's first application was in ophthalmic imaging where it has proven particularly useful in diagnosing, monitoring, and studying glaucoma. Diagnosing glaucoma is difficult and it often goes undetected until significant damage to the subject's visual field has occurred. As glaucoma progresses, neural tissue dies, the nerve fiber layer thins, and the cup-to-disk ratio increases. Unfortunately, most current measurement techniques are subjective and inherently unreliable, making it difficult to monitor small changes in the nervehead geometry. To our knowledge, this paper presents the first published results on optic nervehead segmentation and geometric characterization from OCT data. We develop complete, autonomous algorithms based on a parabolic model of cup geometry and an extension of the Markov model introduced by Koozekanani, et al. to segment the retinal-nervehead surface, identify the choroid-nervehead boundary, and identify the extent of the optic cup. We present thorough experimental results from both normal and pathological eyes, and compare our results against those of an experienced, expert ophthalmologist, reporting a correlation coefficient for cup diameter above 0.8 and above 0.9 for the disk diameter.

This paper addresses the range image registration problem for views having low overlap and which may include substantial noise. The current state of the art in range image registration is best represented by the well-known iterative closest point (ICP) algorithm and numerous variations on it. Although this method is effective in many domains, it nevertheless suffers from two key limitations: It requires prealignment of the range surfaces to a reasonable starting point and it is not robust to outliers arising either from noise or low surface overlap. This paper proposes a new approach that avoids these problems. To that end, there are two key, novel contributions in this work: a new, hybrid genetic algorithm (GA) technique, including hillclimbing and parallel-migration, combined with a new, robust evaluation metric based on surface interpenetration. Up to now, interpenetration has been evaluated only qualitatively; we define the first quantitative measure for it. Because they search in a space of transformations, GAs are capable of registering surfaces even when there is low overlap between them and without need for prealignment. The novel GA search algorithm we present offers much faster convergence than prior GA methods, while the new robust evaluation metric ensures more precise alignments, even in the presence of significant noise, than mean squared error or other well-known robust cost functions. The paper presents thorough experimental results to show the improvements realized by these two contributions.

Today's commercial satellite images enable experts to classify region types in great detail. In previous work, we considered discriminating rural and urban regions [23]. However, a more detailed classification is required for many purposes. These fine classifications assist government agencies in many ways including urban planning, transportation management, and rescue operations. In a step toward the automation of the fine classification process, this paper explores graph theoretical measures over grayscale images. The graphs are constructed by assigning photometric straight line segments to vertices, while graph edges encode their spatial relationships. We then introduce a set of measures based on various properties of the graph. These measures are nearly monotonic (positively correlated) with increasing structure (organization) in the image. Thus, increased cultural activity and land development are indicated by increases in these measures-without explicit extraction of road networks, buildings, residences, etc. These latter, time consuming (and still only partially automated) tasks can be restricted only to "promising" image regions, according to our measures. In some applications our measures may suffice. We present a theoretical basis for the measures followed by extensive experimental results in which the measures are first compared to manual evaluations of land development. We then present and test a method to focus on, and (pre)extract, suburban-style residential areas. These are of particular importance in many applications, and are especially difficult to extract. In this work, we consider commercial IKONOS data. These images are orthorectified to provide a fixed resolution of 1 meter per pixel on the ground. They are, therefore, metric in the sense that ground distance is fixed in scale to pixel distance. Our data set is large and diverse, including sea and coastline, rural, forest, residential, industrial, and urban areas.

Vegetation indices have been used extensively to estimate the vegetation density from satellite and airborne images for many years. In this paper, we focus on one of the most popular of such indices, the normalized difference vegetation index (NDVI), and we introduce a statistical framework to analyze it. As the degree of vegetation increases, the corresponding NDVI values begin to saturate and cannot represent highly vegetated regions reliably. By adopting the statistical viewpoint, we show how to obtain a linearized and more reliable measure. While the NDVI uses only red and near-infrared bands, we use the statistical framework to introduce new indices using the blue and green bands as well. We compare these indices with that obtained by linearizing the NDVI with extensive experimental results on real IKONOS multispectral images.

We introduce a set of measures based on straight lines to assess land development levels in high-resolution (1 m) panchromatic satellite images. Most urban areas locally (such as in a 400/spl times/400 m/sup 2/ area) exhibit a preponderance of straight-line features, generally appearing in fairly simple quasi-periodic organizations. Wilderness and rural areas produce line structures in more random spatial arrangements. We use this observation to perform an initial triage on the image to restrict the attention of subsequent more computationally intensive analyses. Statistical measures based on straight lines guide the analysis. We base these measures on length, contrast, orientation, periodicity, and location. On these, we trained and tested parametric and nonparametric classifiers. These tests were for a two-class problem (urban versus rural). However, because our ultimate goal is to extract residential regions, we then extended these ideas to address the detection of suburban regions. To do so, some use of spatial coherence is required; suburban regions are especially difficult to detect. Therefore, we introduce a decision system to perform suburban region classification via an overlapping voting method for consensus discovery. Our data were taken from regions all around the world, which underscores the robustness of our approach. Based on extensive testing, we can report very promising results in distinguishing developed areas.

It is well known that combining spatial and spectral information can improve land use classification from satellite imagery. Human activity on the ground, such as construction, induces changes in both the photometric structure of the image and in its spectral content owing to, primarily, changes in vegetation density and surface materials. This paper introduces a novel approach to combine spatial (more precisely, structural) information extracted from (1-m resolution) panchromatic Ikonos imagery with the multispectral response (4-m resolution) available from the same sensor. Of the prior work combining spatial and spectral information, none has extracted structural features as we do, and none has combined these information sources as early in the process. The classifier we describe here, discriminating urban and rural regions, is a front-end component of a fairly complete satellite image analysis system that identifies suburban residential areas and extracts their street networks and single-family houses. We extract structural information in the form of photometric straight lines and their spatial arrangement over relatively small neighborhoods. To capture the multispectral information, we turn to the well-known normalized difference vegetation index (NDVI) and an improved linearized version of our own development (details of the structural analysis and the theoretical development of the linearized NDVI appear elsewhere). This paper addresses the novel combination of these types of features (hybrids) by using the structural features, straight line support regions based on gradient orientation, as cue regions for multispectral analysis. We test the hybrid features in a range of parametric and nonparametric classifiers. We also implement and test a probabilistic relaxation algorithm followed by the maximum a priori decision rule. We report extensive results that indicate significant improvements in classification accuracy using the hybrid features.

This paper presents a novel range image segmentation method employing an improved robust estimator to iteratively detect and extract distinct planar and quadric surfaces. Our robust estimator extends M-estimator Sample Consensus/Random Sample Consensus (MSAC/RANSAC) to use local surface orientation information, enhancing the accuracy of inlier/outlier classification when processing noisy range data describing multiple structures. An efficient approximation to the true geometric distance between a point and a quadric surface also contributes to effectively reject weak surface hypotheses and avoid the extraction of false surface components. Additionally, a genetic algorithm was specifically designed to accelerate the optimization process of surface extraction, while avoiding premature convergence. We present thorough experimental results with quantitative evaluation against ground truth. The segmentation algorithm was applied to three real range image databases and competes favorably against eleven other segmenters using the most popular evaluation framework in the literature. Our approach lends itself naturally to parallel implementation and application in real-time tasks. The method fits well, into several of today's applications in man-made environments, such as target detection and autonomous navigation, for which obstacle detection, but not description or reconstruction, is required. It can also be extended to process point clouds resulting from range image registration.

Most range image registration techniques are based on variants of the ICP (iterative closest point) algorithm. The ICP algorithm has two main drawbacks, the possibility of convergence to a local minimum and the need to prealign the images. Genetic algorithms (GAs) are known to be robust in relation to search and optimization problems and were recently applied to range image registration, providing good convergence results without the constraints observed in the ICP approaches. To improve range image registration by GAs, we explored 3 novel approaches: a hybrid algorithm that combines a GA with hillclimbing heuristics (GH), a parallel migration GA (MGA), and a MGA using hillclimbing (MGH). We also define a new robust evaluation measure, called the surface interpenetration, to compare the obtained registration results. Up to now, interpenetration has been evaluated only qualitatively; we define the first quantitative measure for it. The experimental results show that our methods yield more accurate registration results than either ICP or standard GA approaches.

Vegetation indices have been used extensively to estimate the vegetation density from satellite and airborne images for many years. In this paper, we first focus on one of the most popular vegetation indices, the normalized difference vegetation index (NDVI) by J. W. Rouse (1974) and introduce a statistical framework to analyze it. We propose a solution to the saturation problem of this index based on our statistical framework. We investigate the relationship of this index with the ratio vegetation index (RVI) by C. F. Jordan (1969), another popular measure. Using the established statistical framework, we introduce new vegetation indices using blue and green bands in addition to the red and the near-infrared.

Optical coherence tomography (OCT) is a new ophthalmic imaging modality generating cross sectional views of the retina. OCT systems are essentially Michelson interferometers that form images in 1.5 s by directing a superluminescent diode (SLD) beam over the retinal surface. Involuntary eye motions frequently cause incorrect locations to be imaged. This motion may leave no obvious artifacts in the scan data and can easily go undetected. For glaucoma monitoring especially, knowing the measurement path, typically a circle concentric with the nerve head, is crucial. The commercially available OCT system displays a near-infrared video of the retina showing the SLD beam. This paper presents a prototype system to detect the nerve head and SLD beam in the video, and report the true scan path relative to the nerve head. Low image contrast and limited resolution make the reliable detection of retinal features difficult. In an adaptive model construction phase, the system directly detects retinal vasculature and the nerve head and incrementally builds a model of the current subject's vascular pattern relative to the optic disk. The nerve head identification is multitiered, using a novel dual eigenspace technique and a geometric comparison of detected vessel positions and nerve head hypotheses. In its operational phase, a correspondence is achieved between the currently detected vasculature and the model. Using subjects not included in training, the system located the optic nerve head to within 5 pixels (0.07 optic disk diameters, an error well below clinical significance) in 99.75% of 2800 video fields. In current clinical practice, motions as large as 1-2 disc diameters may go undetected, so this is a vast improvement.

Optical coherence tomography (OCT) is a novel ophthalmic imaging modality generating cross sectional views of the retina. OCT systems form images in 1.5 seconds by directing a superluminescent diode (SLD) beam over the retinal surface. Involuntary ocular motion may occur, however, causing incorrect locations to be imaged. This motion may leave no obvious artifacts and thus go undetected. For glaucoma monitoring especially, knowing the measurement location is crucial. The commercially available OCT system displays a near-IR video of the SLD beam traversing the retinal around the optic nerve head. We developed a prototype system to detect the nerve head and SLD beam position in this video, and report the actual scan path relative to the nerve head. This system must cope with low image contrast and few reliable retinal features. In its adaptive model generation phase, the system directly detects vasculature and the nerve head and builds an individual model of the vascular pattern. The nerve head identification is multi-tiered, using a novel, dual-eigenspace technique and a geometric comparison of detected vessel positions and nerve head hypotheses. In its operational phase, a correspondence is achieved between detected vasculature and the model. The system was evaluated on video of three subjects not used to form the eigenspaces. The system located the optic nerve head to within 5 pixels in 99% of 2800 video fields manually inspected, and was thus able to determine the true scan path relative to the nerve head.

We present a system for detecting retinal boundaries in optical coherence tomography (OCT) B-scans. OCT is a relatively new imaging modality giving cross-sectional images that are qualitatively similar to ultrasound. However, the axial resolution with OCT is much higher, on the order of 10 microm. Objective, quantitative measures of retinal thickness may be made from OCT images. Knowledge of retinal thickness is important in the evaluation and treatment of many ocular diseases. The boundary-detection system presented here uses a one-dimensional edge-detection kernel to yield edge primitives. These edge primitives are rated, selected, and organized to form a coherent boundary structure by use of a Markov model of retinal boundaries as detected by OCT. Qualitatively, the boundaries detected by the automated system generally agreed extremely well with the true retinal structure for the vast majority of OCT images. Only one of the 1450 evaluation images caused the algorithm to fail. A quantitative evaluation of the retinal boundaries was performed as well, using the clinical application of automatic retinal thickness determination. Retinal thickness measurements derived from the algorithm's results were compared with thickness measurements from manually corrected boundaries for 1450 test images. The algorithm's thickness measurements over a 1-mm region near the fovea differed from the corrected thickness measurements by less than 10 microm for 74% of the images and by less than 25 microm (10% of normal retinal thickness) for 98.4% of the images. These errors are near the machine's resolution limit and still well below clinical significance. Current, standard clinical practice involves a qualitative, visual assessment of retinal thickness. A robust, quantitatively accurate system such as ours can be expected to improve patient care.

We present a highly accurate, robust system for measuring retinal thickness in optical coherence tomography (OCT) images. OCT is a relatively new imaging modality giving cross sectional images that are qualitatively similar to ultrasound but with 10 /spl mu/m resolution. We begin with a 1-dimensional edge detection kernel to yield edge primitives, which are then selected, corrected, and grouped to form a coherent boundary by use of a Markov model of retinal structure. We have tested the system extensively, and only one of 650 evaluation images caused the algorithm to fail. The anticipated clinical application for this work is the automatic determination of retinal thickness. A careful quantitative evaluation of the system performance over a 1 mm region near the fovea reveals that in more than 99% of the cases, the automatic and manual measurements differed by less than 25 /spl mu/m (below clinical significance), and in 89% of the tests the difference was less than 10 /spl mu/m (near the resolution limit). Current clinical practice involves only a qualitative, visual assessment of retinal thickness. Therefore, a robust, quantitatively accurate system should significantly improve patient care.

We propose four measures of image organizational change which can be used to monitor construction activity. The measures are based on the thesis that the progress of construction will see a change in the individual image feature attributes as well as an evolution in the relationships among these features. This change in the relationship is captured by the eigenvalues and eigenvectors of the relation graph embodying the organization among the image features. We demonstrate the ability of the measures to differentiate between no development, the onset of construction, and full development, on the available real test image set.

We present an eigenvalue or spectral representation for CAD models to be used in conjunction with the more traditional attributed graph based representation of these models. The eigenvalues provide a gross description of the structure of the objects, and help to divide a large modelbase into structurally homogeneous partitions. Models in each partition are next hierarchically organized according to the algorithm presented in Sengupta and Boyer (1995). In recognition, gross features computed from a hypothesized object in a range image are used to prune the modelbase by selecting a few "favorable" partitions in which the correct object model is likely to lie. The partitioning experiments presented here are for real range images using a modelbase of 125 CAD objects with planar, cylindrical, and spherical surfaces.

Presents a hierarchically structured approach to organizing large structural modelbases using an information theoretic criterion. Objects (patterns) are modeled in the form of random parametric structural descriptions (RPSDs), an extension of the parametric structural description graph-theoretic formalism. Objects in scenes are modeled as parametric structural descriptions (PSDs). The organization process is driven by pair-wise dissimilarity values between RPSDs. The authors also introduce the node pointer lists, which are computed offline during modelbase organization. During recognition, the only exponential matching process involved is between the scene PSD and the RPSD at the root of the organized tree. Using the organized hierarchy along with the node pointer lists, the remaining work simplifies to a series of inexpensive linear tests at the subsequent levels of the tree search. The authors develop the theory and present three modelbases: 50 objects built from real image data, 100 CAD models, and 1000 synthetic abstract models.>

We previously presented (Sarkar and Boyer, 1993) the Perceptual Inference Network (PIN), a formalism based on Bayesian Networks, to reason among a set of object or feature hypotheses and to integrate multiple sources of information in the context of perceptual organization. The design of a PIN requires knowledge of the dependency structure among the organizations of interest and the specification of the conditional probabilities. This design was done manually with large doses of tedium and guesswork. In this paper we present an algorithm based on structural entropic measures and random parametric structural descriptions (RPSDs) to design a PIN automatically and in a (more) theoretically sound fashion. Experimental results present evidence of the robustness of the algorithm and make performance comparisons on real image data with a manually structured PIN. Since PINs are a form of Bayesian Network, we hope that this work will also prove useful towards structuring Bayesian Networks in other computer vision contexts.>

Two key issues in computer vision are: solving the correspondence and camera calibration problems in stereo. We present a novel approach to autonomous relative camera orientation and stereo matching that uses the relationship between the correspondence problem and the camera pose estimation problem and combines these two into a single interactive algorithm embedded in a multiresolution framework. Our results show that this technique exhibits advantages in accuracy and versatility over existing methods.

The aim is to generate a hierarchical description of the scene using preattentive and attentive modules. The preattentive module provides evidence in terms of primitive organizations like parallelism, continuity, closure, and strands. The attentive organization integrates this preattentive evidence to hypothesize more complex organizations such as parallelograms, circles, ellipses, and ribbons. This attentive part is realized by the perceptual inference network (PIN) which is a form of Bayesian network. The output set of hypotheses of the PIN is large and redundant. A set of lines is described as a parallelogram and/or ellipse and/or circle. There is considerable ambiguity in such a description. The strategy is to use special-purpose modules to resolve the ambiguous hypotheses and to generate a comprehensive scene description. These special purpose modules tend to be computationally expensive and have limited applicability. Therefore, we want to apply them only when and where we expect the greatest amount of information gain per unit computational resource.

Presents an autonomous, statistically robust, sequential function approximation approach to simultaneous parameterization and organization of (possibly partially occluded) surfaces in noisy, outlier-ridden (not Gaussian), functional range data. At the core of this approach is the Robust Sequential Estimator, a robust extension to the method of sequential least squares. Unlike most existing surface characterization techniques, the authors' method generates complete surface hypotheses in parameter space. Given a noisy depth map of an unknown 3-D scene, the algorithm first selects appropriate seed points representing possible surfaces. For each nonredundant seed it chooses the best approximating model from a given set of competing models using a modified Akaike Information Criterion. With this best model, each surface is expanded from its seed over the entire image, and this step is repeated for all seeds. Those points which appear to be outliers with respect to the model in growth are not included in the (possibly disconnected) surface. Point regions are deleted from each newly grown surface in the prune stage. Noise, outliers, or coincidental surface alignment may cause some points to appear to belong to more than one surface. These ambiguities are resolved by a weighted voting scheme within a 5/spl times/5 decision window centered around the ambiguous point. The isolated point regions left after the resolve stage are removed and any missing points in the data are filled by the surface having a majority consensus in an 8-neighborhood.>

Tagare and deFigueiredo (see ibid., vol. 12, p. 1186-1189, 1990 ) present a localization performance measure for edge detectors. They correctly point out a flaw in Canny's formulation of the localization criterion, which was subsequently adopted by Sarkar and Boyer (1991). They motivate their form of the localization criterion along a different line of reasoning. In this comment, the authors show that although Canny's derivation was in error, the final form of his criterion is adequate and can, in fact, be derived from Tagare and deFigueiredo's formulation of the problem. The authors also point out some problems with Tagare and deFigueiredo's localization criterion.>

Presents an efficient computational structure for preattentive perceptual organization. By perceptual organization the authors refer to the ability of a vision system to organize features detected in images based on viewpoint consistency and other Gestaltic perceptual phenomena. This usually has two components, a primarily bottom up preattentive part and a top down attentive part, with meaningful features emerging in a synergistic fashion from the original set of (very) primitive features. In this work the authors advance a computational structure for preattentive perceptual organization. The authors propose a hierarchical approach, using voting methods to build associations through consensus and relational graphs to represent the organization at each level. The voting method is very efficient in terms of time and space and performs impressively for a wide range of organizations. The graphical representation allows the ready extraction of higher order features, or perceptual tokens, because the relational information is rendered explicit.>

A hierarchically structured approach to organizing large structural model bases using an information theoretic criterion is presented. Objects (patterns) are modeled in the form of random parametric structural descriptions (RPSDs), an extension of the parametric structural description graph-theoretic formalism. Hierarchically clustering the RPSDs reduces the computational work to O(log N). The node pointers allow a mapping between the observation and a stored representation at one level, and the mapping to all potential models at all subsequent levels is reduced to mere tests, eliminating the exponential search for the best interprimitive mapping function for each stored candidate pattern.>

The formalism of Bayesian networks provides a very elegant solution, in a probabilistic framework, to the problem of integrating top-down and bottom-up visual processes, as well serving as a knowledge base. The formalism is modified to handle spatial data, and thus the application of Bayesian networks is extended to visual processing. The modified form is called the perceptual inference network (PIN). The theoretical background of a PIN is presented, and its viability is demonstrated in the context of perceptual organization. Perceptual organization imparts robustness, efficiency, and a qualitative and holistic nature to vision. Thus far, the approaches to the problem of perceptual organization have been purely bottom up, without much top-down knowledge-base influence, and are therefore entirely dependent on the inputs, which are obviously imperfect. The knowledge base, besides coping with such input imperfection, also makes it possible to integrate multiple organizations and form a composite organization hypothesis. The PIN imparts an active inferential and integrating nature to perceptual organization in an elegant probabilistic framework.>

Depth maps are frequently analyzed as if the errors are normally, identically, and independently distributed. This noise model does not consider at least two types of anomalies encountered in sampling: a few large deviations in the data (outliers) and a uniformly distributed error component arising from rounding and quantization. The theory of robust statistics, which formally addresses these problems, is used in a robust sequential estimator (RSE) of the authors' design. The RSE assigns different weights to each observation based on maximum-likelihood analysis, assuming that the errors follow a t distribution which represents the outliers more realistically. This concept is extended to several well-known maximum-likelihood estimators (M-estimators). Since most M-estimators do not have a target distribution, the weights are obtained by a simple linearization and then embedded in the same RSE algorithm. Experimental results over a variety of real and synthetic range imagery are presented, and the performance of these estimators is evaluated under different noise conditions.>

The role of perceptual organization in computer vision systems is explored. This is done from four vantage points. A brief history of perceptual organization research in both humans and computer vision is offered. A classificatory structure in which to cast perceptual organization research to clarify both the nomenclature and the relationships among the many contributions is proposed. The perceptual organization work in computer vision in the context of this classificatory structure is reviewed. The array of computational techniques applied to perceptual organization problems in computer vision is surveyed.>

Modeling of the unknown surface, a key first step in the perception of surfaces in range images using the function approximation approach, is considered. Akaike's entropy-based information criterion (AIC) is a simple but powerful tool for choosing the best fitting model among several competing models. However, the AIC presupposes a fixed data set and a normality assumption on the error's distribution. The AIC is extended to a t-distribution noise model, which more realistically represents anomalies in the data such as outliers and quantization errors. This criterion is modified to be used with a robust sequential algorithm to accommodate the variable data size resulting from fitting different models. The modified criterion is applied to real range data, and its performance is compared with that of AIC and Consistent AIC.>

It is shown that the formalism of Bayesian networks provides an elegant solution, in a probabilistic framework, to the problem of integrating top-down and bottom-up visual processes as well serving as a knowledge base. The formalism is modified to handle spatial data and thus extends the applicability of Bayesian networks to visual processing. The modified form is called the perceptual inference network (PIN). The theoretical background of a PIN is presented, and its viability is demonstrated in the context of perceptual organization. The PIN imparts an active inferential and integrating nature to perceptual organization.>

Presents an efficient hierarchical computational paradigm for perceptual organization. Organization at each level of the hierarchy is done by graph enumeration on a set of Gestalt graphs. Efficient voting methods are proposed. The authors develop the method in detail and analyze its computational efficiency, considering both time and space. The theoretical and practical results are very encouraging. They strongly advocate the idea of an organizational hierarchy, constructed using graph enumeration. Graph theoretic representations enable one to extract various structures with considerable ease. They evaluated the performance using real images, with good results.>

The authors solve the stereo correspondence problem in uncalibrated domains using extended edge contours as a source of primitives, as opposed to traditional point-based algorithms. This work represents a novel approach to implementation of the structural stereopsis concept of K.L. Boyr and A.C. Kak (1988), in particular with regard to speed. Judiciously exploiting the contiguity relation among primitives, correspondence solutions without prior knowledge of the epipolar condition for feature-rich stereopairs, previously requiring several days of processing by the Boyer and Kak algorithm are now acquired in tens of seconds on the same equipment.>

The authors outline the design of an optimal, computationally efficient, infinite impulse response edge detection filter. The optimal filter is computed based on Canny's high signal to noise ratio, good localization criteria, and a criterion on the spurious response of the filter to noise. An expression for the width of the filter, which is appropriate for infinite-length filters, is incorporated directly in the expression for spurious responses. The three criteria are maximized using the variational method and nonlinear constrained optimization. The optimal filter parameters are tabulated for various values of the filter performance criteria. A complete methodology for implementing the optimal filter using approximating recursive digital filtering is presented. The approximating recursive digital filter is separable into two linear filters, operating in two orthogonal directions. The implementation is very simple and computationally efficient. has a constant time of execution for different sizes of the operator, and is readily amenable to real-time hardware implementation.>

The problem of decomposing an extended boundary or contour into simple primitives is addressed with particular emphasis on Laplacian-of-Gaussian zero-crossing contours. A technique is introduced for partitioning such contours into constant curvature segments. A nonlinear 'blip' filter matched to the impairment signature of the curvature computation process, an overlapped voting scheme, and a sequential contiguous segment extraction mechanism are used. This technique is insensitive to reasonable changes in algorithm parameters and robust to noise and minor viewpoint-induced distortions in the contour shape, such as those encountered between stereo image pairs. The results vary smoothly with the data, and local perturbations induce only local changes in the result. Robustness and insensitivity are experimentally verified.>

Dynamic edge warping (DEW), a technique for recovering reasonably accurate disparity maps from uncalibrated stereo image pairs, is presented. No precise knowledge of the epipolar camera geometry is assumed. The technique is embedded in a system including structural stereopsis on the front end and robust estimation in digital photogrammetry on the other for the purpose of self-calibrating stereo image pairs. Once the relative camera orientation is known, the epipolar geometry is computed and the system can use this information to refine its representation of the object space. Such a system will find application in the autonomous extraction of terrain maps from stereo aerial photographs, for which camera position and orientation are unknown a priori, and for online autonomous calibration maintenance for robotic vision applications, in which the cameras are subject to vibration and other physical disturbances after calibration. This work thus forms a component of an intelligent system that begins with a pair of images and, having only vague knowledge of the conditions under which they were acquired, produces an accurate, dense, relative depth map. The resulting disparity map can also be used directly in some high-level applications involving qualitative scene analysis, spatial reasoning, and perceptual organization of the object space. The system as a whole substitutes high-level information and constraints for precise geometric knowledge in driving and constraining the early correspondence process.>

A technique, dynamic edge warping, for recovering reasonable disparity maps from uncalibrated stereo image pairs is presented. No precise knowledge of the epipolar camera geometry is assumed. The technique is part of a system including structural stereopsis and digital photogrammetry for self-calibrating stereo image pairs with application in autonomous extraction of terrain maps from stereo aerial photographs and autonomous calibration maintenance in robotic vision. The system substitutes high-level information and constraints for precise geometric knowledge, in constraining the matching process.>

The design of an optimal, efficient, infinite-impulse-response (IIR) edge detection filter is described. J. Canny (1986) approached the problem by formulating three criteria designed in any edge detection filter: good detection, good localization, and low spurious response. He maximized the product of the first two criteria while keeping the spurious response criterion constant. Using the variational approach, he derived a set of finite extent step edge detection filters corresponding to various values of the spurious response criterion, approximating the filters by the first derivative of a Gaussian. A more direct approach is described in this paper. The three criteria are formulated as appropriate for a filter of infinite impulse response, and the calculus of variations is used to optimize the composite criteria. Although the filter derived is also well approximated by first derivative of a Gaussian, a superior recursively implemented approximation is achieved directly. The approximating filter is separable into two linear filters operating in two orthogonal directions allowing for parallel edge detection processing. The implementation is very simple and computationally efficient.>

In a recent paper by J.S. Chen et al. (ibid., vol.PAMI-9, p.584-90, July 1987) the authors presented a means of decomposing the Laplacian-of-Gaussian (LoG) kernel into the product of a Gaussian and a (smaller) LoG mask. They then proceeded to develop a fast algorithm for convolution which exploits the spatial frequency properties of these operators to allow the image to be decimated (subsampled). Although this approach is both novel and interesting, it is contended that the exposition suffers from some inconsistencies and minor errors. The commenters clarify matters for those who wish to implement this technique. The original authors acknowledge two of the three points raised, and provide further clarification of the other one namely, the claim that the masks (Gaussian and LoG) are too small.>

A novel approach to solving the stereo correspondence problem in computer vision is described. Structural descriptions of two two-dimensional views of a scene are extracted by one of possibly several available low-level processes, and a new theory of inexact matching for such structures is derived. An entropy-based figure of merit for attribute selection and ordering is defined. Experimental results applying these techniques to real image pairs are presented. Some manipulation experiments are briefly presented.>

In this paper, we discuss a novel strategy for rapid acquisition of the range map of a scene employing color-encoded structured light. This technique offers several potential advantages including increased speed and improved accuracy. In this approach we illuminate the scene with a single encoded grid of colored light stripes. The indexing problem, that of matching a detected image plane stripe with its position in the projection grid, is solved from a knowledge of the color grid encoding. In fact, the possibility exists for the first time to acquire high-resolution range data in real time for modest cost, since only a single projection and single color image are required. Grid to grid alignment problems associated with previous multistripe techniques are eliminated, as is the requirement for dark interstices between grid stripes. Scene illumination is more uniform, simplifying the stripe detection problem, and mechanical difficulties associated with the equipment design are significantly reduced.