A key challenge in extending vision-linguistic models to new video domains is curating large annotated datasets. We propose a framework that leverages videos with noisy linguistic descriptions, such as sports broadcasts, to train a model using an uncurated dataset. We introduce an unsupervised model that uses the corpus membership between a target and an auxiliary corpus to assign a relevance probability to the linguistic description of examples in the target domain. We examine these probabilities to evaluate the effect of noisy data in the video-text retrieval task. Our framework provides a domain-invariant recipe for enhancing multi-modal datasets by reducing the noise without requiring the costly manual curation effort. We show that our unsupervised model improves the performance of the video-text retrieval model using readily available hockey broadcast videos with closed-captioning. Furthermore, we propose a multi-modal cross-correlation objective function to obtain additional performance gains. We showcase our proposed framework in the context of a new multi-modal dataset of temporally labeled hockey videos with noisy textual descriptions.

An appearance model is critical for most modern trackers. While numerous novel appearance models have been proposed with demonstrated success, challenges such as occlusion and drifting are still not well addressed. In this paper, we propose a novel contextual bag-of-words (CBOW) discriminative appearance model that appropriately handles drifting and occlusion. Specifically, a contextual region containing both the target and its surroundings is explored to construct a compact representation with two bags-of-words. Each word carries discriminative appearance information that is learned by Bayesian inference. An adaptive updating approach, where the background BOWs of the CBOW model acts as a "sentinel" to prevent the integration of the background appearance with the object model, is introduced to alleviate the drifting problem. Based on CBOW, visual tracking is posed within a Bayesian framework. Moreover, an explicit detection method is employed to handle severe occlusions, which further reduces drifting. Two trackers based on the same CBOW model are implemented using either handcrafted color/texture or deep convolutional features. Our trackers are evaluated based on the popular OTB50 and VOT2015 benchmarks and perform competitively against the current state of the art. In addition, they outperform two recent BOWs trackers by a large margin using the currently available figures of merit. To take into account a tracking breakdown, we propose a new figure of merit called the mean maximum-tracked-frame ratio (MTFR) that evaluates a tracker's temporal persistence without any interruption. Experiments with OTB50 demonstrate the superior robustness of our tracker compared with all other evaluated trackers on the basis of MTFR.

In this letter, we introduce an adaptive metric learning (AML) method for person reidentification. Different from conventional metric learning approaches, which treat all the negative samples equally, AML adaptively classifies the negative samples into three groups and pays different attention to them. By emphasizing the influence of hard negative samples, AML can better mine the discriminative information between positive and negative samples, and thus generate a more effective metric. Furthermore, we also propose a probe-specific reranking (PSR) algorithm to refine the initial ranking list measured by the learned metric. For each probe, PSR constructs a corresponding hypergraph to capture the neighborhood relationship between the probe and its top 100 ranked gallery images. Then, these images are reranked based on their neighborhood affinity in the hypergraph. Extensive experiments on three challenging datasets demonstrate the superiority of both AML and PSR.

We present a novel fusion method based on a multi-task robust sparse representation (MRSR) model and spatial context information to address the fusion of multi-focus gray-level images with misregistration. First, we present a robust sparse representation (RSR) model by replacing the conventional least-squared reconstruction error by a sparse reconstruction error. We then propose a multi-task version of the RSR model, viz., the MRSR model. The latter is then applied to multi-focus image fusion by employing the detailed information regarding each image patch and its spatial neighbors to collaboratively determine both the focused and defocused regions in the input images. To achieve this, we formulate the problem of extracting details from multiple image patches as a joint multi-task sparsity pursuit based on the MRSR model. Experimental results demonstrate that the suggested algorithm is competitive with the current state-of-the-art and superior to some approaches that use traditional sparse representation methods when input images are misregistered.

During a forensic interview, high-stakes deception is very prevalent notwithstanding the heavy consequences that might result. This paper proposes an automated computer vision solution for detecting high-stakes deception based on facial clues. Four deceptive cues (eye-blink, eyebrow motion, wrinkle occurrence and mouth motion) were identified and integrated into a single facial behavior pattern vector for discerning deception and honesty. A Random Forest classifier was trained using an unconstrained video database and applied to classify facial patterns into either deceptive or truthful categories. The labeled database we created was based on open sources such as YouTube. The interview videos used for training and testing the classifier were selected on the basis of high-stakes criminal situations, such as murder or kidnapping, which were later verified by criminal trials. Despite the many uncontrolled factors (illumination, head pose and facial occlusion) in the videos, we have achieved an accuracy of 76.92% when discriminating liars from truth-tellers. This compares well with 80.9% [1], the best extant accurary obtained by experienced interrogators.

This paper discusses the design and implementation of a fully automated comprehensive facial expression detection and classification framework. It uses a proprietary face detector (PittPatt) and a novel classifier consisting of a set of Random Forests paired with support vector machinelabellers. The system performs at real-time rates under imaging conditions, with no intermediate human intervention. The acted still-image Binghamton University 3D Facial Expression database was used for training purposes, while a number of spontaneous expression-labelled video databases were used for testing. Quantitative evidence for qualitative and intuitive facial expression recognition constitutes the main theoretical contribution to the field.

We present a novel approach for video parsing and simultaneous online learning of dominant and anomalous behaviors in surveillance videos. Dominant behaviors are those occurring frequently in videos and hence, usually do not attract much attention. They can be characterized by different complexities in space and time, ranging from a scene background to human activities. In contrast, an anomalous behavior is defined as having a low likelihood of occurrence. We do not employ any models of the entities in the scene in order to detect these two kinds of behaviors. In this paper, video events are learnt at each pixel without supervision using densely constructed spatio-temporal video volumes. Furthermore, the volumes are organized into large contextual graphs. These compositions are employed to construct a hierarchical codebook model for the dominant behaviors. By decomposing spatio-temporal contextual information into unique spatial and temporal contexts, the proposed framework learns the models of the dominant spatial and temporal events. Thus, it is ultimately capable of simultaneously modeling high-level behaviors as well as low-level spatial, temporal and spatio-temporal pixel level changes.

We address the issue of visual saliency from three perspectives. First, we consider saliency detection as a frequency domain analysis problem. Second, we achieve this by employing the concept of nonsaliency. Third, we simultaneously consider the detection of salient regions of different size. The paper proposes a new bottom-up paradigm for detecting visual saliency, characterized by a scale-space analysis of the amplitude spectrum of natural images. We show that the convolution of the image amplitude spectrum with a low-pass Gaussian kernel of an appropriate scale is equivalent to an image saliency detector. The saliency map is obtained by reconstructing the 2D signal using the original phase and the amplitude spectrum, filtered at a scale selected by minimizing saliency map entropy. A Hypercomplex Fourier Transform performs the analysis in the frequency domain. Using available databases, we demonstrate experimentally that the proposed model can predict human fixation data. We also introduce a new image database and use it to show that the saliency detector can highlight both small and large salient regions, as well as inhibit repeated distractors in cluttered images. In addition, we show that it is able to predict salient regions on which people focus their attention.

Detection of suspicious activities in public transport areas using video surveillance has attracted an increasing level of attention. In general, automated offline video processing systems have been used for post-event analysis, such as forensics and riot investigations. However, very little has been achieved regarding real-time event recognition. In this paper, we introduce a framework that processes raw video data received from a fixed color camera installed at a particular location, which makes real-time inferences about the observed activities. First, the proposed framework obtains 3-D object-level information by detecting and tracking people and luggage in the scene using a real-time blob matching technique. Based on the temporal properties of these blobs, behaviors and events are semantically recognized by employing object and interobject motion features. A number of types of behavior that are relevant to security in public transport areas have been selected to demonstrate the capabilities of this approach. Examples of these are abandoned and stolen objects, fighting, fainting, and loitering. Using standard public data sets, the experimental results presented here demonstrate the outstanding performance and low computational complexity of this approach. We also discuss the advantages over other approaches in the literature.

We propose a new saliency detection model by combining global information from frequency domain analysis and local information from spatial domain analysis. In the frequency domain analysis, instead of modeling salient regions, we model the nonsalient regions using global information; these so-called repeating patterns that are not distinctive in the scene are suppressed by using spectrum smoothing. In spatial domain analysis, we enhance those regions that are more informative by using a center-surround mechanism similar to that found in the visual cortex. Finally, the outputs from these two channels are combined to produce the saliency map. We demonstrate that the proposed model has the ability to highlight both small and large salient regions in cluttered scenes and to inhibit repeating objects. Experimental results also show that the proposed model outperforms existing algorithms in predicting objects regions where human pay more attention. 1

Many state-of-the-art algorithms for object class recognition have recently appeared in the literature. These algorithms recognize one object at a time in that a dedicated classifier needs to be trained for each object class. However, no paper has yet reported a single classifier capable of recognizing the object class of any one of a number of classes. This paper sets out to recognize objects belonging to multiple classes, irrespective of the viewing and illumination conditions, using just a single classifier. This is achieved by means of the dimensionality reduction technique, Locally Linear Embedding. Object clusters corresponding to a specific range of views in the lower dimensional space are generated using supervised LLE. A test image is then projected into this space where it is labeled by a simple classifier. We have tested our technique on five object classes and attained recognition results comparable to those achieved in the literature using multiple, not single classifiers. 1

Sensor-based discovery (i.e. dynamic) path planning is problematic because the path needs to be continually recomputed as new information is discovered. A process-based client-server approach is presented that permits concurrent sensor-based map and localization-correction updates as well as concurrent path computation and execution. A potential function is created by solving Laplace's equation (i.e. a harmonic function) using an iteration kernal convolved with an occupancy-grid representation of the current free space. The path produced (i.e. by steepest gradient descent on the harmonic function) is optimal in the sense of globally minimizing the distance to the goal as well as locally minimizing a hitting probability. This helps alleviate the influence of uncertainty on path planning. On a regular grid, the computation of the harmonic function is linear in the total number of grid elements, thus constraining this planner to be local. A global planner-provided that an a priori CAD map of the fixed objects exists-provides information to the local planner about the effects of a global goal. An algorithm for the three typical local scenarios found in indoor office-like environments is presented showing how a global goal is projected into the local context for each case. Since objects are sensed on-the-fly it is possible to discover that passage through a hallway or room may be blocked. A zero gradient vector in the local potential function is used to signal a blocked passage and subsequently initiate global replanning. The technique has been used to control a Nomadics 200 robot.

A novel approach to 3D part segmentation is presented. It is a well-known physical fact that electrical charge on the surface of a conductor tends to accumulate at a sharp convexity and vanish at a sharp concavity. Thus, object part boundaries, which are usually denoted by a sharp surface concavity, can be detected by simulating the electrical charge density over the object surface and locating surface points which exhibit local charge density minima. Beginning with single- or multiview range data of a 3D object, we simulate the charge density distribution over an object's surface which has been tessellated by a triangular mesh. We detect the deep surface concavities by tracing local charge density minima and then decompose the object into parts at these points. The charge density computation does not require an assumption on surface smoothness and uses weighted global data to produce robust local surface features for part segmentation.

A novel approach to 3D part segmentation is presented. Beginning with range data of a 3D object, we simulate the charge density distribution over an object's surface which has been tessellated by a triangular mesh. We then locate the object part boundary at deep surface concavities by tracing local charge density minima. Finally, we decompose the object into parts at the part boundary points.

Provides an overview of the technical articles and features presented in this issue.

At the retinal level, the strategies utilized by biological visual systems allow them to outperform machine vision systems, serving to motivate the design of electronic or "smart" sensors based on similar principles. Design of such sensors in silicon first requires a model of retinal information processing which captures the essential features exhibited by biological retinas. In this paper, a simple retinal model is presented, which qualitatively accounts for the achromatic information processing in the primate cone system. The computer retina model exhibits many of the properties found in biological retinas such as data reduction through nonuniform sampling, adaptation to a large dynamic range of illumination levels, variation of visual acuity with illumination level, and enhancement of spatiotemporal contrast information. The main emphasis of the model presented here is to demonstrate how different adaptation mechanisms play a role in extending the operating range of the primate retina.

For pt.I. see ibid., p.259-74. This paper presents the results of experiments performed to validate the computer retina model presented in part I. Experiments commonly performed by electrophysiologists on biological retinas are simulated and the computer retina outputs compared with published recordings of cells in biological (principally primate) retinas. Experiments with more complex stimuli further reveal how the computer retina enhances spatio-temporal contrast information and adapts to a wide range of illumination levels much like the primate retina.

We present a machine vision system in which segmentation is computed in conjunction with a structural description of objects in the scene. It is assumed that contrast edges capture all relevant object information. The principles which dictate how edge features are grouped to infer objects are based upon detecting SYMMETRICAL ENCLOSING edge configurations. These are detected using ANNULAR OPERATORS applied at multiple scales to edge data which have been extracted at multiple scales from a gray level image. The subsequent grouping of symmetry points results in a set of PARTS which make it possible to identify the LOCATION of objects within an image. These parts are used as a basis for constructing coarse graph-based DESCRIPTORS for the PERCEPTUALLY SIGNIFICANT objects found in the scene. Results are presented to illustrate the method's performance on several images.>

It is essential that autonomous robots be able to locate and identify objects in their environment. A novel approach for visually extracting such object information from images is presented. Annular operators are used to identify existing symmetric relationships between sets of edge elements. Operators are applied at multiple scales to edge data which have been extracted at multiple scales from a gray-scale image. From the resulting symmetry points, the authors identify a set of object parts in the scene. These are used as the basis for constructing coarse graph-based object descriptors. Preliminary results are presented to illustrate the approach using natural image data.

Concerns the problem of range image registration for the purpose of building surface models of 3D objects. The registration task involves finding the translation and rotation parameters which properly align overlapping views of the object so as to reconstruct from these partial surfaces, an integrated surface representation of the object. The registration task is expressed as an optimization problem. We define a function which measures the quality of the alignment between the partial surfaces contained in two range images as produced by a set of motion parameters. This function computes a sum of Euclidean distances from control points on one surfaces to corresponding points on the other. The strength of this approach is in the method used to determine point correspondences. It reverses the rangefinder calibration process, resulting in equations which can be used to directly compute the location of a point in a range image corresponding to an arbitrary point in 3D space. A stochastic optimization technique, very fast simulated reannealing (VFSR), is used to minimize the cost function. Dual-view registration experiments yielded excellent results in very reasonable time. A multiview registration experiment took a long time. A complete surface model was then constructed from the integration of multiple partial views. The effectiveness with which registration of range images can be accomplished makes this method attractive for many practical applications where surface models of 3D objects must be constructed.>

Focuses on approximating object part shapes by distinctive types of volumetric primitives. Shape approximation is accomplished by fitting volumetric models called 'parametric geons' to multiview range data of single-part objects and classifying the fitting residuals. Parametric geons are seven qualitative shape types defined by parameterized equations which control the size and degree of tapering and bending. Model fitting is performed by minimizing an objective function which measures the similarity in both size and shape between models and objects. Multiple view data, global shape constraints and global optimization are employed to obtain unique models and to compensate for noise and minor variations in object shape. This approach has been studied in experiments with both synthetic 3D data and actual rangefinder data of perfect and imperfect geon-like objects.>

Computer vision systems are becoming more and more anthropomorphic. In addition to taking an active vision approach, researchers are now investigating robot heads with biologically motivated space-variant sensors. We have designed such an exploring eye system for robot gaze control when observing an arbitrary scene. Our approach uses "retinal" images constructed from overlapping receptive fields that are mapped to form log-polar "cortical" images. We have implemented this system on a parallel SIMD machine using an efficient parallelization algorithm. In order to demonstrate our approach, we have formulated various types of attentional operators that are used to select interest points to drive the gaze control mechanism. We show that our system efficiently integrates images from space-variant snapshots in an arbitrary natural environment.

We have studied the problem of deriving object part approximations by a new set of distinct volumetric shape types called parametric geons from multiview and single-view range data. This is accomplished by fitting the models to range data of single-part objects and then classifying the fitting residuals. We investigate how the number of object views can affect the ultimate shape approximation. Experimental results show that qualitative shape information can be recovered using data taken from single general views, and that multiview data significantly improve the accuracy of the quantitative model information.

Extracting geometric primitives from geometric sensor data is an important problem in model-based vision. A minimal subset is the smallest number of points necessary to define a unique instance of a geometric primitive. A genetic algorithm based on a minimal subset representation is used to perform primitive extraction. It is shown that the genetic approach is an improvement over random search and is capable of extracting more complex primitives than the Hough transform.>

Primal access recognition of visual objects (PARVO), a computer vision system that addresses the problem of fast and generic recognition of unexpected 3D objects from single 2D views, is considered. Recently, recognition by components (RBC), which is a new human image understanding theory, based on some psychological results, has been proposed as an explanation of how PARVO works. However, no systematic computational evaluation of its many aspects has yet been reported. The PARVO system discussed is a first step toward this goal, since its design respects and makes explicit the main assumptions of the proposed theory. It analyzes single-view 2D line drawings of 3D objects typical of the ones used in human image understanding studies. It is designed to handle partially occluded objects of different shape and dimension in various spatial orientations and locations in the image plane. The system is shown to successfully compute generic descriptions and then recognize many common man-made objects.>

The problems of segmenting a noisy intensity image and tracking a nonrigid object in the plane are discussed. In evaluating these problems, a technique based on an active contour model commonly called a snake is examined. The technique is applied to cell locomotion and tracking studies. The snake permits both the segmentation and tracking problems to be simultaneously solved in constrained cases. A detailed analysis of the snake model, emphasizing its limitations and shortcomings, is presented, and improvements to the original description of the model are proposed. Problems of convergence of the optimization scheme are considered. In particular, an improved terminating criterion for the optimization scheme that is based on topographic features of the graph of the intensity image is proposed. Hierarchical filtering methods, as well as a continuation method based on a discrete sale-space representation, are discussed. Results for both segmentation and tracking are presented. Possible failures of the method are discussed.>

A genetic algorithm based on a minimal subset representation of a geometric primitive is used to perform primitive extraction. A genetic algorithm is an optimization method that uses the metaphor of evolution, and a minimal subset is the smallest number of points necessary to define a unique instance of a geometric primitive. The approach is capable of extracting more complex primitives than the Hough transform. While similar to a hierarchical merging algorithm, it does not suffer from the problem of premature commitment.>

The problem of geometric primitive extraction in a range image is considered. A segmentation algorithm segments the image into quadric patches and provides an efficient initialization for the extraction of geometric primitives by a genetic algorithm. These algorithms are shown to be simple, fast and robust to noise. A parallel implementation has been realized and results are shown on real data.>

Proposes an approach for representing an object by its constituent parts and their spatial connections based on a set of primitive volumes called geons. The authors utilize the edge-junction graph of an object range image as input, then decompose this graph into different subgraphs which correspond to the constituent parts of the object. For each part, edge characteristics guide one to an inference regarding the corresponding geon shape. Also, the spatial connections among these parts can be deduced from the concave edges and T-junctions. Experimental results with a laser rangefinder are presented.>

A method for shape description of planar objects that integrates both region and boundary features is presented. The method is an implementation of a 2D dynamic grassfire that relies on a distance surface on which elastic contours minimize an energy function. The method is based on an active contour model. Numerous implementation aspects of the shape description method were optimized. A Euclidean metric was used for optimal accuracy, and the active contour model permits bypassing some of the discretization limitations inherent in using a digital grid. Noise filtering was performed on the basis of both contour feature measures and region measures, that is, curvature extremum significance and ridge support, respectively, to obtain robust shape descriptors. Other improvements and variations of the algorithmic implementation are proposed.>

The authors describe the geometrical criteria which define viewpoint-invariant features to be extracted from 2-D line drawings of 3-D objects. They also discuss the extraction of these features, which forms the initial stage of a generic object recognition system, the Primal Access Recognition of Visual Objects (PARVO) system. In this system, part-based qualitative descriptions are built and matched to coarse 3-D object models for recognition. The segmentation and labeling of the constituent parts of an object rely on the 3-D properties inferred from the presence of its 2-D features. The original motivation for PARVO its recognition by components, a theory of human image understanding from the field of psychology. Definitions of the geometrical criteria defining the viewpoint-invariant features are introduced. Examples of results obtained by applying these criteria to a typical line drawing are shown.>

The problem of segmenting an image provided by a geometric sensor into geometric primitives is addressed by a two-step iterative process. In the first step, the largest connected region bounded by edge pixels is hypothesized as containing a geometric primitive. In the second step, the resulting set of pixels is sent to a robust fitter based on the least median of squares algorithm, to verify whether this hypothesis holds. If the fit is successful, then the geometric primitive is removed from the data; otherwise, a different hypothesis is obtained by decreasing the edge threshold. The process is repeated until the entire image is segmented. The fact that the robust fitter is tolerant of outliers means that the correct segmentation is produced from many different initial hypotheses. The algorithm is demonstrated on a number of range images which are known to contain particular geometric primitives.>

An approach to the problem of finding surfaces of revolution in range data is proposed. It is based on the fact that at least one of two focal surfaces for a surface of revolution degenerates into the axis of rotation. First, the centers of principal curvature are obtained for each point in the range image. Then, the Hough transform is used to detect the axes of rotation by finding the centers which lie on straight lines in space. Finally, the solid of revolution is volumetrically determined by estimating the cross-sectional radius function along each axis. It is concluded that the method can be used for describing and recognizing 3-D objects partially constructed of an unrestricted class of surfaces of revolution which are partially visible in a range image.>

An approach to the computation of a representation of objects with a planar and curved faces using discontinuity features in range images is presented. Edge maps are computed from real laser triangulation images using local operators and shadow analysis, and then structured into an edge-junction graph embedding and quantitative information. This is achieved by appealing to certain concepts of line-drawing analysis adapted to the three-dimensional nature of range imaging. The edge-oriented method is primarily useful for objects that are well-described by their edges. The main advantage of edge-based descriptions is that no fixed surface primitives are assumed. On the other hand, this scheme can only represent objects without any surface crease edges (such as a sphere) by viewpoint-dependent limb edges. This information may be insufficient for the ensuing high-level task. Thus, a generalization of the description format might include a surface analysis to provide a richer representation.>

An apparent contradiction in the shape-from-shading literature is examined. Although practical experience suggests that shape can be inferred from the local analysis of shading, mathematical analyses support the opposite. A criterion for exact surface recovery is derived, and it is shown that, as a result of surface geometry, elliptic, hyperbolic, and doubly curved surfaces could be recovered to reasonable accuracy provided that surface curvature and/or foreshortening were limited. In other words, by relaxing the requirement that surfaces be recovered exactly, it is found that local analysis of shading can provide useful descriptions of shape as evidenced by the results presented.>

The authors describe a hybrid approach to the problem of image segmentation in range data analysis, where hybrid refers to a combination of both region- and edge-based considerations. The range image of 3-D objects is divided into surface primitives which are homogeneous in their intrinsic differential geometric properties and do not contain discontinuities in either depth of surface orientation. The method is based on the computation of partial derivatives, obtained by a selective local biquadratic surface fit. Then, by computing the Gaussian and mean curvatures, an initial region-gased segmentation is obtained in the form of a curvature sign map. Two additional initial edge-based segmentations are also computed from the partial derivatives and depth values, namely, jump and roof-edge maps. The three image maps are then combined to produce the final segmentation. Experimental results obtained for both synthetic and real range data of polyhedral and curved objects are given.>

An automatic joint guidance system was developed as part of a project to develop an expert welding robot. The apparatus consists of a servo-robot laser scanner mounted on a conventional CLOOS model ROMAT 75/76 industrial welding robot. The computer vision system is able to recognize and track the joint to be welded without external information by using range data obtained from scanning the workpiece. An approach and some experiments related to achieving this ultimate goal are discussed. Three different joint types are identified. Tracking of the joint geometry is done by building, online, a three-dimensional model of the joint during the welding process. This approach is attractive because of its complete autonomy for joint recognition and its ability to trace the geometry while at the same time avoiding an elaborate teaching pass prior to the process. The algorithms used have been tested in a simulation using real laser data. The results indicate that it is possible to recognize automatically the joint type and then to track the joint during the process of welding.>

Much of the emphasis in computer vision research has been on recognizing objects. However, in order to perform tasks such as manipulating and avoiding collisions with objects, one needs to consider how to derive appropriate three-dimensional descriptions of objects from available sensor data. A description is given of a computational approach that has been successful in deriving renditions of shaded images in terms of volumetric primitives such as ellipsoids and cylinders.>

Adaptive control and sensory processing techniques in robotic arc welding are discussed. The gas metal arc welding and gas tungsten arc welding processes are considered, along with a literature review of aspects of welding automation. Topics covered include process modeling, detection and measurement of process features, real-time control, and implementation considerations. An approach for an adaptive welding system is presented. The proposed architecture fits within the scope of an ambitious project to develop an expert welding robot. Different levels of automation are discussed, from the decision level to the closed-loop control of process variables and torch trajectory.>

This paper presents a new algorithm to compute skeletons of noisy images of objects which can be described as ``amorphous blobs.'' Such a requirement arose from our research to obtain a better understanding of the role of the pseudopod in leukocyte locomotion. It involves the modeling and detection of pseudopods which are by their nature nonrigid bodies appearing on the cell's surface membrane. By computing skeletons at different resolutions, a filtered version can be produced without violating the constraints imposed by the semantic knowledge of pseudopod morphology. The filtered version incorporates all the significant ``events'' that occur at the different resolutions. The resolution at which the shape is examined is related to the degree of smoothing, in that the lower the resolution gets, the higher the degree of smoothing. Skeleton branches that persist over several scales arise from convexities that are locally as well as globally significant. Their stability is related to their perceptual significance. Our approach is to combine an initial region centered description (skeleton) with a boundary analysis executed at different resolutions. Having computed the skeleton at different scales, we then use those computed at the lower resolutions as a measure of how global the underlying convexity is. Clearly the skeletons computed at higher resolutions represent the exact location and orientation of the underlying convexities.

This paper introduces a general purpose performance measurement scheme for image segmentation algorithms. Performance parameters that function in real-time distinguish this method from previous approaches that depended on an a priori knowledge of the correct segmentation. A low level, context independent definition of segmentation is used to obtain a set of optimization criteria for evaluating performance. Uniformity within each region and contrast between adjacent regions serve as parameters for region analysis. Contrast across lines and connectivity between them represent measures for line analysis. Texture is depicted by the introduction of focus of attention areas as groups of regions and lines. The performance parameters are then measured separately for each area. The usefulness of this approach lies in the ability to adjust the strategy of a system according to the varying characteristics of different areas. This feedback path provides the means for more efficient and error-free processing. Results from areas with dissimilar properties show a diversity in the measurements that is utilized for dynamic strategy setting.

A major problem in robotic vision is the segmentation of images of natural scenes in order to understand their content. This paper presents a new solution to the image segmentation problem that is based on the design of a rule-based expert system. General knowledge about low level properties of processes employ the rules to segment the image into uniform regions and connected lines. In addition to the knowledge rules, a set of control rules are also employed. These include metarules that embody inferences about the order in which the knowledge rules are matched. They also incorporate focus of attention rules that determine the path of processing within the image. Furthermore, an additional set of higher level rules dynamically alters the processing strategy. This paper discusses the structure and content of the knowledge and control rules for image segmentation.

This paper presents a model of motion suitable for cell tracking. It includes a representation for cell dynamics enabling it to maintain a correspondence between successive images of cells undergoing morphological changes. This model is based on a minimization problem whose computational solution is similar in form to a Newton-Rhapson iteration. The model is supported by experimental results from an actual tracking problem.

The objective of a computer vision system is to outline the objects in a picture and label them with an appropriate interpretation. This paper proposes a new paradigm for a modular computer vision system which is both data directed and knowledge based. The system consists of three different types of units, two of which are associative data memories implemented as relational databases. The short-term memory (STM) contains the raw color picture data and the most current interpretations and deductions about the original scene. The long-term memory (LTM) contains a detailed model of the scene under consideration. A collection of analysis processors, each of which is specialized for a particular task, can communicate with both of these memories. The information in the LTM remains unchanged during the analysis, while the STM is being continually updated and revised by the appropriate processors. The latter may be conceived of as being activated by certain data conditions in the STM, and using the information in both the LTM and STM to alter the status of the STM.

An algorithm is presented which partitions a given sample from a multimodal fuzzy set into unimodal fuzzy sets. It is proven that if certain assumptions are satisfied, then the algorithm will derive the optimal partition in the sense of maximum separation.