We explore clustering the softmax predictions of deep neural networks and introduce a novel probabilistic clustering method, referred to ask-sBetas. In the general context of clustering discrete distributions, the existing methods focused on exploring distortion measures tailored to simplex data, such as the KL divergence, as alternatives to the standard euclidean distance. We provide a general maximum a posteriori (MAP) perspective of clustering distributions, emphasizing that the statistical models underlying the existing distortion-based methods may not be descriptive enough. Instead, we optimize a mixed-variable objective measuring data conformity within each cluster to the introduced$\mathtt {sBeta}$density function, whose parameters are constrained and estimated jointly with binary assignment variables. Our versatile formulation approximates various parametric densities for modeling simplex data and enables the control of the cluster-balance bias. This yields highly competitive performances for the unsupervised adjustment of black-box model predictions in various scenarios.

We introduce a Parametric Information Maximization (PIM) model for the Generalized Category Discovery (GCD) problem. Specifically, we propose a bi-level optimization formulation, which explores a parameterized family of objective functions, each evaluating a weighted mutual information between the features and the latent labels, subject to supervision constraints from the labeled samples. Our formulation mitigates the class-balance bias encoded in standard information maximization approaches, thereby handling effectively both short-tailed and long-tailed data sets. We report extensive experiments and comparisons demonstrating that our PIM model consistently sets new state-of-the-art performances in GCD across six different datasets, more so when dealing with challenging fine-grained problems. Our code: https://github.com/ThalesGroup/pim-generalized-category-discovery.

The detection of epileptic seizures in EEG signals is a challenging task because it requires careful review of multi-channel EEG recordings over a lengthy time interval. In general, EEG-based seizure detection is strongly dependent on the ability to select descriptive features that are also stable in the sense that they are not sensitive to changes in the training data. This study proposes and investigates a patient-independent seizure detection model that uses stable EEG-based features obtained by comparing multiple feature selection methods. The schemes considered can be divided into five categories, often referred to as similarity, information theoretic, sparse learning, statistical, and graph centrality feature selection methods. The stability of the features these schemes produce was evaluated by random forest classification, using the intersection, frequency, correlation, and similarity measures. In experiments with Temple University Hospital’s Seizure database of EEG records of 341 patients, unsupervised graph centrality feature selection was the most effective method, with a correct classification rate of 91.5%.

In the context of recent deep clustering studies, discriminative models dominate the literature and report the most competitive performances. These models learn a deep discriminative neural network classifier in which the labels are latent. Typically, they use multinomial logistic regression posteriors and parameter regularization, as is very common in supervised learning. It is generally acknowledged that discriminative objective functions (e.g., those based on the mutual information or the KL divergence) are more flexible than generative approaches (e.g., K-means) in the sense that they make fewer assumptions about the data distributions and, typically, yield much better unsupervised deep learning results. On the surface, several recent discriminative models may seem unrelated to K-means. This study shows that these models are, in fact, equivalent to K-means under mild conditions and common posterior models and parameter regularization. We prove that, for the commonly used logistic regression posteriors, maximizing the L2L2 regularized mutual information via an approximate alternating direction method (ADM) is equivalent to minimizing a soft and regularized K-means loss. Our theoretical analysis not only connects directly several recent state-of-the-art discriminative models to K-means, but also leads to a new soft and regularized deep K-means algorithm, which yields competitive performance on several image clustering benchmarks.

This study investigates a regularised differentiation method to estimate image derivatives. The scheme minimises an integral functional containing an anti‐differentiation data discrepancy term and a smoothness regularisation term. When discretised, the Euler–Lagrange necessary conditions for a minimum of the functional yield a large scale sparse system of linear equations, which can be solved efficiently by Jacobi/Gauss–Seidel iterations. The authors investigate the impact of the method in the context of two important problems in computer vision: optical flow and scene flow estimation. Quantitative results, using the Middlebury dataset and other real and synthetic images, show that the authors’ regularised differentiation scheme outperforms standard derivative definitions by smoothed finite differences, which are commonly used in motion analysis. The method can be readily used in various other image analysis problems.

This study investigates the recovery of region boundary patterns in an image by a variational level set method which drives an active curve to coincide with boundaries on which a feature distribution matches a reference distribution. We formulate the scheme for both the Kullback-Leibler and the Bhattacharyya similarities, and apply it in two conditions: the simultaneous recovery of all region boundaries consistent with a given outline pattern, and segmentation in the presence of faded boundary segments. The first task uses an image-based geometric feature, and the second a photometric feature. In each case, the corresponding curve evolution equation can be viewed as a geodesic active contour (GAC) flow having a variable stopping function which depends on the feature distribution on the active curve. This affords a potent global representation of the target boundaries, which can effectively drive active curve segmentation in a variety of otherwise adverse conditions. Detailed experimentation shows that the scheme can significantly improve on current region and edge-based formulations.

Road network extraction from digitized map consists in partitioning a map into two different classes (road and background) distinguished with their intensity values and geometric shape. In this paper, we propose a semi-automatic method where user intervention is minimal. We adopt a variational framework where a minimization energy is proposed. Roads are tracked by active contours which evolve according to evolution equations derived from the energy minimization. We consider that the energy is composed of three main terms which have never been combined and used in such application. The first term is the geodesic active contour (GAC) which is contour-based and which evolves toward high gradients in the image. The second term matches the intensity distribution inside the contour with a model distribution. The third term is a geometric constraint which describes the thin structure of routes. The user intervention is restricted to learning the road intensity model.

The purpose of this study is to investigate multiregion graph cut image partitioning via kernel mapping of the image data. The image data is transformed implicitly by a kernel function so that the piecewise constant model of the graph cut formulation becomes applicable. The objective function contains an original data term to evaluate the deviation of the transformed data, within each segmentation region, from the piecewise constant model, and a smoothness, boundary preserving regularization term. The method affords an effective alternative to complex modeling of the original image data while taking advantage of the computational benefits of graph cuts. Using a common kernel function, energy minimization typically consists of iterating image partitioning by graph cut iterations and evaluations of region parameters via fixed point computation. A quantitative and comparative performance assessment is carried out over a large number of experiments using synthetic grey level data as well as natural images from the Berkeley database. The effectiveness of the method is also demonstrated through a set of experiments with real images of a variety of types such as medical, synthetic aperture radar, and motion maps.

This study investigates an active curve functional which measures a similarity between the distribution of an image feature on the curve and a model distribution learned a priori. The curve evolution equation resulting from the minimization of this contour-based functional can be viewed as a geodesic active contour with a variable stopping function. The variable stopping function depends on the distribution of image feature on the curve and, therefore, can deal with difficult cases where the desired boundary corresponds to very weak image transitions. We ran several experiments supported by quantitative performance evaluations over several examples of segmentation and tracking of the left ventricle inner and outer boundaries in cardiac magnetic resonance image sequences. The results are significantly more accurate than with region-based and edge-based functionals.

A novel multiregion graph cut image partitioning method combined with kernel mapping is presented. A kernel function transforms implicitly the image data into data of a higher dimension so that the piecewise constant model of the graph cut formulation becomes applicable. The method yields an effective alternative to complex modeling of the original image data while taking advantage of the rapidity of graph cuts. A variety of noise models are, thus, considered by a single model. Using a common kernel function, we minimize the objective functional by iterating (1) regions parameters update and (2) image partitioning by graph cut iterations. A comparative performance evaluation is carried out over a large set of experiments using synthetic grey level data. Besides, a set of tests with real images such as SAR and medical images is shown to demonstrate the validity of the method.

Surgical instrumentation for the Adolescent idiopathic scoliosis (AIS) is a complex procedure involving many difficult decisions. Selection of the appropriate fusion level remains one of the most challenging decisions in scoliosis surgery. Currently, the Lenke classification model is generally followed in surgical planning. The purpose of our study is to investigate a computer aided method for Lenke classification and scoliosis fusion level selection. The method uses a self organizing neural network trained on a large database of surgically treated AIS cases. The neural network produces two maps, one of Lenke classes and the other of fusion levels. These two maps show that the Lenke classes are associated with the the proper fusion level categories everywhere in the map except at the Lenke class transitions. The topological ordering of the Cobb angles in the neural network justifies determining a patient scoliotic treatment instrumentation using directly the fusion level map rather than via the Lenke classification.

This study investigates level set multiphase image segmentation by kernel mapping and piecewise constant modeling of the image data thereof. A kernel function maps implicitly the original data into data of a higher dimension so that the piecewise constant model becomes applicable. This leads to a flexible and effective alternative to complex modeling of the image data. The method uses an active curve objective functional with two terms: an original term which evaluates the deviation of the mapped image data within each segmentation region from the piecewise constant model and a classic length regularization term for smooth region boundaries. Functional minimization is carried out by iterations of two consecutive steps: 1) minimization with respect to the segmentation by curve evolution via Euler-Lagrange descent equations and 2) minimization with respect to the regions parameters via fixed point iterations. Using a common kernel function, this step amounts to a mean shift parameter update. We verified the effectiveness of the method by a quantitative and comparative performance evaluation over a large number of experiments on synthetic images, as well as experiments with a variety of real images such as medical, satellite, and natural images, as well as motion maps.

A novel level set multiphase image segmentation method combined with kernel mapping is presented. A kernel function maps implicitly the original data into data of a higher dimension so that the piecewise constant model becomes applicable. The goal is to consider several types of noise by a single model. Gradient flow equations are iteratively derived in order to minimize the segmentation functional with respect to the partition, in a first step, and the regions parameters in a second step. Using a common kernel function, we verified the effectiveness of the method by a quantitative and comparative performance evaluation over experiments on synthetic images, as well as a variety of real images such as medical, SAR, and natural images.

This study investigates Bayes classification of online Arabic characters represented by histograms of tangent differences and Gibbs modeling of the class-conditional probability density functions. The parameters of these Gibbs density functions are estimated following the Zhu, Wu, and Mumford constrained maximum entropy formalism, originally introduced for image and shape synthesis. We investigate two partition function estimation methods: one uses the training sample and the other draws from a reference distribution. The efficiency of the corresponding Bayes decision methods, and of a combination of these, is shown in experiments using a database of 9504 freely written samples by 22 scriptors. Comparisons to the nearest neighbor rule method and Kohonen neural network methods are provided.

In current level set image segmentation methods, the number of regions is assumed to known beforehand. As a result, it remains constant during the optimization of the objective functional. How to allow it to vary is an important question which has been generally avoided. This study investigates a region merging prior related to regions area to allow the number of regions to vary automatically during curve evolution, thereby optimizing the objective functional implicitly with respect to the number of regions. We give a statistical interpretation to the coefficient of this prior to balance its effect systematically against the other functional terms. We demonstrate the validity and efficiency of the method by testing on real images of intensity, color, and motion.

This study investigates a new multiphase minimization scheme which embeds a simple, efficient partition constraint directly in multiple level set evolution. Starting from an arbitrary initial partition, the minimization of the N-region segmentation functional is carried out following a first order expansion of the data term with an embedded partition constraint: if a point leaves a region, it goes to single other region. The method has a computational advantage over previous multiphase schemes, is stepwise optimal, i.e., permit to effect the maximum decrease of the functional at each evolution step, and is robust to initialization. The method is discussed by comparison with previous methods and experimental results are included to this effect.

We address unsupervised variational segmentation of multi-look complex polarimetric images using a Wishart observation model via level sets. The methods consists of minimizing a functional containing an original data term derived from maximum likelihood Wishart approximation and a classical boundary length prior. The minimization is carried out efficiently by first order expansion of the data term and a new multiphase method which embeds a simple partition constraint directly in curve evolution. Results are shown on both synthetic and real images. Quantitative performance evaluation and comparisons with another method are also given.

This study investigates a level set method for complex polarimetric image segmentation. It consists of minimizing a functional containing an original observation term derived from maximum-likelihood approximation and a complex Wishart/Gaussian image representation and a classical boundary length prior. The minimization is carried out efficiently by a new multiphase method which embeds a simple partition constraint directly in curve evolution to guarantee a partition of the image domain from an arbitrary initial partition. Results are shown on both synthetic and real images. Quantitative performance evaluation and comparisons are also given.

This study investigates a variational, active curve evolution method for dense three-dimentional (3D) segmentation and interpretation of optical flow in an image sequence of a scene containing moving rigid objects viewed by a possibly moving camera. This method jointly performs 3D motion segmentation, 3D interpretation (recovery of 3D structure and motion), and optical flow estimation. The objective functional contains two data terms for each segmentation region, one based on the motion-only equation which relates the essential parameters of 3D rigid body motion to optical flow, and the other on the Horn and Schunck optical flow constraint. It also contains two regularization terms for each region, one for optical flow, the other for the region boundary. The necessary conditions for a minimum of the functional result in concurrent 3D-motion segmentation, by active curve evolution via level sets, and linear estimation of each region essential parameters and optical flow. Subsequently, the screw of 3D motion and regularized relative depth are recovered analytically for each region from the estimated essential parameters and optical flow. Examples are provided which verify the method and its implementation.

The purpose of this study is to investigate a variational method for joint segmentation and parametric estimation of image motion by basis function representation of motion and level set evolution. The functional contains three terms. One term is of classic regularization to bias the solution toward a segmentation with smooth boundaries. A second term biases the solution toward a segmentation with boundaries which coincide with motion discontinuities, following a description of motion discontinuities by a function of the image spatio-temporal variations. The third term refers to region information and measures conformity of the parametric representation of the motion of each region of segmentation to the image spatio-temporal variations. The components of motion in each region of segmentation are represented as functions in a space generated by a set of basis functions. The coefficients of the motion components considered combinations of the basis functions are the parameters of representation. The necessary conditions for a minimum of the functional, which are derived taking into consideration the dependence of the motion parameters on segmentation, lead to an algorithm which condenses to concurrent curve evolution, implemented via level sets, and estimation of the parameters by least squares within each region of segmentation. The algorithm and its implementation are verified on synthetic and real images using a basis of cosine transforms.

Studies have shown that the Weibull distribution can model accurately a wide variety of images. Its parameters index a family of distributions which includes the exponential and approximations of the Gaussian and the Raleigh models widely used in image segmentation. This study investigates the Weibull distribution in unsupervised image segmentation and classification by a variational method. The data term of the segmentation functional measures the conformity of the image intensity in each region to a Weibull distribution whose parameters are determined jointly with the segmentation. Minimization of the functional is implemented by active curves via level sets and consists of iterations of two consecutive steps: curve evolution via Euler-Lagrange descent equations and evaluation of the Weibull distribution parameters. Experiments with synthetic and real images are described which verify the validity of method and its implementation.

The purpose of this study is to investigate a variational method for joint multiregion three-dimensional (3-D) motion segmentation and 3-D interpretation of temporal sequences of monocular images. Interpretation consists of dense recovery of 3-D structure and motion from the image sequence spatiotemporal variations due to short-range image motion. The method is direct insomuch as it does not require prior computation of image motion. It allows movement of both viewing system and multiple independently moving objects. The problem is formulated following a variational statement with a functional containing three terms. One term measures the conformity of the interpretation within each region of 3-D motion segmentation to the image sequence spatiotemporal variations. The second term is of regularization of depth. The assumption that environmental objects are rigid accounts automatically for the regularity of 3-D motion within each region of segmentation. The third and last term is for the regularity of segmentation boundaries. Minimization of the functional follows the corresponding Euler-Lagrange equations. This results in iterated concurrent computation of 3-D motion segmentation by curve evolution, depth by gradient descent, and 3-D motion by least squares within each region of segmentation. Curve evolution is implemented via level sets for topology independence and numerical stability. This algorithm and its implementation are verified on synthetic and real image sequences. Viewers presented with anaglyphs of stereoscopic images constructed from the algorithm's output reported a strong perception of depth.

We investigate a method of segmentation of multichannel polarimetric images, such as in laser illuminated or synthetic aperture radar, into a given but arbitrary number of regions via curve evolution and level sets. The algorithm consists of evolving closed curve, within an explicit correspondence between the interiors of curves and regions segmentation, to minimize a multivariate criterion corresponding to the complex Gaussian polarimetric model and a term of smoothness of the boundaries of regions. Results are shown on a polarimetric image.

The purpose of this study is to investigate Synthetic Aperture Radar (SAR) image segmentation into a given but arbitrary number of gamma homogeneous regions via active contours and level sets. The segmentation of SAR images is a difficult problem due to the presence of speckle which can be modeled as strong, multiplicative noise. The proposed algorithm consists of evolving simple closed planar curves within an explicit correspondence between the interiors of curves and regions of segmentation to minimize a criterion containing a term of conformity of data to a speckle model of noise and a term of regularization. Results are shown on both synthetic and real images.

The paper investigates a new representation of shape and its use in handwritten on-line character recognition. This representation is based on the empirical distribution of features such as tangents, and tangent differences at distant points along the character signal. Recognition is carried out by a Kohonen associative memory (also called Kohonen self organizing feature map), trained using this representation, and the Hellinger distance, which measures the distance between distributions. We report on extensive experiments that show the pertinence of the representation and the superior performance of the scheme.

The aim of this study is to introduce a novel method for the simultaneous motion segmentation and dense 3D interpretation of temporal sequences of monocular images. The problem is to recover simultaneously 3D structure, 3D motion, and a motion-based segmentation from the image sequence spatio-temporal variations. Motion in space is considered relative to the viewing system so that both the viewing system and environmental objects are allowed to move. The problem is stated as a 3D motion segmentation problem with simultaneous depth estimation within the regions of segmentation. The Euler-Lagrange equations of minimization of the objective functional lead to curve evolution PDE implemented via level sets.

The purpose of this study is to investigate image segmentation from the viewpoint of image data regularized clustering. From this viewpoint, segmentation into a fixed but arbitrary number N of regions is stated as the simultaneous minimization of N - 1 energy functional, each involving a single region and its complement. The resulting Euler-Lagrange curve evolution equations yield a partition at convergence provided the curves are initialized so as to define an arbitrary partition of the image domain. The method is implemented via level sets, and results are shown on synthetic and natural vectorial images.

Active contours and level sets provide a solid formal framework for image segmentation. The problem, stated as the minimization of a functional containing terms of conformity to data and regularization, is solved by curve evolution implemented via level set partial differential equations (PDE). The purpose of this study is to investigate approximation by basis functions as a model for image representation in segmentation by level set PDE. This model is mathematically yielding, affords more generality than current piecewise constant and Gaussian models, and can be just as efficient as the most general piecewise smooth model. We state the problem using this model to measure conformity of segmentation to data. The resulting functional is minimized via level set evolution PDE. Experimental results are shown to demonstrate the formulation.

The purpose of this study is to investigate a method of tracking moving objects with a moving camera. This method estimates simultaneously the motion induced by camera movement. The problem is formulated as a Bayesian motion-based partitioning problem in the spatiotemporal domain of the image quence. An energy functional is derived from the Bayesian formulation. The Euler-Lagrange descent equations determine imultaneously an estimate of the image motion field induced by camera motion and an estimate of the spatiotemporal motion undary surface. The Euler-Lagrange equation corresponding to the surface is expressed as a level-set partial differential equation for topology independence and numerically stable implementation. The method can be initialized simply and can track multiple objects with nonsimultaneous motions. Velocities on motion boundaries can be estimated from geometrical properties of the motion boundary. Several examples of experimental verification are given using synthetic and real-image sequences.

The purpose of this study is to prove convergence results for the Horn and Schunck optical-flow estimation method. Horn and Schunck stated optical-flow estimation as the minimization of a functional. When discretized, the corresponding Euler-Lagrange equations form a linear system of equations We write explicitly this system and order the equations in such a way that its matrix is symmetric positive definite. This property implies the convergence Gauss-Seidel iterative resolution method, but does not afford a conclusion on the convergence of the Jacobi method. However, we prove directly that this method also converges. We also show that the matrix of the linear system is block tridiagonal. The blockwise iterations corresponding to this block tridiagonal structure converge for both the Jacobi and the Gauss-Seidel methods, and the Gauss-Seidel method is faster than the (sequential) Jacobi method.

The purpose of this study is to investigate a method for high performance on-line Arabic characters recognition. This method is based on the use of Kohonen maps and their corresponding confusion matrices which serve to prune them of error-causing nodes, and to combine them consequently. We use two Kohonen maps obtained using two distinct character representations, namely, Fourier descriptors and tangents extracted along the characters online signals. The two Kohonen maps are then combined using a majority vote decision rule that, for each character, favors the most reliable map. This combination, without adding in any significant way to the process complexity, affords a much better recognition rate.

The purpose of this study is to investigate a method of tracking moving objects with a moving camera. This method interprets tracking as detection of the surface generated by motion boundaries in the spatio-temporal domain and estimates simultaneously the motion induced by camera movement. The problem is formulated as a Bayesian motion-based partitioning problem in the spatio-temporal domain of the image sequence. An energy functional is derived from the Bayesian formulation. The Euler Lagrange descent equations determine simultaneously an estimate of the image motion field induced by camera motion and an estimate of the spatio-temporal motion boundary surface. The Euler-Lagrange equation corresponding to the surface is expressed as a level set partial differential equation for topology independence and numerically stable implementation. The method has a simple initialization and allows the tracking of multiple objects with non simultaneous motions. Optical velocities on motion boundaries is estimated from geometrical properties of the motion boundary surface. Several examples of experimental verification are given using synthetic and real image sequences.

In this paper, we propose a novel algorithm for region tracking without motion computation that uses as its starting point the Bayesian framework for tracking previously developed [A.-R. Mansouri, July 2002], [A.R. Mansouri, A. Mitiche, 2002], extending it to a multi-frame tracking algorithm in which tracking is expressed as segmentation in the spatio-temporal domain [A.Mitiche, et al., April 2002]. Our proposed algorithm is expressed as the solution of level set partial differential equations and the tracked region in a particular frame of the sequence is then obtained as the time slice of the level surface given by the level set equations. The main benefit of our proposed algorithm is that contrary to numerous other tracking algorithms, it is a multi-frame tracking algorithm which does not assume the motion to be small [M. Bertalmio, et al., 1998], nor the background to be stationary [S. Jehan-Besson, et al., 2000], nor the region to be uniform in intensity on a uniform background [N. Paragios, R. Deriche, 2000], nor does it assume any motion models [A. Mitiche, et al., April 2002]. This leads to a tracking algorithm which combines the advantages of the Bayesian formulation developed for frame-to-frame tracking [A.-R. Mansouri, July 2002] with those of the formulation in the spatio-temporal domain [A. Mitiche, et al., April 2002]. We illustrate the performance of our algorithm on real image sequences with natural motion.

Motion-based image segmentation is one of the basic problems of image processing and computer vision, with numerous applications to object based image coding and computer vision. This paper describes a novel algorithm for motion-based figure-ground segmentation based on the minimization of a functional for maximum separation between the motions of figure and ground. The Euler-Lagrange descent equations corresponding the minimization of this functional are expressed as level set partial differential equations for a topology-free formulation and stable numerical implementation. These level set partial differential equations dictate the evolution of a surface whose zero level set at convergence is the desired figure ground segmentation. Preliminary results validate our proposed algorithm and demonstrate that accurate figure-ground segmentation based on motion alone and with no prior computation of motion is obtained.

The purpose of this paper is to introduce a fast automated white-noise estimation method which gives reliable estimates in images with smooth and textured areas. This method is a block-based method that takes the image structure into account and uses a measure other than the variance to determine if a block is homogeneous. It uses no thresholds and automates the way that block-based methods stop the averaging of block variances. The proposed method selects intensity-homogeneous blocks in an image by rejecting blocks of structure using a new structure analyzer. The analyzer used is based on high-pass operators and special masks for comers to allow implicit detection of structure and to stabilize the homogeneity estimation. For a typical image quality (PSNR of 20-40 dB) the proposed method outperforms other methods significantly and the worst-case estimation error is 3 dB which is suitable for real applications such as video surveillance or broadcasts. The method performs well even in images with few smooth areas and in highly noisy images.

The basic problem we address in this paper is the following. Given a set of sparse point correspondences, how does one obtain a dense motion field and a motion-based segmentation of the image sequence? We propose a novel solution to this problem by formulating it as pursuit in segmentation space. This segmentation is defined by level set evolution equations, allowing changes in segmentation topology.

Tracking of regions in image sequences plays a fundamental role in applications (search and retrieval in video databases, object based coding such as in MPEG-4, surveillance), and although numerous approaches to region tracking have been developed, they all suffer from severe constraints imposed on the nature of the image sequence. Some assume a particular motion model or constrain the range of interframe motion, while others constrain both the region tracked and the background to have uniform and contrasting intensities. As a result, these tracking algorithms become byproducts of algorithms for motion or intensity boundary detection, and thus have limited applicability. We propose a novel algorithm for region tracking that uses the Bayesian framework for tracking previously developed. We extend this framework by re-expressing tracking in terms of Kullback-Leibler divergence of specific probability distributions and generalizing these to empirical distributions computed over image neighborhoods, leading to level set equations in terms of local image statistics. The main novelty of our proposed algorithm is that contrary to other tracking algorithms which are expressed as level set PDEs, the motion is not assumed to be small, nor is the background assumed to be stationary, nor is the region supposed to be uniform and have strong contrast with the background. We illustrate the performance of our algorithm on real image sequences with natural motion.

In this study we seek a fast method for robust, boundary preserving estimation of optical flow. Several studies have addressed this topic and proposed methods that account for velocity boundaries at the cost of significant computational complexity. The answer we bring consists of adapting the benchmark algorithm of Horn and Schunck such that it produces robust boundary preserving estimates, while retaining its simplicity and speed of execution. Experimental verification on real image sequences is conducted.

The purpose of this paper is two-fold: to give a synoptic description of favored neural networks and to characterize the potency of these neural networks as pattern classifiers, against the background of the familiar nearest neighbors classification. We limit the study to those neural network structures most commonly used for pattern classification: the multilayer perceptron, the Kohonen associative memory, and the Carpenter–Grossberg clustering network, for which we give a tutorial description with the aim of making the driving concepts apparent. The nearest neighbors rule is presented with improved nearest neighbor search and reference data sample pruning. To gain some familiarity with the classifiers, we expound the sequence of computations implicated in pattern category assignment by each classifier. A characterization of the classifiers is drawn from observed and expected properties and from experiments in automatic target recognition and optical character recognition as summarized in comparative tables of performance. This characterization supports the suggestion that nearest neighbors classification always be considered before endorsing alternative pattern classifiers such as neural networks.

This paper presents an approach to tracking human motion in a sequence of monocular images. The process consists of detecting motion, segmenting moving subjects by recovering the background and, finally, tracking the subject of interest. The usual assumptions of small image motion, a fixed viewing system and constant velocity are systematically relaxed. Two cases are studied: (1) a viewing system with negligible motion, and (2) a viewing system with non-negligible motion.

We give a methodology for the evaluation and comparison of object recognition systems. The methodology is based on indicators of two kinds: (1) statistical and (2) algorithmic. Statistical indicators measure the significance of the performance difference between different systems and rank the performances when the difference is significant. Of the various statistical indicators, we use the Kruskal-Wallis H test. Algorithmic indicators include the usual space and time complexity measures, and various performance curves in variables of error and test data sample size. We illustrate the methodology to evaluate a number of pattern classifiers.

Locating the image of moving objects in a scene and estimating their motion are among the main issues in dynamic scene analysis. They are relevant to a large variety of tasks such as autonomous navigation, tracking, obstacle detection and surveillance. A central problem faced by existing methods is related to the occurrence of motion boundaries. With the aim of achieving correct and robust interpretation in the face of occurrence of motion boundaries, noise, and insufficiently informative image brightness pattern regions, the method we propose is articulated about the fundamental concepts of gradient-based multiconstraint, statistically robust regression, and regularization. >

Independent decisions by two high performance nearest-neighbor hand-printed digit classifiers are combined in a principled manner. Three combination methods are investigated: Bayesian combination, Dempster-Shafer evidential reasoning, and dynamic classifier selection. On a test set of 60,000 hand-printed digits, dynamic classifier selection performs slightly better than Bayesian or Dempster-Shafer evidential reasoning, but the lowest error rate is obtained by K-nearest-neighbor combination. Single-parameter classifier combination is used to generate error-reject curves. Essential error-free classification is obtained at the cost of 4% rejects. The zero-reject error rate decreases from 1.18% for the best single classifier system to 0.67% for the combined classifier.>

The problem is considered of using two distinct views taken from unknown viewpoints of a number of points of which optical flow was known, to recover the position of the points in space, their motion, and the relative displacement of the viewing systems. The viewing systems were assumed stationary and the object moving. A linear method was proposed where the relative displacement of the viewpoints was first determined. Once this displacement was computed, the position of the points in space, and their motion, were given straightforwardly.>

A brief review is given of the current formulations of the problem of recovering 3D structure and motion from image line correspondences, and their minimum requirement is formally established in terms of the number of views and lines. Only general rigid motion is considered. Computation of the rotational component and translation component of general motion can be separated. When the rotational component is determined, computation of the translation component is straightforward.>

Discusses the problem of interpreting the images of straight lines in space. The organized use of computational units tuned to the occurrence of special yet commonly found, configurations in space is suggested. The organized use of these units circumvents the inherent difficulties in interpreting general configurations of lines. A comprehensive rule-based system is described for space interpretation which, in particular, will provide the capability of recognizing particular space configurations, or hypothesizing the existence of such configurations and assessing the consistency or inconsistency of resulting interpretations.>

The authors present vision system based on the object-properties-relationships abstraction. It combines different approaches to knowledge representation and uses structured rules which support goal-directed as well as data driven inferencing, and accommodates some degree of parallel processing. The purpose of the system is to draw a distinction between domain-independent and domain-specific structures and processes.>

Two measures of smoothness of a vector field over a domain are suggested, together with associated smoothing operators that satisfy given constrains. The technique is illustrated using operators that smooth the invariants of optical flow curl and divergence, estimated from intensity profiles of moving nondeformable objects. The operators have been applied with satisfactory results to derived flow estimates with high noise levels.>

The problem of computing structure and motion from the observation of points in two distinct images of a scene is considered. The proposed method explicitly utilizes the principle of conservation of distance during rigid-body motion. The formulation is such that it separates the problem of estimating object position from that of determining motion parameters. The equations of invariance of distance for a rigid body are solved for the points' position in space. When these coordinates in space are known, motion parameters are computed in a simple and straightforward manner. Examples are given to illustrate the efficiency of the algorithm.>

In this correspondence, algorithms are introduced to infer surface orientation and structure of visible object surfaces using grid coding. We adopt the active lighting technique to spatially ``encode'' the scene for analysis. The observed objects, which can have surfaces of arbitrary shape, are assumed to rest on a plane (base plane) in a scene which is ``encoded'' with light cast through a grid plane. Two orthogonal grid patterns are used, where each pattern is obtained with a set of equally spaced stripes marked on a glass pane. The scene is observed through a camera and the object surface orientation is determined using the projected patterns on the object surface. If the surfaces under consideration obey certain smoothness constraints, a dense orientation map can be obtained through proper interpolation. The surface structure can then be recovered given this dense orientation map. Both planar and curved surfaces can be handled in a uniform manner. The algorithms we propose yield reasonably accurate results and are relatively tolerant to noise, especially when compared to shape-from-shading techniques. In contrast to other grid coding techniques reported which match the grid junctions for depth reconstruction under the stereopsis principle, our techniques use the direction of the projected stripes to infer local surface orientation and do not require any correspondence relationship between either the grid lines or the grid junctions to be specified. The algorithm has the ability to register images and can therefore be embedded in a system which integrates knowledge from multiple views.

This correspondence deals with the computation of structure and motion of rigid objects in space from image positions and optical flow. A test for rigid motion of objects in space is introduced which yields a new formulation of the problem. Assuming a central projection model for the viewing system, it is shown that image positions and optical flow at four points can achieve this perception.

Range data provide an important source of 3-D shape information. This information can be used to extract jump boundaries which correspond to occluding boundaries of objects in a scene and ``edges'' which correspond to points lying between significantly different regions on the surface of objects. We are mainly interested in range data obtained from sensors such as lasers. The main problem with this type of range finder is the fact that the accuracy of the measurements depends on the power of the signal that reaches the receiver. This study describes how a range edge detection procedure can be designed that has low sensitivity to noise and imbeds all the knowledge available on the range measurement accuracy.

Range data provide an important source of 3-D shape information. This information can be used to extract jump boundaries which correspond to occluding boundaries of objects in a scene and "edges" which correspond to points lying between significantly different regions on the surface of objects. We are mainly interested in range data obtained from sensors such as lasers. The main problem with this type of range finders is the fact that the accuracy of the measurements depends on the power of the signal that reaches the receiver. This study describes how a range edge detection procedure can be designed that has low sensitivity to noise and imbeds all the knowledge available on the range measurement accuracy.