Human vision is able to immediately recognize novel visual categories after seeing just one or a few training examples. We describe how to add a similar capability to ConvNet classifiers by directly setting the final layer weights from novel training examples during low-shot learning. We call this process weight imprinting as it directly sets weights for a new category based on an appropriately scaled copy of the embedding layer activations for that training example. The imprinting process provides a valuable complement to training with stochastic gradient descent, as it provides immediate good classification performance and an initialization for any further fine-tuning in the future. We show how this imprinting process is related to proxy-based embeddings. However, it differs in that only a single imprinted weight vector is learned for each novel category, rather than relying on a nearest-neighbor distance to training instances as typically used with embedding methods. Our experiments show that using averaging of imprinted weights provides better generalization than using nearest-neighbor instance embeddings.

We present an unsupervised learning framework for the task of monocular depth and camera motion estimation from unstructured video sequences. In common with recent work [10, 14, 16], we use an end-to-end learning approach with view synthesis as the supervisory signal. In contrast to the previous work, our method is completely unsupervised, requiring only monocular video sequences for training. Our method uses single-view depth and multiview pose networks, with a loss based on warping nearby views to the target using the computed depth and pose. The networks are thus coupled by the loss during training, but can be applied independently at test time. Empirical evaluation on the KITTI dataset demonstrates the effectiveness of our approach: 1) monocular depth performs comparably with supervised methods that use either ground-truth pose or depth for training, and 2) pose estimation performs favorably compared to established SLAM systems under comparable input settings.

For many computer vision and machine learning problems, large training sets are key for good performance. However, the most computationally expensive part of many computer vision and machine learning algorithms consists of finding nearest neighbor matches to high dimensional vectors that represent the training data. We propose new algorithms for approximate nearest neighbor matching and evaluate and compare them with previous algorithms. For matching high dimensional features, we find two algorithms to be the most efficient: the randomized k-d forest and a new algorithm proposed in this paper, the priority search k-means tree. We also propose a new algorithm for matching binary features by searching multiple hierarchical clustering trees and show it outperforms methods typically used in the literature. We show that the optimal nearest neighbor algorithm and its parameters depend on the data set characteristics and describe an automated configuration procedure for finding the best algorithm to search a particular data set. In order to scale to very large data sets that would otherwise not fit in the memory of a single machine, we propose a distributed nearest neighbor matching framework that can be used with any of the algorithms described in the paper. All this research has been released as an open source library called fast library for approximate nearest neighbors (FLANN), which has been incorporated into OpenCV and is now one of the most popular libraries for nearest neighbor matching.

We present Local Naive Bayes Nearest Neighbor, an improvement to the NBNN image classification algorithm that increases classification accuracy and improves its ability to scale to large numbers of object classes. The key observation is that only the classes represented in the local neighborhood of a descriptor contribute significantly and reliably to their posterior probability estimates. Instead of maintaining a separate search structure for each class's training descriptors, we merge all of the reference data together into one search structure, allowing quick identification of a descriptor's local neighborhood. We show an increase in classification accuracy when we ignore adjustments to the more distant classes and show that the run time grows with the log of the number of classes rather than linearly in the number of classes as did the original. Local NBNN gives a 100 times speed-up over the original NBNN on the Caltech 256 dataset. We also provide the first head-to-head comparison of NBNN against spatial pyramid methods using a common set of input features. We show that local NBNN outperforms all previous NBNN based methods and the original spatial pyramid model. However, we find that local NBNN, while competitive with, does not beat state-of-the-art spatial pyramid methods that use local soft assignment and max-pooling.

Vision-based tracking of tissue is a key component to enable augmented reality during a surgical operation. Conven- tional tracking techniques in computer vision rely on identifying strong edge features or distinctive textures in a well-lit environ- ment; however endoscopic tissue images do not have strong edge features, are poorly lit and exhibit a high degree of specular reflection. Therefore, prior work in achieving densely populated 3D features for describing tissue surface profiles require complex image processing techniques and have been limited in providing stable, long-term tracking or real-time processing. In this paper, we present an integrated framework for ac- curately tracking tissue in surgical stereo-cameras at real-time speeds. We use a combination of the STAR feature detector and Binary Robust Independent Elementary Features to acquire salient features that can be persistently tracked at high frame rates. The features are then used to acquire a densely-populated map of the deformations of tissue surface in 3D. We evaluate the method against popular feature algorithms in in-vivo animal study video sequences, and we also apply the proposed method to human partial nephrectomy video sequences. We extend the salient feature framework to support region tracking in order to maintain the spatial correspondence of a tracked region of tissue or a medical image registration to the surrounding tissue. In-vitro tissue studies show registration accuracies of 1.3-3.3 mm using a rigid-body transformation method.

A robust robot perception system intended to enable object manipulation needs to be able to accurately identify objects and their pose at high speeds. Since objects vary considerably in surface properties, rigidity and articulation, no single detector or object estimation method has been shown to provide reliable detection across object types to date. This indicates the need for an architecture that is able to quickly swap detectors, pose estimators, and filters, or to run them in parallel or serial and combine their results, preferably without any code modifications at all. In this paper, we present our implementation of such an infrastructure, ReIn (REcognition INfrastructure), to answer these needs. ReIn is able to combine a multitude of 2D/3D object recognition and pose estimation techniques in parallel as dynamically loadable plugins. It also provides an extremely efficient data passing architecture, and offers the possibility to change the parameters and initial settings of these techniques during their execution. In the course of this work we introduce two new classifiers designed for robot perception needs: BiGGPy (Binarized Gradient Grid Pyramids) for scalable 2D classification and VFH (Viewpoint Feature Histograms) for 3D classification and pose. We then show how these two classifiers can be easily combined using ReIn to solve object recognition and pose identification problems.

There has been significant progress recently in object recognition research, but many of the current approaches still fail for object classes with few distinctive features, and in settings with significant clutter and viewpoint variance. One such setting is visual search in mobile robotics, where tasks such as finding a mug or stapler require robust recognition. The focus of this paper is on integrating stereo vision with appearance based recognition to increase accuracy and efficiency. We propose a model that utilizes a chamfer-type silhouette classifier which is weighted by a prior on scale, which is robust to missing stereo depth information. Our approach is validated on a set of challenging indoor scenes containing mugs and shoes, where we find that priors remove a significant number of false positives, improving the average precision by 0.2 on each dataset. We additionally experiment with an additional classifer by Felzenszwalb et al. to demonstrate the approach's robustness.

This paper studies the sequential object recognition problem faced by a mobile robot searching for specific objects within a cluttered environment. In contrast to current state-of-the-art object recognition solutions which are evaluated on databases of static images, the system described in this paper employs an active strategy based on identifying potential objects using an attention mechanism and planning to obtain images of these objects from numerous viewpoints. We demonstrate the use of a bag-of-features technique for ranking potential objects, and show that this measure outperforms geometric matching for invariance across viewpoints. Our system implements informed visual search by prioritising map locations and re-examining promising locations first. Experimental results demonstrate that our system is a highly competent object recognition system that is capable of locating numerous challenging objects amongst distractors.

This paper introduces an affine invariant shape descriptor for maximally stable extremal regions (MSER). Affine invariant feature descriptors are normally computed by sampling the original grey-scale image in an invariant frame defined from each detected feature, but we instead use only the shape of the detected MSER itself. This has the advantage that features can be reliably matched regardless of the appearance of the surroundings of the actual region. The descriptor is computed using the scale invariant feature transform (SIFT), with the resampled MSER binary mask as input. We also show that the original MSER detector can be modified to achieve better scale invariance by detecting MSERs in a scale pyramid. We make extensive comparisons of the proposed feature against a SIFT descriptor computed on grey-scale patches, and also explore the possibility of grouping the shape descriptors into pairs to incorporate more context. While the descriptor does not perform as well on planar scenes, we demonstrate various categories of full 3D scenes where it outperforms the SIFT descriptor computed on grey-scale patches. The shape descriptor is also shown to be more robust to changes in illumination. We show that a system can achieve the best performance under a range of imaging conditions by matching both the texture and shape descriptors.

We apply a biologically inspired model of visual object recognition to the multiclass object categorization problem. Our model modifies that of Serre, Wolf, and Poggio. As in that work, we first apply Gabor filters at all positions and scales; feature complexity and position/scale invariance are then built up by alternating template matching and max pooling operations. We refine the approach in several biologically plausible ways, using simple versions of sparsification and lateral inhibition. We demonstrate the value of retaining some position and scale information above the intermediate feature level. Using feature selection we arrive at a model that performs better with fewer features. Our final model is tested on the Caltech 101 object categories and the UIUC car localization task, in both cases achieving state-of-the-art performance. The results strengthen the case for using this class of model in computer vision.

This paper describes a vision-based system for 3-D localization of a mobile robot in a natural environment. The system includes a mountable head with three on-board charge-coupled device cameras that can be installed on the robot. The main emphasis of this paper is on the ability to estimate the motion of the robot independently from any prior scene knowledge, landmark, or extra sensory devices. Distinctive scene features are identified using a novel algorithm, and their 3-D locations are estimated with high accuracy by a stereo algorithm. Using new two-stage feature tracking and iterative motion estimation in a symbiotic manner, precise motion vectors are obtained. The 3-D positions of scene features and the robot are refined by a Kalman filtering approach with a complete error-propagation modeling scheme. Experimental results show that good tracking and localization can be achieved using the proposed vision system.

We have previously developed a mobile robot system which uses scale-invariant visual landmarks to localize and simultaneously build three-dimensional (3-D) maps of unmodified environments. In this paper, we examine global localization, where the robot localizes itself globally, without any prior location estimate. This is achieved by matching distinctive visual landmarks in the current frame to a database map. A Hough transform approach and a RANSAC approach for global localization are compared, showing that RANSAC is much more efficient for matching specific features, but much worse for matching nonspecific features. Moreover, robust global localization can be achieved by matching a small submap of the local region built from multiple frames. This submap alignment algorithm for global localization can be applied to map building, which can be regarded as alignment of multiple 3-D submaps. A global minimization procedure is carried out using the loop closure constraint to avoid the effects of slippage and drift accumulation. Landmark uncertainty is taken into account in the submap alignment and the global minimization process. Experiments show that global localization can be achieved accurately using the scale-invariant landmarks. Our approach of pairwise submap alignment with backward correction in a consistent manner produces a better global 3-D map.

We present a complete system architecture for fully automated markerless augmented reality (AR). The system constructs a sparse metric model of the real-world environment, provides interactive means for specifying the pose of a virtual object, and performs model-based camera tracking with visually pleasing augmentation results. Our approach does not require camera pre-calibration, prior knowledge of scene geometry, manual initialization of the tracker or placement of special markers. Robust tracking in the presence of occlusions and scene changes is achieved by using highly distinctive natural features to establish image correspondences.

The problem considered in this paper is the fully automatic construction of panoramas. Fundamentally, this problem requires recognition, as we need to know which parts of the panorama join up. Previous approaches have used human input or restrictions on the image sequence for the matching step. In this work we use object recognition techniques based on invariant local features to select matching images, and a probabilistic model for verification. Because of this our method is insensitive to the ordering, orientation, scale and illumination of the images. It is also insensitive to 'noise' images which are not part of the panorama at all, that is, it recognises panoramas. This suggests a useful application for photographers: the system takes as input the images on an entire flash card or film, recognises images that form part of a panorama, and stitches them with no user input whatsoever.

This paper describes a vision-based system for 3D localization and tracking of a mobile robot in an unmodified environment. The system includes a mountable head with three on-board stereo CCD cameras that can be installed on the robot. There the main emphasis is on the ability to estimate the geometric information of the robot independently from any prior scene knowledge, landmark or extra sensory device. Distinctive scene features are identified using a novel algorithm and their 3D locations are estimated with a stereo algorithm. Using multi-stage feature tracking and motion estimation in a symbolic manner, precise motion vectors are obtained. The 3D positions of the scene features are updated by a Kalman filtering process. Experimental results show that robust tracking and localization can be achieved using our vision system.

This paper approaches the problem of finding correspondences between images in which there are large changes in viewpoint, scale and illumination. Recent work has shown that scale-space ‘interest points ’ may be found with good repeatability in spite of such changes. Furthermore, the high entropy of the surrounding image regions means that local descriptors are highly discriminative for matching. For descriptors at interest points to be robustly matched between images, they must be as far as possible invariant to the imaging process. In this work we introduce a family of features which use groups of interest points to form geometrically invariant descriptors of image regions. Feature descriptors are formed by resampling the image relative to canonical frames defined by the points. In addition to robust matching, a key advantage of this approach is that each match implies ahypothesis of the local 2D (projective) transformation. This allows us to immediately reject most of the false matches using a Hough transform. We reject remaining outliers using RANSAC and the epipolar constraint. Results show that dense feature matching can be achieved in a few seconds of computation on 1GHz Pentium III machines. 1

Corner extraction is an important task in many computer vision systems. The quality of the corners and the efficiency of the detection method are two very important aspects that can greatly impact the accuracy, robustness and real-time performance of the corresponding corner-based vision system. In this paper we introduce a fast corner detector based on local binary-image regions. We verify the performance of the proposed method by measuring the repeatability rate under various illumination, scale and motion conditions. Our experimental results show that while the quality of the features is comparable with other conventional methods, ours delivers a faster performance.

We consider the problem of creating a consistent alignment of multiple 3D submaps containing distinctive visual landmarks in an unmodified environment. An efficient map alignment algorithm based on landmark specificity is proposed to align submaps. This is followed by a global minimization using the close-the-loop constraint. Landmark uncertainty is taken into account in the pairwise alignment and the global minimization process. Experiments show that the pairwise alignment of submaps with backward correction produces a consistent global 3D map. Our vision-based mapping approach using sparse 3D data is different from other existing approaches which use dense 2D range data from laser or sonar rangefinders.

We have previously developed a mobile robot system which uses scale invariant visual landmarks to localize and simultaneously build a 3D map of the environment In this paper, we look at global localization, also known as the kidnapped robot problem, where the robot localizes itself globally, without any prior location estimate. This is achieved by matching distinctive landmarks in the current frame to a database map. A Hough transform approach and a random sample consensus (RANSAC) approach for global localization are compared, showing that RANSAC is much more efficient. Moreover, robust global localization can be achieved by matching a small sub-map of the local region built from multiple frames.

There have been important recent advances in object recognition through the matching of invariant local image features. However, the existing approaches are based on matching to individual training images. This paper presents a method for combining multiple images of a 3D object into a single model representation. This provides for recognition of 3D objects from any viewpoint, the generalization of models to non-rigid changes, and improved robustness through the combination of features acquired under a range of imaging conditions. The decision of whether to cluster a training image into an existing view representation or to treat it as a new view is based on the geometric accuracy of the match to previous model views. A new probabilistic model is developed to reduce the false positive matches that would otherwise arise due to loosened geometric constraints on matching 3D and non-rigid models. A system has been developed based on these approaches that is able to robustly recognize 3D objects in cluttered natural images in sub-second times.

A key component of a mobile robot system is the ability to localize itself accurately and build a map of the environment simultaneously. In this paper, a vision-based mobile robot localization and mapping algorithm is described which uses scale-invariant image features as landmarks in unmodified dynamic environments. These 3D landmarks are localized and robot ego-motion is estimated by matching them, taking into account the feature viewpoint variation. With our Triclops stereo vision system, experiments show that these features are robustly matched between views, 3D landmarks are tracked, robot pose is estimated and a 3D map is built.

Our mobile robot system uses scale-invariant visual landmarks to localize itself and build a 3D map of the environment simultaneously. As image features are not noise-free, we carry out error analysis and use Kalman filters to track the 3D landmarks, resulting in a database map with landmark positional uncertainty. By matching a set of landmarks as a whole, our robot can localize itself globally based on the database containing landmarks of sufficient distinctiveness. Experiments show that recognition of position within a map without any prior estimate can be achieved using the scale-invariant landmarks.

This paper presents a vision-based tracking system suitable for autonomous robot vehicle guidance. The system includes a head with three on-board CCD cameras, which can be mounted anywhere on a mobile vehicle. By processing consecutive trinocular sets of precisely aligned and rectified images, the local 3D trajectory of the vehicle in an unstructured environment can be tracked. First, a 3D representation of stable features in the image scene is generated using a stereo algorithm. Next, motion is estimated by trading matched features over time. The motion equation with 6-DOF is then solved using an iterative least squares fit algorithm. Finally, a Kalman filter implementation is used to optimize the world representation of scene features.

An object recognition system has been developed that uses a new class of local image features. The features are invariant to image scaling, translation, and rotation, and partially invariant to illumination changes and affine or 3D projection. These features share similar properties with neurons in inferior temporal cortex that are used for object recognition in primate vision. Features are efficiently detected through a staged filtering approach that identifies stable points in scale space. Image keys are created that allow for local geometric deformations by representing blurred image gradients in multiple orientation planes and at multiple scales. The keys are used as input to a nearest neighbor indexing method that identifies candidate object matches. Final verification of each match is achieved by finding a low residual least squares solution for the unknown model parameters. Experimental results show that robust object recognition can be achieved in cluttered partially occluded images with a computation time of under 2 seconds.

We present a method of indexing 3D objects from single 2D images. The method does not rely on invariant features. This allows a richer set of shape information to be used in the recognition process. We also suggest the kd-tree as an alternative indexing data structure to the standard hash table. This makes hypothesis recovery more efficient in high-dimensional spaces, which are necessary to achieve specificity in large model databases. Search efficiency is maintained in these regimes by the use of best-bin first search. Neighbors recovered from the index are used to generate probability estimates, local within the feature space, which are then used to rank hypotheses for verification. On average, the ranking process greatly reduces the number of verifications required. Our approach is general in that it can be applied to any real-valued feature vector. In addition, it is straightforward to add to our index information from real images regarding the true probability distributions of the feature groupings used for indexing.

We present an integrated system in which an operator uses a simulated environment to program part-mating and contact tasks. Generation of models within this virtual environment is facilitated using a fast, occlusion tolerant, 3D grey-scale vision system which can recognize and accurately locate objects within the work site. A major goal of this work is to make robotic programming easy and intuitive for untrained users working with standard desktop hardware. Simulation offers the ease-of-use benefits of "programming by demonstration", coupled with the ability to create a programmer-friendly virtual environment. Within a simulated environment, it is also straightforward to track and interpret an operator's actions. The simulator models objects as polyhedra and implements full 3D contact dynamics. When a manipulation task is completed, local planning techniques are used to turn the virtual environment's motion sequence history into a set of robot motion commands capable of realizing the prescribed task.

Shape indexing is a way of making rapid associations between features detected in an image and object models that could have produced them. When model databases are large, the use of high-dimensional features is critical, due to the improved level of discrimination they can provide. Unfortunately, finding the nearest neighbour to a query point rapidly becomes inefficient as the dimensionality of the feature space increases. Past indexing methods have used hash tables for hypothesis recovery, but only in low-dimensional situations. In this paper we show that a new variant of the k-d tree search algorithm makes indexing in higher-dimensional spaces practical. This Best Bin First, or BBF search is an approximate algorithm which finds the nearest neighbour for a large fraction of the queries, and a very close neighbour in the remaining cases. The technique has been integrated into a fully developed recognition system, which is able to detect complex objects in real, cluttered scenes in just a few seconds.

We describe an implemented model-based telerobotic system designed to investigate assembly and other tasks involving contact and manipulation of known objects. Key features of our system include ease of maintaining a world model at the operator site and a task-centric operator interface. Our system incorporates gray-scale model-based vision to assist in building and maintaining the local model. The local model is used to provide a task-centric operator interface, emphasizing the natural and direct manipulation of objects, with the robot's presence indicated in a more abstract fashion. The operator interface is designed to work with widely available and inexpensive desktop computers with low DOF input devices (such as a mouse). We also describe experimental results to date, which include performing assembly-like tasks over the Internet.

This paper introduces the idea of temporally extending the results of a stereo algorithm in order to improve the algorithm's performance. This approach anticipates the changes between two consecutive depth maps resulting from the motion of the cameras. Uncertainties in motion are accounted for by computation of an ambiguity area and a resulting disparity range for each pixel. The computation is used to verify and refine the anticipated values, rather than calculate them without prior knowledge. The paper compares the performance of the algorithm under different constraints on motion. Speedups of up to 400% are achieved without significant errors.

An algorithm is described which rapidly verifies the potential rigidity of three-dimensional point correspondences from a pair of two-dimensional views under perspective projection. The output of the algorithm is a simple yes or no answer to the question "Could these corresponding points from two views be the projection of a rigid configuration?" Potential applications include 3D object recognition from a single previous view and correspondence matching for stereo or motion over widely separated views. The rigidity checking problem is different from the structure-from-motion problem because it is often the case that two views cannot provide an accurate structure-from-motion estimate due to ambiguity and ill conditioning, whereas it is still possible to give an accurate yes/no answer to the rigidity question. Rigidity checking verifies point correspondences using 3D recovery equations as a matching condition. The proposed algorithm improves upon other methods that fall under this approach because it works with as few as six corresponding points under full perspective projection, handles correspondences from widely separated views, makes full use of the disparity of the correspondences, and is integrated with a linear algorithm for 3D recovery due to Kontsevich (1993). Results are given for experiments with synthetic and real image data. A complete implementation of this algorithm is being made publicly available.

An algorithm is described which rapidly verifies the potential rigidity of three dimensional point correspondences from a pair of three dimensional views under perspective projection. The output of the algorithm is a simple yes or no answer to the question "Could these corresponding points from two views be the projection of a rigid configuration?" Potential applications include 3D object recognition from a single previous view and correspondence matching for stereo or motion over widely separated views. Rigidity checking verifies point correspondences by using 3D recovery equations as a matching condition. The proposed algorithm improves upon other methods that fall under this approach because it works with as few as six corresponding points under full perspective projection, handles correspondences from widely separated views, makes full use of the disparity of the correspondences, and is integrated with a linear algorithm for 3D recovery due to Kontsevich (1991). Results are given for experiments with synthetic and real image data. A complete implementation of this algorithm is being made publicly available.>

Nearest-neighbor interpolation algorithms have many useful properties for applications to learning, but they often exhibit poor generalization. In this paper, it is shown that much better generalization can be obtained by using a variable interpolation kernel in combination with conjugate gradient optimization of the similarity metric and kernel size. The resulting method is called variable-kernel similarity metric (VSM) learning. It has been tested on several standard classification data sets, and on these problems it shows better generalization than backpropagation and most other learning methods. The number of parameters that must be determined through optimization are orders of magnitude less than for backpropagation or radial basis function (RBF) networks, which may indicate that the method better captures the essential degrees of variation in learning. Other features of VSM learning are discussed that make it relevant to models for biological learning in the brain.

Indexing is an efficient method of recovering match hypotheses in model-based object recognition. Unlike other methods, which search for viewpoint-invariant shape descriptors to use as indices, we use a learning method to model the smooth variation in appearance of local feature sets (LFS). Indexing from LFS effectively deals with the problems of occlusion and missing features. The indexing functions generated by the learning method are probability distributions describing the possible interpretations of each index value. During recognition, this information can be used to select the least ambiguous features for matching. A verification stage follows so that the final reliability and accuracy of the match is greater than that from indexing alone. This approach has the potential to work with a wide range of image features and model types.>

Vista is a software environment supporting the modular implementation and execution of computer vision algorithms. Because it is extensible, portable, and freely available, Vista is an appropriate medium for the exchange of standard implementations of algorithms. This paper, an overview of Vista, describes its file format, its data abstraction, its conventions for UNIX filter programs and library routines, and its user interface toolkit. Unlike systems that are designed principally to support image processing, Vista provides for the easy creation and use of arbitrary data types, such as are needed for many areas of computer vision research.>

To recognize an object in an image an internal model is required to indicate how that object may appear. The authors show how to learn such a model from a series of training images depicting a class of objects, producing a model that represents a probability distribution over the variation in object appearance. Features identified in an image through perceptual organization are represented by a graph whose nodes include feature labels and numeric measurements. A learning procedure generalizes multiple image graphs to form a model graph in which the numeric measurements are characterized by probability distributions. A matching procedure, using a similarity metric based on a non-parametric probability density estimator, compares model and image graphs to identify an instance of a modeled object in an image. Experimental results are presented from a system constructed to test this approach. The system learns to recognize partially occluded 2-D objects in 2-D images using shape cues. It can recognize objects as similar in general appearance while distinguishing them by their detailed features. >

Model-based recognition and motion tracking depend upon the ability to solve for projection and model parameters that will best fit a 3-D model to matching 2-D image features. The author extends current methods of parameter solving to handle objects with arbitrary curved surfaces and with any number of internal parameters representing articulation, variable dimensions, or surface deformations. Numerical stabilization methods are developed that take account of inherent inaccuracies in the image measurements and allow useful solutions to be determined even when there are fewer matches than unknown parameters. The Levenberg-Marquardt method is used to always ensure convergence of the solution. These techniques allow model-based vision to be used for a much wider class of problems than was possible with previous methods. Their application is demonstrated for tracking the motion of curved, parameterized objects.>

The problem of multiclass pattern classification using adaptive layered networks is addressed. A special class of networks, i.e., feed-forward networks with a linear final layer, that perform generalized linear discriminant analysis is discussed, This class is sufficiently generic to encompass the behavior of arbitrary feed-forward nonlinear networks. Training the network consists of a least-square approach which combines a generalized inverse computation to solve for the final layer weights, together with a nonlinear optimization scheme to solve for parameters of the nonlinearities. A general analytic form for the feature extraction criterion is derived, and it is interpreted for specific forms of target coding and error weighting. An important aspect of the approach is to exhibit how a priori information regarding nonuniform class membership, uneven distribution between train and test sets, and misclassification costs may be exploited in a regularized manner in the training phase of networks.>

A system is described for real-time motion tracking of 3-D objects, including those with variable internal parameters. Unlike most previous work on model-based motion tracking, this system provides for the integrated treatment of matching and measurement errors during motion tracking. These two sources of error have very different distributions and must be handled by separate computational mechanisms. These errors can be treated in an integrated way by using the computation of variance in predicted feature measurements to greatly decrease the amount of search required for subsequent matching. In return, the search process is used to eliminate the need to treat outliers during the analysis of measurement errors. The resulting system can robustly track models with many degrees of freedom while running on relatively inexpensive hardware. These same techniques can be used to speed verification during model-based recognition, and they point the way to merging of systems for recognition and motion tracking.>

The authors present a method for texture segmentation that does not assume any prior knowledge about either the type of textures or the number of textured regions present in the image. Local orientation and spatial frequencies are used as the key parameters for classifying texture. The information is obtained by creating a local multifrequency multiorientation channel decomposition of the image, with the width of each frequency band constant on a logarithmic scale. This decomposition is implemented by applying a set of Gabor-like functions that were modified to have a decreased frequency selectivity when the filter's center frequency increases. The set of filter outputs is then used to create robust texture descriptors. The segmentation algorithm uses the similarity of the descriptors to determine the existence of texture regions and to outline their border rather than concentrating on segregating the textures. The method has been applied to image containing natural textures, resulting in a good segmentation of the texture regions.>

A method for unsupervised segmentation of textured regions is presented. Rather than identifying boundaries between texture patches, this method detects regions of uniform texture in real images. No a priori knowledge regarding the image, the texture types, or their scales is assumed. The images may contain an unknown number of texture regions including regions with no texture at all. The method is most useful for identifying textures in which sharp intensity changes constitute the most distinctly perceived characteristic. Texture features are computed over image subregions from the distributions of local orientations and the separations of zero-crossing points. The segmentation algorithm establishes the existence of texture regions by finding neighboring subregions that share one or more nonaccidental properties of the computed features, e.g., a distribution of local orientations with a significant peak. Regions' accurate boundaries are identified by extending the seed regions using a region-growing technique that is applied to the computed texture features. The growth is directed by a region-specific self-adaptive thresholding scheme. No assumption is made regarding the texture scale, and the feature analysis is performed across multiple neighborhood (window) sizes. As a result different textures in the image may be segmented using different window sizes.>

A computer vision system has been implemented that can recognize three-dimensional objects from unknown viewpoints in single gray-scale images. Unlike most other approaches, the recognition is accomplished without any attempt to reconstruct depth information bottom-up from the visual input. Instead, three other mechanisms are used that can bridge the gap between the two-dimensional image and knowledge of three-dimensional objects. First, a process of perceptual organization is used to form groupings and structures in the image that are likely to be invariant over a wide range of viewpoints. Second, a probabilistic ranking method is used to reduce the size of the search space during model-based matching. Finally, a process of spatial correspondence brings the projections of three-dimensional models into direct correspondence with the image by solving for unknown viewpoint and model parameters. A high level of robustness in the presence of occlusion and missing data can be achieved through full application of a viewpoint consistency constraint. It is argued that similar mechanisms and constraints form the basis for recognition in human vision.

The SYNVIEW system implements cooperative structuring of information through an explicit representation for debate between the users of the system and through a voting mechanism for resolving disputes. This paper reviews the original design of the system and describes modifications that are necessary for near-term applications. In particular, we examine ways to interface to existing information in the form of traditional documents, and we describe simplifications to the debate representation that reduce the amount of effort required to enter information. A proposal is given for a demonstration of a cooperative information system that would be used by a group of scientists working in a particular area of research. Cooperative information systems have an excellent long-term potential, but there are many issues that must be resolved to encourage their more immediate use.

A class of inferences is described which allows the recovery of three-dimensional structures from the two-dimensional curves in an image. Unlike most previous methods, these inferences do not require restrictive assumptions or prior knowledge regarding the scene. They are based on the assumption that the camera viewpoint and the positions of the illumination sources are independent of the objects in the scene. From these independence assumptions, it can be shown that many potential interpretations of image curves are highly improbable. By eliminating these improbable interpretations it is possible to segment the image into sets of related image features and derive many three-space relations.