Scene content understanding facilitates a large number of applications, ranging from content-based image retrieval to other multimedia applications. Material detection refers to the problem of identifying key semantic material types (such as sky, grass, foliage, water, and snow in images). In this paper, we present a holistic approach to determining scene content, based on a set of individual material detection algorithms, as well as probabilistic spatial context models. A major limitation of individual material detectors is the significant number of misclassifications that occur because of the similarities in color and texture characteristics of various material types. We have developed a spatial context-aware material detection system that reduces misclassification by constraining the beliefs to conform to the probabilistic spatial context models. Experimental results show that the accuracy of materials detection is improved by 13% using the spatial context models over the individual material detectors themselves.

Natural object detection in outdoor scenes, i.e., identifying key object types such as sky, grass, foliage, water, and snow, can facilitate content-based applications, ranging from image enhancement to other multimedia applications. A major limitation of individual object detectors is the significant number of misclassifications that occur because of the similarities in color and texture characteristics of various object types and lack of context information. We have developed a spatial context-aware object-detection system that first combines the output of individual object detectors to produce a composite belief vector for the objects potentially present in an image. Spatial context constraints, in the form of probability density functions obtained by learning, are subsequently used to reduce misclassification by constraining the beliefs to conform to the spatial context models. Experimental results show that the spatial context models improve the accuracy of natural object detection by 13% over the individual object detectors themselves.

There is no such thing as a disembodied mind. We posit that cognitive development can only occur through interaction with the physical world. To this end, we are developing a robotic platform for the purpose of studying cognition. We suggest that the central component of cognition is a memory which is primarily associative, one where learning occurs as the correlation of events from diverse inputs. We also posit that human-like cognition requires a well-integrated sensory-motor system, to provide these diverse inputs. As implemented in our robot, this system includes binaural hearing, stereo vision, tactile sense, and basic proprioceptive control. On top of these abilities, we are implementing and studying various models of processing, learning and decision making. Our goal is to produce a robot that will learn to carry out simple tasks in response to natural language requests. The robot's understanding of language will be learned concurrently with its other cognitive abilities. We have already developed a robust system and conducted a number or experiments on the way to this goal, some details of which appear in this paper. This is a first progress report of what we believe will be a long term project with significant implications.

This paper presents a novel feature-matching based approach for rigid object tracking. The proposed method models the tracking problem as discovering the affine transforms of object images between frames according to the extracted feature correspondences. False feature matches (outliers) are automatically detected and removed with a new SVM regression technique, where outliers are iteratively identified as support vectors with the gradually decreased insensitive margin /spl epsi/. This method, in addition to object tracking, can also be used for general feature-based epipolar constraint estimation, in which it can quickly detect outliers even if they make up, in theory, over 50% of the whole data. We have applied the proposed method to track real objects under cluttering backgrounds with very encouraging results.

This paper presents a novel landmark-based shape deformation method. This method effectively solves two problems inherent in landmark-based shape deformation: (a) identification of landmark points from a given input image, and (b) regularized deformation the shape of an an object defined in a template. The second problem is solved using a new constrained support vector machine (SVM) regression technique, in which a thin-plate kernel is utilized to provide non-rigid shape deformations. This method offers several advantages over existing landmark-based methods. First, it has a unique capability to detect and use multiple candidate landmark points in an input image to improve landmark detection. Second, it can handle the case of missing landmarks, which often arises in dealing with occluded images. We have applied the proposed method to extract the scalp contours from brain cryosection images with very encouraging results.

In this paper, we present a novel statistical approach, called the weighted voting method, for automatic news video story categorization based on the closed captioned text. News video is initially segmented into stories using the demarcations in the closed captioned text, then a set of

An edge orientation-based algorithm for multi-view object recognition is presented. The distribution of edge point orientations, combined with the normalized second moments, is taken as a feature vector to describe and index each object instance. For each unknown test object, a set of likelihood weights for all the possible candidate objects is obtained by computing the Euclidean distances between the unknown feature set and all the available template feature vectors. A convincing coefficient is introduced to evaluate the confidence of the best match. New views (photo shots) will be automatically taken if the best match is thought to be insufficiently convincing. In experiments, our algorithm has achieved an average of 91.5% correct recognition rate under the 5-view scheme for 320 testing images taken from eight natural objects.