This paper describes a system for multimedia event detection and recounting. The goal is to detect a high level event class in unconstrained web videos and generate event oriented summarization for display to users. For this purpose, we detect informative segments and collect observations for them, leading to our ISOMER system. We combine a large collection of both low level and semantic level visual and audio features for event detection. For event recounting, we propose a novel approach to identify event oriented discriminative video segments and their descriptions with a linear SVM event classifier. User friendly concepts including objects, actions, scenes, speech and optical character recognition are used in generating descriptions. We also develop several mapping and filtering strategies to cope with noisy concept detectors. Our system performed competitively in the TRECVID 2013 Multimedia Event Detection task with near 100,000 videos and was the highest performer in TRECVID 2013 Multimedia Event Recounting task.

We consider the problem of autonomous navigation in unstructured outdoor terrains using vision sensors. The goal is for a robot to come into a new environment, map it and move to a given goal at modest speeds (1 m/sec). The biggest challenges are in building good maps and keeping the robot well localized as it advances towards the goal. In this paper, we concentrate on showing how it is possible to build a consistent, globally correct map in real time, using efficient precise stereo algorithms for map making and visual odometry for localization. While we have made advances in both localization and mapping using stereo vision, it is the integration of the techniques that is the biggest contribution of the research. The validity of our approach is tested in blind experiments, where we submit our code to an independent testing group that runs and validates it on an outdoor robot

Tracking multiple people in a dynamic environment is important in many applications. Recent research in this area has focused either on geometric analysis or appearance models. In this paper we distinguish four types of tracking problems, and then describe an approach for combining geometric analysis and appearance-based tracking to "hold on to" people in three of these situations.

Stereo sequences promise to be a powerful method for segmenting images for applications such as tracking human figures. We present a method of statistical background modeling for stereo sequences that improves the reliability and sensitivity of segmentation in the presence of object clutter. The dynamic version of the method, called gated background adaptation, can reliably learn background statistics in the presence of corrupting foreground motion. The method has been used with a simple head discriminator to detect and track people using a stereo head mounted on a pan/tilt platform. It runs at video rates using standard PC hardware.

A representation paradigm for instantiating and refining multiple, concurrent descriptions of an object from a sequence of imagery is presented. It is designed for the perception system of an autonomous robot that needs to describe many types of objects, initially detects objects at a distance and gradually acquires higher resolution data, and continuously collects sensory input. Since the data change significantly over time, the paradigm supports the evolution of descriptions, progressing from crude 2-D 'blob' descriptions to complete semantic models. To control this accumulation of new descriptions, the authors introduce the idea of representation space, a lattice of representations that specifies the order in which they should be considered for describing an object. A system, TraX, that constructs and refines models of outdoor objects detected in sequences of range data is described.>

An approach to representing objects viewed over long periods of time and with changing resolutions is presented. The basic strategy is to apply different representations as they become appropriate. As a result, the model of an object typically goes through a sequence of representation as new data are gathered and processed. One of these sequences might start with a crude blob description of an initially detected object, include a detailed structural model derived from a set of high-resolution images, and end with a semantic label based on the object's description and the sensor system's task. This evolution in representation is guided by a structure referred to as representation space: a lattice or representation that is traversed as new information about an object becomes available. One of the representations is associated with an object only after it has been judged to be valid. One approach for evaluating the validity of an object's description is described, based on the temporal stability of the description. These ideas are illustrated with results from a system which constructs and refines models of outdoor objects detected in sequences of range data.>

A technique for building reliable scene descriptions by evaluating the temporal stability of detected objects is presented. The approach is designed to avoid mistakes and to increase the competence of the sensing system by tracking objects from image to image and evaluating the stability of their descriptions over time. Since the information available about an object can change significantly over time, the authors introduce the idea of a representation space, which is a lattice of representations progressing from crude blob descriptions to complete semantic models, such as bush, rock, and tree. One of these representations is associated with an object only after the object has been described multiple times in the representation and the parameters of the representation are stable in a statistical sense enhanced by a set of explanations describing valid reasons for deviations. To illustrate the power of these ideas, the authors have implemented a system, called TraX, that constructs and refines models of outdoor objects detected in sequences of range data.>

The previous implementations of the authors' epipolar-plane image-analysis mapping technique demonstrated the feasibility and benefits of the approach, but were carried out for restricted camera geometries. The question of more general geometries made the technique's utility for autonomous navigation uncertain. The authors have developed a generalization of the analysis that: (1) enables varying view direction, including varying over time; (2) provides three-dimensional connectivity information for building coherent spatial descriptions of observed objects; and (3) operates sequentially, allowing initiation and refinement of scene feature estimates while the sensor is in motion. To implement this generalization it was necessary to develop an explicit description of the evolution of images over time. They achieved this by building a process that creates a set of two-dimensional manifolds defined at the zeros of a three-dimensional spatiotemporal Laplacian. These manifolds represent explicitly both the spatial and temporal structure of the temporally evolving imagery and are termed spatiotemporal surfaces.>

A technique for verifying an object hypothesis by comparing hypothesized edge patterns to detected edge patterns is presented. Positive evidence for a hypothesis is identified as those portions of the model edges that are found in the image, either as geometrical features (such as straight lines or circular arcs} or as linked edges of arbitrary shape. Occluded portions of a model edge are detected by analyzing junctions of image and model edge chains. Negative evidence is identified as those portions of the model edges that are not found and are not occluded. We describe the types of junctions that can occur between image and model edges and discuss a technique for identifying occluded regions based on these junctions. Preliminary results of these techniques are also presented.

In this paper we offer a critical evaluation of the partitioning (perceptual organization) problem, noting the extent to which it has distinct formulations and parameterizations. We show that most partitioning techniques can be characterized as variations of four distinct paradigms, and argue that any effective technique must satisfy two general principles. We give concrete substance to our general discussion by introducing new partitioning techniques for planar geometric curves, and present experimental results demonstrating their effectiveness.

A strategy for recognizing and locating three-dimensional objects in range data is presented. The strategy combines information derived from models of the objects and edges and surfaces detected in the data to efficiently match objects. Given a set of objects to be found, the set of object features are partitioned into subsets having similar intrinsic properties. An ordered tree of features to be considered is set up for each subset. These search trees are designed to maximize the use of the information as it is obtained and minimize the time required to recognize objects. A detailed example of this approach being used to recognize moderately complex castings in a jumble is presented.

A brief history of SRI industrial vision research and its application to part acquisition tasks is presented. The goal is to explore techniques to locate parts visually and to simplify as much as possible the proramming required for each new task. Part acquisition tasks that involve two-dimensional and three-dimensional uncertainties are discussed.