The success of a multimedia information system depends heavily on the way the data is represented. Although there are "natural" ways to represent numerical data, it is not clear what is a good way to represent multimedia data, such as images, video, or sound. We investigate various image representations where the quality of the representation is judged based on how well a system for searching through an image database can perform-although the same techniques and representations can be used for other types of object detection tasks or multimedia data analysis problems. The system is based on a machine learning method used to develop object detection models from example images that can subsequently be used for examples to detect-search-images of a particular object in an image database. As a base classifier for the detection task, we use support vector machines (SVM), a kernel based learning method. Within the framework of kernel classifiers, we investigate new image representations/kernels derived from probabilistic models of the class of images considered and present a new feature selection method which can be used to reduce the dimensionality of the image representation without significant losses in terms of the performance of the detection-search-system.

We present a general example-based framework for detecting objects in static images by components. The technique is demonstrated by developing a system that locates people in cluttered scenes. The system is structured with four distinct example-based detectors that are trained to separately find the four components of the human body: the head, legs, left arm, and right arm. After ensuring that these components are present in the proper geometric configuration, a second example-based classifier combines the results of the component detectors to classify a pattern as either a "person" or a "nonperson." We call this type of hierarchical architecture, in which learning occurs at multiple stages, an adaptive combination of classifiers (ACC). We present results that show that this system performs significantly better than a similar full-body person detector. This suggests that the improvement in performance is due to the component-based approach and the ACC data classification architecture. The algorithm is also more robust than the full-body person detection method in that it is capable of locating partially occluded views of people and people whose body parts have little contrast with the background.

This paper presents a new paradigm for signal reconstruction and superresolution, correlation kernel analysis (CKA), that is based on the selection of a sparse set of bases from a large dictionary of class-specific basis functions. The basis functions that we use are the correlation functions of the class of signals we are analyzing. To choose the appropriate features from this large dictionary, we use support vector machine (SVM) regression and compare this to traditional principal component analysis (PCA) for the task of signal reconstruction. The testbed we use in this paper is a set of images of pedestrians. Based on the results presented here, we conclude that, when used with a sparse representation technique, the correlation function is an effective kernel for image reconstruction.

Current systems for object detection in video sequences rely on explicit dynamical models like Kalman filters or hidden Markov models. There is significant overhead needed in the development of such systems as well as the a priori assumption that the object dynamics can be described with such a dynamical model. This paper describes a new pattern classification technique for object detection in video sequences that uses a rich, overcomplete dictionary of wavelet features to describe an object class. Unlike previous work where a small subset of features was selected from the dictionary, this system does no feature selection and learns the model in the full 1,326 dimensional feature space. Comparisons using different sized sets of several types of features are given. We extend this representation into the time domain without assuming any explicit model of dynamics. This data driven approach produces a model of the physical structure and short-time dynamical characteristics of people from a training set of examples; no assumptions are made about the motion of people, just that short sequences characterize their dynamics sufficiently for the purposes of detection. One of the main benefits of this approach is that transient false positives are reduced. This technique compares favorably with the static detection approach and could be applied to other object classes. We also present a real-time version of one of our static people detection systems.

Robust, fast object detection systems are critical to the success of next-generation automotive vision systems. An important criteria is that the detection system be easily configurable to a new domain or environment. In this paper, we present work on a general object detection system that can be trained to detect different types of objects; we focus on the task of pedestrian detection. This paradigm of learning from examples allows us to avoid the need for a hand-crafted solution. Unlike many pedestrian detection systems, the core technique does not rely on motion information and makes no assumptions on the scene structured or the number of objects present. We discuss an extension to the system that takes advantage of dynamical information when processing video sequences to enhance accuracy. We also describe a real, real-time version of the system that has been integrated into a DaimlerChrysler test vehicle.

The paper presents a method for analyzing coupled time series using Markov models in a domain where the state space is immense. To make the parameter estimation tractable, the large state space is represented as the Cartesian product of smaller state spaces, a paradigm known as factorial Markov models. The transition matrix for this model is represented as a mixture of the transition matrices of the underlying dynamical processes. This formulation is know as mixed memory Markov models. Using this framework, the author analyzes the daily exchange rates for five currencies-British pound, Canadian dollar, Deutschmark, Japanese yen, and Swiss franc-as measured against the US dollar.

This paper presents a general trainable framework for object detection in static images of cluttered scenes. The detection technique we develop is based on a wavelet representation of an object class derived from a statistical analysis of the class instances. By learning an object class in terms of a subset of an overcomplete dictionary of wavelet basis functions, we derive a compact representation of an object class which is used as an input to a support vector machine classifier. This representation overcomes both the problem of in-class variability and provides a low false detection rate in unconstrained environments. We demonstrate the capabilities of the technique in two domains whose inherent information content differs significantly. The first system is face detection and the second is the domain of people which, in contrast to faces, vary greatly in color, texture, and patterns. Unlike previous approaches, this system learns from examples and does not rely on any a priori (hand-crafted) models or motion-based segmentation. The paper also presents a motion-based extension to enhance the performance of the detection algorithm over video sequences. The results presented here suggest that this architecture may well be quite general.

There has been a surge in interest in the analysis and prediction of high frequency time series in recent years. We consider the problem of predicting the direction of change in tick data of the U.S. dollar/Swiss Franc exchange rate. To accomplish this, we show that a Markov model can find regularities in certain local regions of the data and can be used to predict the direction of the next tick. Predictability seems to decrease in more recent years. With transaction costs, the model is unlikely to be profitable.

This paper presents a trainable object detection architecture that is applied to detecting people in static images of cluttered scenes. This problem poses several challenges. People are highly non-rigid objects with a high degree of variability in size, shape, color, and texture. Unlike previous approaches, this system learns from examples and does not rely on any a priori (hand-crafted) models or on motion. The detection technique is based on the novel idea of the wavelet template that defines the shape of an object in terms of a subset of the wavelet coefficients of the image. It is invariant to changes in color and texture and can be used to robustly define a rich and complex class of objects such as people. We show how the invariant properties and computational efficiency of the wavelet template make it an effective tool for object detection.