In this paper we present a video surveillance system for automated border and checkpoint analysis. The described system employs automated feature extraction and tracking to ascertain vehicle size, speed, and response to an interrogating vibration for vehicle bounce signature analysis. To increase the overall robustness of the surveillance system, we introduce a novel approach to invalid feature filtering. In particular, we use a hidden Markov model trained to simultaneously recognize specific coarse motion trajectories and tracking failures. The proposed recognition and filtering scheme effectively identifies erroneously tracked features and removes them prior to any subsequent motion analysis tasks. The result is a significant increase in classification and recognition accuracy. We demonstrate the efficacy of the suggested technique on a variety of video surveillance sequences.

We propose a new and effective method of predicting tracking failures and apply it to the robust analysis of gait and human motion. We define a tracking failure as an event and describe its temporal characteristics using a hidden Markov model (HMM). We represent the human body using a three-dimensional, multicomponent structural model, where each component is designed to independently allow the extraction of certain gait variables. To enable a fault-tolerant tracking and feature extraction system, we introduce a single HMM for each element of the structural model, trained on previous examples of tracking failures. The algorithm derives vector observations for each Markov model using the time-varying noise covariance matrices of the structural model parameters. When transformed with a logarithmic function, the conditional output probability of each HMM is shown to have a causal relationship with imminent tracking failures. We demonstrate the effectiveness of the proposed approach on a variety of multiview video sequences of complex human motion.

This research introduces a new and effective method of predicting motion tracking failures and demonstrates its application towards the analysis of gait and human motion. We define a tracking failure as an event and describe its temporal characteristics using a hidden Markov model (HMM). This stochastic model is trained using previous examples of tracking failures and is applied to the Kalman-based tracking of a parametric, structural model of the human body. With an observation sequence derived from the noise covariance matrices of the structural model parameters, we show a causal relationship between the conditional output probability of the HMM and imminent tracking failures. Results are demonstrated on a variety of multi-view sequences of complex human motion.

This paper presents a new method of detecting and predicting motion tracking failures with applications in human motion and gait analysis. We define a tracking failure as an event and describe its temporal characteristics using a hidden Markov model (HMM). This stochastic model is trained using previous examples of tracking failures. We derive vector observations for the HMM using the noise covariance matrices characterizing a tracked, 3D structural model of the human body. We show a causal relationship between the conditional output probability of the HMM, as transformed using a logarithmic mapping function, and impending tracking failures. Results are illustrated on several multi-view sequences of complex human motion.

This research introduces a method of predicting tracking failures and applies it to the robust analysis of human gait. The body is represented using a multicomponent structural model. For each component, the proposed approach extracts features from tracked noise covariance matrices and uses them to construct an observation sequence for a hidden Markov model (HMM) trained to detect tracking failures. When transformed with a logarithmic function, the conditional output probability of the HMM is shown to have a causal relationship with imminent tracking failures. This fusion of multiple structural models with a reliable means of failure prediction facilitates the successful tracking and extraction of gait variables. Results are demonstrated on numerous video sequences.

We introduce a new approach to motion-based recognition that combines the temporally descriptive abilities of a hidden Markov model (HMM) with the inferential power of a Bayesian belief network. We define activities using a collection of multiple Markov models, each associated with a unique set of body model parameters or gait variables. A single Bayesian network integrates the models by operating on virtual evidence derived from the HMM conditional output probabilities. We introduce both fundamental and auxiliary models for characterizing events and tracking failures, respectively. We demonstrate the system using multi-view video sequences corrupted by occlusion, noise, and entirely missing observations.

This research presents a new model-based approach toward the three-dimensional (3-D) tracking and extraction of gait and human motion. We suggest the use of a hierarchical, structural model of the human body that introduces the concept of soft kinematic constraints. These constraints take the form of a priori, stochastic distributions learned from previous configurations of the body exhibited during specific activities; they are used to supplement an existing motion model limited by hard kinematic constraints. We use time-varying parameters of the structural model to measure gait velocity, stance width, stride length, stance times, and other gait variables with multiple degrees of accuracy and robustness. To characterize tracking performance, we also introduce a novel geometric model of expected tracking failures. We demonstrate and quantify the performance of the suggested models using multi-view, video sequences of human movement captured in a complex home environment.

We describe a thorough quantitative analysis of a novel, kinematic model of the human body used for the tracking and extraction of gait variables. The model includes multiple levels of structural complexity coupled with absolute and probabilistic kinematic constraints; while the relevant variables include velocity, stride length, stance width, and stance times. The analysis is based on the accuracy of extracted gait variables for various permutations of the proposed kinematic body model. Using patterns of complex human motion collected from a multi-camera video monitoring system, we demonstrate the precise costs and benefits associated with various model components and kinematic constraints.

We present a novel method for the spatial integration of multiple views as a means for tracking point features in the presence of occlusion. The proposed technique employs a dynamic, multi-dimensional Bayesian network to combine information from multiple views. To achieve real-time performance, the system is implemented in a distributed fashion; the two-dimensional tracking for each view, as well as the spatial integration, occurs on a dedicated processor. We demonstrate the efficacy of the proposed spatial integration on the multi-view tracking of a person in a home environment. Our results show a considerable increase in the accuracy of tracking features throughout periods of occlusion.

We propose a distributed, real-time computing platform for tracking multiple interacting persons in motion. To combat the negative effects of occlusion and articulated motion we use a multiview implementation, where each view is first independently processed on a dedicated processor. This monocular processing uses a predictor-corrector filter to weigh reprojections of three-dimensional (3-D) position estimates, obtained by the central processor, against observations of measurable image motion. The corrected state vectors from each view provide input observations to a Bayesian belief network, in the central processor, with a dynamic, multidimensional topology that varies as a function of scene content and feature confidence. The Bayesian net fuses independent observations from multiple cameras by iteratively resolving independency relationships and confidence levels within the graph, thereby producing the most likely vector of 3-D state estimates given the available data. To maintain temporal continuity, we follow the network with a layer of Kalman filtering that updates the 3-D state estimates. We demonstrate the efficacy of the proposed system using a multiview sequence of several people in motion. Our experiments suggest that, when compared with data fusion based on averaging, the proposed technique yields a noticeable improvement in tracking accuracy.

Traditional approaches to the estimation of motion in video sequences have relied on the appropriate selection of various algorithm parameters. This dependence becomes a prohibitive drawback in applications where automation is desirable or necessary or in sequences where a single set of parameters can not achieve sufficiently accurate results. We investigate a number of techniques for locally adapting both the spatio-temporal filters and the hierarchical structure used in the estimation of optical flow. The surviving technique utilizes projected active contours and gradient-based Chamfer distance images to adapt the filters and a temporally-based Kolmogorov-Smirnov metric to locally adapt the hierarchical structure. The advantages of using these adaptive variations are demonstrated on articulated and self-occluding motion.