We propose a linear dimensionality reduction method, Composite Discriminant Factor (CDF) analysis, which searches for a discriminative but compact feature subspace that can be used as input to classifiers that suffer from problems such as multi-collinearity or the curse of dimensionality. The subspace selected by CDF maximizes the performance of the entire classification pipeline, and is chosen from a set of candidate subspaces that are each discriminative. Our method is based on Partial Least Squares (PLS) analysis, and can be viewed as a generalization of the PLS1 algorithm, designed to increase discrimination in classification tasks. We demonstrate our approach on the UCF50 action recognition dataset, two object detection datasets (INRIA pedestrians and vehicles from aerial imagery), and machine learning datasets from the UCI Machine Learning repository. Experimental results show that the proposed approach improves significantly in terms of accuracy over linear SVM, and also over PLS in terms of compactness and efficiency, while maintaining or improving accuracy.

We describe a framework that leverages mixed probabilistic and deterministic networks and their AND/OR search space to efficiently find and track the hands and feet of multiple interacting humans in 2D from a single camera view. Our framework detects and tracks multiple people's heads, hands, and feet through partial or full occlusion; requires few constraints (does not require multiple views, high image resolution, knowledge of performed activities, or large training sets); and makes use of constraints and AND/OR Branch-and-Bound with lazy evaluation and carefully computed bounds to efficiently solve the complex network that results from the consideration of interperson occlusion. Our main contributions are: 1) a multiperson part-based formulation that emphasizes extremities and allows for the globally optimal solution to be obtained in each frame, and 2) an efficient and exact optimization scheme that relies on AND/OR Branch-and-Bound, lazy factor evaluation, and factor cost sensitive bound computation. We demonstrate our approach on three datasets: the public single person HumanEva dataset, outdoor sequences where multiple people interact in a group meeting scenario, and outdoor one-on-one basketball videos. The first dataset demonstrates that our framework achieves state-of-the-art performance in the single person setting, while the last two demonstrate robustness in the presence of partial and full occlusion and fast nontrivial motion.

Detecting vehicles in aerial images has a wide range of applications, from urban planning to visual surveillance. We describe a vehicle detector that improves upon previous approaches by incorporating a very large and rich set of image descriptors. A new feature set called Color Probability Maps is used to capture the color statistics of vehicles and their surroundings, along with the Histograms of Oriented Gradients feature and a simple yet powerful image descriptor that captures the structural characteristics of objects named Pairs of Pixels. The combination of these features leads to an extremely high-dimensional feature set (approximately 70,000 elements). Partial Least Squares is first used to project the data onto a much lower dimensional sub-space. Then, a powerful feature selection analysis is employed to improve the performance while vastly reducing the number of features that must be calculated. We compare our system to previous approaches on two challenging data sets and show superior performance.

Significant research has been devoted to detecting people in images and videos. In this paper we describe a human detection method that augments widely used edge-based features with texture and color information, providing us with a much richer descriptor set. This augmentation results in an extremely high-dimensional feature space (more than 170,000 dimensions). In such high-dimensional spaces, classical machine learning algorithms such as SVMs are nearly intractable with respect to training. Furthermore, the number of training samples is much smaller than the dimensionality of the feature space, by at least an order of magnitude. Finally, the extraction of features from a densely sampled grid structure leads to a high degree of multicollinearity. To circumvent these data characteristics, we employ Partial Least Squares (PLS) analysis, an efficient dimensionality reduction technique, one which preserves significant discriminative information, to project the data onto a much lower dimensional subspace (20 dimensions, reduced from the original 170,000). Our human detection system, employing PLS analysis over the enriched descriptor set, is shown to outperform state-of-the-art techniques on three varied datasets including the popular INRIA pedestrian dataset, the low-resolution gray-scale DaimlerChrysler pedestrian dataset, and the ETHZ pedestrian dataset consisting of full-length videos of crowded scenes.

This paper describes the architecture of a visual surveillance system that combines real time computer vision algorithms with logic programming to represent and recognize activities involving interactions amongst people, packages and the environments through which they move. The low level computer vision algorithms log primitive events of interest as observed facts, while the higher level Prolog based reasoning engine uses these facts in conjunction with predefined rules to recognize various activities in the input video streams. The system is illustrated in action on a multi-camera surveillance scenario that includes both security and safety violations.

We present a new fast algorithm for background modeling and subtraction. Sample background values at each pixel are quantized into codebooks which represent a compressed form of background model for a long image sequence. This allows us to capture structural background variation due to periodic-like motion over a long period of time under limited memory. Our method can handle scenes containing moving backgrounds or illumination variations (shadows and highlights), and it achieves robust detection for compressed videos. We compared our method with other multimode modeling techniques.

Automatic understanding of events happening at a site is the ultimate goal for many visual surveillance systems. Higher level understanding of events requires that certain lower level computer vision tasks be performed. These may include detection of unusual motion, tracking targets, labeling body parts, and understanding the interactions between people. To achieve many of these tasks, it is necessary to build representations of the appearance of objects in the scene. This paper focuses on two issues related to this problem. First, we construct a statistical representation of the scene background that supports sensitive detection of moving objects in the scene, but is robust to clutter arising out of natural scene variations. Second, we build statistical representations of the foreground regions (moving objects) that support their tracking and support occlusion reasoning. The probability density functions (pdfs) associated with the background and foreground are likely to vary from image to image and will not in general have a known parametric form. We accordingly utilize general nonparametric kernel density estimation techniques for building these statistical representations of the background and the foreground. These techniques estimate the pdf directly from the data without any assumptions about the underlying distributions. Example results from applications are presented.

We describe fast background scene modeling and maintenance techniques for real time visual surveillance system for tracking people in an outdoor environment. It operates on monocular gray scale video imagery or on video imagery from an infrared camera. The system learns and models background scene statistically to detect foreground objects, even when the background is not completely stationary (e.g. motion of tree branches) using shape and motion cues. Also, a background maintenance model is proposed for preventing false positives, such as, illumination changes (the sun being blocked by clouds causing changes in brightness), or false negative, such as, physical changes (person detection while he is getting out of the parked car). Experimental results demonstrate robustness and real-time performance of the algorithm.

We describe an appearance-based human body model for tracking multiple people when they are interaction with each others causing significant occlusion amongst them, or when they re-enter the scene. The proposed model allows real time surveillance systems understand "who is who" after multiple people interactions, partial or total occlusions, or when a person "reappeal" in the scene. It combines the grayscale textural appearance and expected shape information together in a 2D dynamic template. Experimental results demonstrates the robustness and real time performance of the proposed model.

W/sup 4/ is a real time visual surveillance system for detecting and tracking multiple people and monitoring their activities in an outdoor environment. It operates on monocular gray-scale video imagery, or on video imagery from an infrared camera. W/sup 4/ employs a combination of shape analysis and tracking to locate people and their parts (head, hands, feet, torso) and to create models of people's appearance so that they can be tracked through interactions such as occlusions. It can determine whether a foreground region contains multiple people and can segment the region into its constituent people and track them. W/sup 4/ can also determine whether people are carrying objects, and can segment objects from their silhouettes, and construct appearance models for them so they can be identified in subsequent frames. W/sup 4/ can recognize events between people and objects, such as depositing an object, exchanging bags, or removing an object. It runs at 25 Hz for 320/spl times/240 resolution images on a 400 MHz dual-Pentium II PC.

We described a video-rate surveillance algorithm to detect and track people from a stationary camera, and to determine if they are carrying objects or moving unencumbered. The contribution of the paper is the shape analysis algorithm that both determines if a person is carrying an object and segments the object from the person so that it can be tracked, e.g., during an exchange of objects between two people. As the object is segmented an appearance model of the object is constructed. The method combines periodic motion estimation with static symmetry analysis of the silhouettes of a person in each frame of the sequence. Experimental results demonstrate robustness and real-time performance of the proposed algorithm.

Ghost is a real time system for estimating human body posture and detecting body parts in monochromatic imagery. It constructs a silhouette based body model to determine the location of the body parts while people are in generic postures. It combines hierarchical body pose estimation, a convex hull analysis of the silhouette, and a partial mapping from the body parts to the silhouette segments using a distance transform method that does not violate the topology of the human body. Experimental results demonstrate robustness and real-time performance of the proposed algorithm.

This paper presents an algorithm for automatic target recognition in, overhead LADAR range imagery using "appearance-based" models. The detection is accomplished using a two stage Hough transform where the target is modelled as a small set of boundary pixels with associated local properties. The first stage Hough transform keeps tracks of all the edge points matched by the model points. The second stage Hough transform then efficiently identifies the candidate space by incrementing the accumulator array along precomputed (for each model point) swaths. A robust estimator, the MAD, is then used to do the template matching directly on the second stage solutions which are already very accurate. Finally, we illustrate our algorithm on some real LADAR images and discuss the error rates.

This paper evaluates the performance both of some texture measures which have been successfully used in various applications and of some new promising approaches. For classification a method based on Kullback discrimination of sample and prototype distributions is used. The classification results for single features with one-dimensional feature value distributions and for pairs of complementary features with two-dimensional distributions are presented.

The authors introduce a general-purpose procedure for image segmentation which combines iterative image enhancement by a symmetric neighborhood filter (SNF) with an iterative, hierarchical connected component (HCC) analysis. Color vector versions of both SNF and HCC are developed, using two common kinds of metrics applied to color vector intensity differences-a simple Euclidean-type metric and a coordinate maximum-type metric. The segmentation procedures are illustrated with three-band color images of indoor and outdoor scenes.>

A procedure is presented to estimate the unconstrained three-dimensional location and orientation of an object with a known shape when it is visible in a single image. Using a generalized Hough transform, all six parameters of the object position are estimated from the distribution of values determined by matching triples of points on the object to possibly corresponding triples in the image. Most likely candidates for location are found, and then the remaining rotational parameters are evaluated. Two solutions are generally admitted to the distribution by every match of triangles. The number of possible matches is reduced by checking a simple geometric relation among triples. Even with partial occlusion, experiments indicate that the procedure is very reliably accurate, although optimization can further improve estimates of the parameters.>

An algorithm is presented which directly computes the 3-D coordinates of the vertices of a quadrangle relative to the camera frame. This method is extended to determine the camera's exterior orientation in a world coordinate system. A shape restoration procedure is designed to reduce the effects of noise and thereby improve the accuracy of the algorithm.

The theory behind selection of the ``uniform error'' threshold which equalizes the probability of misclassification in an image containing two classes is presented. It is shown how this threshold can be estimated using local operations. Some examples and possible extensions are considered.