In this paper, we propose a multi-level shadow identification scheme which is generally applicable without restrictions on the number of light sources, illumination conditions, surface orientations, and object sizes. In the first level, we use a background segmentation technique to identify foreground regions which include moving shadows. In the second step, pixel-based decisions are made by comparing the current frame with the background model to distinguish between shadows and actual foreground. In the third step, this result is improved using blob-level reasoning which works on geometric constraints of identified shadow and foreground blobs. Results on various indoor and outdoor sequences under different illumination conditions show the success of the proposed approach.

We present a novel technique for the detection of thrown objects and other free-flying bodies in video sequences. Our method runs in real-time and was designed to be used as a component in a deployed surveillance system. We detect regions of interesting motion that fit certain size, compactness and speed criteria, and use the expectation maximization algorithm to detect objects on parabolic trajectories over a short time window. The system was shown to successfully detect thrown objects of various sizes in a large test set of indoor and outdoor videos.

This paper presents a novel technique for camera tampering detection. It is implemented in real-time and was developed for use in surveillance and security applications. This method identifies camera tampering by detecting large differences between older frames of video and more recent frames. A buffer of incoming video frames is kept and three different measures of image dissimilarity are used to compare the frames. After normalization, a set of conditions is tested to decide if camera tampering has occurred. The effects of adjusting the internal parameters of the algorithm are examined. The performance of this method is shown to be extremely favorable in real-world settings.

This work presents a method for detecting abandoned objects in real-world conditions. The method presented here addresses the online and real time aspects of such systems, utilizes logic to differentiate between abandoned objects and stationary people, and is robust to temporary occlusion of potential abandoned objects. The capacity to not detect still people as abandoned objects is a major aspect that differentiates this work from others in the literature. Results are presented on 3 hours 36 minutes of footage over four videos representing both sparsely and densely populated real-world situations, also differentiating this work from others in the literature

We address the problem of human motion recognition in this paper. The goal of human motion recognition is to recognize the type of motion recorded in a video clip, which consists of a set of temporarily ordered frames. By defining a Mercer kernel between two video clips directly, we propose in this paper a recognition strategy that can incorporate both the information of each individual frame and the temporal ordering between frames. Combining the proposed kernel with the support vector machine, which is one of the most effective classification paradigms, the resulting recognition strategy exhibits excellent performance over real data sets

We address the problem of automatically learning the layout of a traffic intersection from trajectories of vehicles obtained by a vision tracking system. We present a similarity measure which is suitable for use with spectral clustering in problems that emphasize spatial distinctions between vehicle trajectories. The robustness of the method to small perturbations and its sensitivity to the choice of parameters are evaluated using real-world data

Monitoring traffic intersections in real-time and predicting possible collisions is an important first step towards building an early collision warning system. We present the general vision methods used in a system addressing this problem and describe the practical adaptations necessary to achieve real-time performance. A novel method for three-dimensional vehicle size estimation is presented. We also describe a method for target localization in real-world coordinates which allows for sequential incorporation of measurements from multiple cameras into a single target's state vector. Additionally, a fast implementation of a false-positive reduction method for the foreground pixel masks is developed. Finally, a low-overhead collision prediction algorithm using the time-as-axis paradigm is presented. The proposed system was able to perform in real-time on videos of quarter-VGA (320/spl times/240) resolution. The errors in target position and dimension estimates in a test video sequence are quantified.

Monitoring traffic intersections in real time and predicting possible collisions is an important first step towards building an early collision-warning system. We present a vision-based system addressing this problem and describe the practical adaptations necessary to achieve real-time performance. Innovative low-overhead collision-prediction algorithms (such as the one using the time-as-axis paradigm) are presented. The proposed system was able to perform successfully in real time on videos of quarter-video graphics array (VGA) (320 /spl times/ 240) resolution under various weather conditions. The errors in target position and dimension estimates in a test video sequence are quantified and several experimental results are presented.

This paper presents a vision-based method to automatically detect individuals loitering about inner-city bus stops. Using a stationary camera view of a bus stop, pedestrians are segmented and tracked throughout the scene. The system takes snapshots of individuals when a clean, nonobstructed view of a pedestrian is found. The snapshots are then used to classify the individual images into a database, using an appearance-based method. The features used to correlate individual images are based on short-term biometrics, which are changeable but stay valid for short periods of time; this system uses clothing color. A linear discriminant method is applied to the color information to enhance the differences and minimize similarities between the different individuals in the feature space. To determine if a given individual is loitering, time stamps collected with the snapshots in their corresponding database class can be used to judge how long an individual has been present. An experiment was performed using a 30-min video of a busy bus stop with six individuals loitering about it. Results show that the system successfully classifies images of all six individuals as loitering.

We propose a motion recognition strategy that represents a videoclip as a set of filtered images, each of which encodes a short period of motion history. Given a set of videoclips whose motion types are known, a filtered image classifier is built using support vector machines. In offline classification, the label of a test videoclip is obtained by applying majority voting over its filtered images. In online classification, the most probable type of action at an instance is determined by applying the majority voting over the most recent filtered images, which are within a sliding window. The effectiveness of this strategy was demonstrated on real datasets where the videoclips were recorded using a fixed camera whose optical axis is perpendicular to the person's trajectory. In offline recognition, the proposed strategy outperforms a principal component analysis based recognition algorithm. In online recognition, the proposed strategy cannot only classify motions correctly and identify the transition between different types of motions, but also identify the existence of an unknown motion type. This latter capability and the efficiency of the proposed strategy make it possible to create a real-time motion recognition system that can not only make classifications in real-time, but also learn new types of actions and recognize them in the future.

We introduce a vision-based system to monitor for suspicious human activities at a bus stop. The system currently examines for drug dealing activity. To accomplish this goal, the system must measure how long individuals loiter around the bus stop. To facilitate this, the system must track individuals from the video feed, identify them, and keep a record of how long they spend at the bus stop. The system is broken into three distinct portions: background subtraction, object tracking, and human recognition. The background subtraction and object tracking modules use off-the-shelf algorithms and are shown to work well following people as they walk around a bus stop. The human recognition module segments the image of an individual into three portions corresponding to the head, torso, and legs. Using the median color of each of these regions, two people can be quickly compared to see if they are the same person.

In this paper, we address the problem of recovering the intrinsic and extrinsic parameters of a camera or a group of cameras in a setting overlooking a traffic scene. Unlike many other settings, conventional camera calibration techniques are not applicable in this case. We present a method that uses certain geometric primitives commonly found in traffic scenes in order to recover calibration parameters. These primitives provide needed redundancy and are weighted depending on the significance of there corresponding image features. We show experimentally that these primitives are capable of achieving accurate results suitable for most traffic monitoring applications.

The paper deals with the problem of classification of human activities from video as one way of performing activity monitoring. Our approach uses motion features that are computed very efficiently and subsequently projected into a lower dimension space where matching is performed. Each action is represented as a manifold in this lower dimension space and matching is done by comparing these manifolds. To demonstrate the effectiveness of this approach, it was used on a large data set of similar actions, each performed by many different actors. Classification results are accurate and show that this approach can handle many challenges such as variations in performers' physical attributes, color of clothing, and style of motion. An important result is that the recovery of three-dimensional properties of a moving person, or even two-dimensional tracking of the person's limbs, is not a necessary step that must precede action recognition.

This paper addresses the problem of localization and map construction by a mobile robot in an indoor environment. Instead of trying to build high-fidelity geometric maps, we focus on constructing topological maps, as they are lees sensitive to poor odometry estimates and position errors. We propose a method for incrementally building topological maps for a robot, which uses a panoramic camera to obtain images at various locations along its path and uses the features it tracks in the images to update the topological map. The method is very general and does not require the environment to have uniquely distinctive features.

In this paper, we introduce a novel method for employing image-based rendering to extend the range of use of human motion recognition systems. We demonstrate the use of image-based rendering to generate additional training sets for view-dependent human motion recognition systems. Input views orthogonal to the direction of motion are created automatically to construct the proper view from a combination of non-orthogonal views taken from several cameras. To extend motion recognition systems, image-based rendering can be utilized in two ways: (i) to generate additional training sets for these systems containing a large number of non-orthogonal views, and (ii) to generate orthogonal views (the views those systems are trained to recognize) from a combination of non-orthogonal views taken from several cameras. In this case, image-based rendering is used to generate views orthogonal to the mean direction of motion. We tested the method using an existing view-dependent human motion recognition system on two different sequences of motion, and promising initial results were obtained.

The goal of this project is to monitor activities at traffic intersections for detecting/predicting situations that may lead to accidents. Some of the key elements for robust intersection monitoring are camera calibration, motion tracking, incident detection, etc. In this paper, we consider the motion-tracking problem. A multilevel tracking approach using Kalman filter is presented for tracking vehicles and pedestrians at intersections. The approach combines low-level image-based blob tracking with high-level Kalman filtering for position and shape estimation. An intermediate occlusion-reasoning module serves the purpose of detecting occlusions and filtering relevant measurements. Motion segmentation is performed by using a mixture of Gaussian models which helps us achieve fairly reliable tracking in a variety of complex outdoor scenes. A visualization module is also presented. This module is very useful for visualizing the results of the tracker and serves as a platform for the incident detection module.

This paper presents algorithms for vision-based detection and classification of vehicles in monocular image sequences of traffic scenes recorded by a stationary camera. Processing is done at three levels: raw images, region level, and vehicle level. Vehicles are modeled as rectangular patches with certain dynamic behavior. The proposed method is based on the establishment of correspondences between regions and vehicles, as the vehicles move through the image sequence. Experimental results from highway scenes are provided which demonstrate the effectiveness of the method. We also briefly describe an interactive camera calibration tool that we have developed for recovering the camera parameters using features in the image selected by the user.

This paper presents algorithms for vision-based monitoring of weaving sections. These algorithms have been developed for the Minnesota Department of Transportation in order to acquire data for several weaving sections in the Twin Cities area. Unlike commercially available systems, the proposed algorithms can track and count vehicles as they change lanes. Furthermore, they provide the velocity and the direction of each vehicle in the weaving section. Experimental results from various weaving sections under various weather conditions are presented. The proposed methods are based on the establishment of correspondences among blobs and vehicles as the vehicles move through the weaving section. The blob tracking problem is formulated as a bipartite graph optimization problem.

Grasping is one of the most fundamental and challenging tasks in robotics. Applications range from space missions (e.g., collection of rock samples) to industrial automation. In this work, we use a camera mounted on the end-effector of a manipulator to grasp an unknown object in the workspace. A novel deformable contour model is used to determine plausible grasp axes of the target object. Potential grasp point pairs are generated, ranked based upon measurements taken from the contour, and a vision-guided grasp of the object using the highest ranked grasp point pair is executed. Several experimental results are presented.