Moving object detection is an important component of a traffic surveillance system. Usual background subtraction approaches often poorly perform on a long outdoor traffic video due to vehicles waiting at an intersection and gradual changes of illumination and background shadow position. We present a fast and robust background subtraction algorithm based on unified spatio-temporal background and foreground modeling. The correlation between neighboring pixels provides high levels of detection accuracy in the dynamic background scene. Our Bayesian fusion method, which establishes the traffic scene model, combines both background and foreground models and considers prior probabilities to adapt changes of background in each frame. We explicitly model both temporal and spatial information based on the kernel density estimation (KDE) formulation for background modeling. Then, we use a Gaussian formulation to describe the spatial correlation of moving objects for foreground modeling. In the updating step, a fusion background frame is generated, and reasonable updating rates are also proposed for the traffic scene. The experimental results show that the proposed method outperforms the previous work with less computation and is better suited for the traffic scenes.

We introduce a formulation for ¿orthogonal calibration¿ which is to extract camera calibration parameters from the world coordinates with unknown heights. Such coordinates can be obtained from an aerial photograph or a GPS device. A typical approach would be to introduce a planar surface assumption and apply the plane homography. However, the planar assumption often fails; moreover, the camera parameters recovered from a homography matrix is highly sensitive to noise. We introduce a formulation for the orthogonal calibration which is similar to the fundamental matrix equation. Based on the formulation, we introduce a 6-point algorithm to recover the rotation and translation and a 7-point algorithm to recover the focal lengths in addition. Then, we introduce a noisy-RANSAC method which enables robust parameter recovery from a small number of point correspondences. The noisy-RANSAC naturally incorporates the orthogonal calibration and any available information on the scene structure, which delivers much improved estimates. Experimental results on synthetic datasets and an example application to a real image are presented.

Object detection and tracking has various application areas including intelligent transportation systems. We introduce an object detection and tracking approach that combines the background subtraction algorithm and the feature tracking and grouping algorithm. We first present an augmented background subtraction algorithm which uses a low-level feature tracking as a cue. The resulting background subtraction cues are used to improve the feature detection and grouping result. We then present a dynamic multi-level feature grouping approach that can be used in real time applications and also provides high-quality trajectories. Experimental results from video clips of a challenging transportation application are presented.

Sensor-based control is an emerging challenge in UAV applications. It is essential in a sensing task to account for sensor measurement errors when computing a target position estimate. Source of measurement error includes those in vehicle position and orientation measurements as well as algorithm failures such as missed detections or false detections. Incorporating such errors in aerial sensors is non-trival because of the camera’s perspective geometry. This paper is about a method to incorporate such errors into target position estimates and a calibration methodology to measure the error distributions. A preliminary experiment with real flight data is presented.

A lane-detection system is an important component of many intelligent transportation systems. We present a robust lane-detection-and-tracking algorithm to deal with challenging scenarios such as a lane curvature, worn lane markings, lane changes, and emerging, ending, merging, and splitting lanes. We first present a comparative study to find a good real-time lane-marking classifier. Once detection is done, the lane markings are grouped into lane-boundary hypotheses. We group left and right lane boundaries separately to effectively handle merging and splitting lanes. A fast and robust algorithm, based on random-sample consensus and particle filtering, is proposed to generate a large number of hypotheses in real time. The generated hypotheses are evaluated and grouped based on a probabilistic framework. The suggested framework effectively combines a likelihood-based object-recognition algorithm with a Markov-style process (tracking) and can also be applied to general-part-based object-tracking problems. An experimental result on local streets and highways shows that the suggested algorithm is very reliable.

It is important to understand the factors underlying grade-crossing crashes and to examine potential solutions. We have installed a camera in front of a locomotive to examine grade-crossing accidents (or near accidents). We present a computer vision system that automatically extracts possible near-accident scenes by detecting the activity of vehicles crossing in front of the train after signals are ignited. We present a fast algorithm to detect moving objects recorded by a moving camera with minimal computation. The moving object is detected by: 1) estimating the ego motion of the camera and 2) detecting and tracking feature points whose motion is inconsistent with the camera motion. We introduce a pseudoreal-time ego-motion (camera-motion) estimation method with a robust optimization algorithm. We present experiments on ego-motion estimation and moving-object detection. Our algorithm works in pseudoreal-time and we expect that our algorithm can be applied to real-time applications such as collision warning in the near future, with the development of hardware technology.

Generating vehicle trajectories from video data is an important application of ITS (intelligent transportation systems). We introduce a new tracking approach which uses model-based 3-D vehicle detection and description algorithm. Our vehicle detection and description algorithm is based on a probabilistic line feature grouping, and it is faster (by up to an order of magnitude) and more flexible than previous image-based algorithms. We present the system implementation and the vehicle detection and tracking results.

Computing 3D object descriptions from images is an important goal of computer vision. A key problem here is the evaluation of a hypothesis based on evidence that is uncertain. There have been few efforts on applying formal reasoning methods to this problem. In multiview and multimode object description problems, reasoning is required on evidence features extracted from multiple images and nonintensity data. One challenge here is that the number of the evidence features varies at runtime because the number of images being used is not fixed and some modalities may not always be available. We introduce an augmented Bayesian network, the expandable Bayesian network (EBN), which instantiates its structure at runtime according to the structure of input. We introduce the use of hidden variables to handle correlation of evidence features across images. We show an application of an EBN to a multiview building description system. Experimental results show that the proposed method gives significant and consistent performance improvement to others.

We present a model-based approach to detecting and describing compositions of buildings with complex rooftops. Previous approaches have dealt with either simpler models or models which lack geometric information. In spite of increasing model complexity, we maintain the computation affordable by effectively using multiple overlapping images. We obtain rooftop hypotheses in 3-D by using 3-D lines and junctions generated from multiple images. Image-derived unedited elevation data is used to assist feature matching, and to generate rough cues of the presence of 3-D structures. Experimental results are shown on complex buildings.

Machine perception can benefit from the use of features extracted from data provided by a variety of sensor modalities. Recent advances in sensor design makes it possible to incorporate multiple sensors into vision systems for increased capability. Two important issues must be considered for the integration task. The sensors must be spatially coregistered and the phenomenologies must be compatible. In this paper we address these issues as they apply to the problem of automatic modeling of building structures from aerial views. We present a methodology to incorporate cues extracted from IFSAR (Interferometric Synthetic Aperture Radar) to significantly improve the performance and the quality of the results of an existing system that relies on electro-optical panchromatic images, while reducing processing time. Quantitative evaluations are given.

3-D building detection and description is a practical application of 3-D object description, a key task of computer vision. We present an approach to detecting and describing buildings of polygonal rooftops by using multiple, overlapping images of the scene. First, 3-D features are generated by using multiple images, and rooftop hypotheses are generated by neighborhood searches on those features. For robust generation of 3-D features, we present a probabilistic approach to address the epipolar alignment problem in line matching. Image-derived unedited elevation data is used to assist feature matching, and to generate rough cues of the presence of 3-D structures. These cues help reduce the search space significantly. Experimental results are shown on some complex buildings.