A self-driving car, to be deployed in real-world driving environments, must be capable of reliably detecting and effectively tracking of nearby moving objects. This paper presents our new, moving object detection and tracking system that extends and improves our earlier system used for the 2007 DARPA Urban Challenge. We revised our earlier motion and observation models for active sensors (i.e., radars and LIDARs) and introduced a vision sensor. In the new system, the vision module detects pedestrians, bicyclists, and vehicles to generate corresponding vision targets. Our system utilizes this visual recognition information to improve a tracking model selection, data association, and movement classification of our earlier system. Through the test using the data log of actual driving, we demonstrate the improvement and performance gain of our new tracking system.

We describe a real-time pedestrian detection system intended for use in automotive applications. Our system demonstrates superior detection performance when compared to many state-of-the-art detectors and is able to run at a speed of 14 fps on an Intel Core i7 computer when applied to 640×480 images. Our approach uses an analysis of geometric constraints to efficiently search feature pyramids and increases detection accuracy by using a multiresolution representation of a pedestrian model to detect small pixel-sized pedestrians normally missed by a single representation approach. We have evaluated our system on the Caltech Pedestrian benchmark which is currently the largest publicly available pedestrian dataset at the time of this publication. Our system shows a detection rate of 61% with 1 false positive per image (FPPI) whereas recent other state-of-the-art detectors show a detection rate of 50% ~ 61% under the `reasonable' test scenario (explained later). Furthermore, we also demonstrate the practicality of our system by conducting a series of use case experiments on selected videos of Caltech dataset.

This paper presents a monocular vision based 3D bicycle tracking framework for intelligent vehicles based on a detection method exploiting a deformable part model and a tracking method using an Interacting Multiple Model (IMM) algorithm. Bicycle tracking is important because bicycles share the road with vehicles and can move at comparable speeds in urban environments. From a computer vision standpoint, bicycle detection is challenging as bicycle's appearance can change dramatically between viewpoints and a person riding on the bicycle is a non-rigid object. To this end, we present a tracking-by-detection method to detect and track bicycles that takes into account these difficult issues. First, a mixture model of multiple viewpoints is defined and trained via a Latent Support Vector Machine (LSVM) to detect bicycles under a variety of circumstances. Each model uses a part based representation. This robust bicycle detector provides a series of measurements (i.e., bounding boxes) in the context of the Kalman filter. Second, to exploit the unique characteristics of bicycle tracking, two motion models based on bicycle's kinematics are fused using an IMM algorithm. For each motion model, an extended Kalman filter (EKF) is used to estimate the position and velocity of a bicycle in the vehicle coordinates. Finally, a single bicycle tracking method using an IMM algorithm is extended to that of multiple bicycle tracking by incorporating a Rao-Blackwellized Particle Filter which runs a particle filter for a data association and an IMM filter for each bicycle tracking. We demonstrate the effectiveness of this approach through a series of experiments run on a new bicycle dataset captured from a vehicle-mounted camera.

This paper presents a vision-based framework for intelligent vehicles to detect and track people riding bicycles in urban traffic environments. To deal with dramatic appearance changes of a bicycle according to different viewpoints as well as nonrigid nature of human appearance, a method is proposed which employs complementary detection and tracking algorithms. In the detection phase, we use multiple view-based detectors: frontal, rear, and right/left side view. For each view detector, a linear Support Vector Machine (SVM) is used for object classification in combination with Histograms of Oriented Gradients (HOG) which is one of the most discriminative features. Furthermore, a real-time enhancement for the detection process is implemented using the Integral Histogram method and a coarse-to-fine cascade approach. Tracking phase is performed by a multiple patch-based Lucas-Kanade tracker. We first run the Harris corner detector over the bounding box which is the result of our detector. Each of the corner points can be a good feature to track and, in consequence, becomes a template of each instance of multiple Lucas-Kanade trackers. To manage the set of patches efficiently, a novel method based on spectral clustering algorithm is proposed. Quantitative experiments have been conducted to show the effectiveness of each component of the proposed framework.