Online Multi-Object Tracking (MOT) from videos is a challenging computer vision task which has been extensively studied for decades. Most of the existing MOT algorithms are based on the Tracking-by-Detection (TBD) paradigm combined with popular machine learning approaches which largely reduce the human effort to tune algorithm parameters. However, the commonly used supervised learning approaches require the labeled data (e.g., bounding boxes), which is expensive for videos. Also, the TBD framework is usually suboptimal since it is not end-to-end, i.e., it considers the task as detection and tracking, but not jointly. To achieve both label-free and end-to-end learning of MOT, we propose a Tracking-by-Animation framework, where a differentiable neural model first tracks objects from input frames and then animates these objects into reconstructed frames. Learning is then driven by the reconstruction error through backpropagation. We further propose a Reprioritized Attentive Tracking to improve the robustness of data association. Experiments conducted on both synthetic and real video datasets show the potential of the proposed model. Our project page is publicly available at: https://github.com/zhen-he/tracking-by-animation

Long Short-Term Memory (LSTM) is a popular approach to boosting the ability of Recurrent Neural Networks to store longer term temporal information. The capacity of an LSTM network can be increased by widening and adding layers. However, usually the former introduces additional parameters, while the latter increases the runtime. As an alternative we propose the Tensorized LSTM in which the hidden states are represented by tensors and updated via a cross-layer convolution. By increasing the tensor size, the network can be widened efficiently without additional parameters since the parameters are shared across different locations in the tensor; by delaying the output, the network can be deepened implicitly with little additional runtime since deep computations for each timestep are merged into temporal computations of the sequence. Experiments conducted on five challenging sequence learning tasks show the potential of the proposed model.

In the last decade, autonomous driving technology has become an important research topic due to its potential economic and social benefits. There has been considerable research activities contributed to make the autonomous driving system adapt to complex environments. Motion control is vital to the overall autonomous driving system, especially when the autonomous vehicle is driving in complex off-road environments. The aim of our work in this paper is to develop a high-precision motion controller for autonomous driving system running on rugged mountain roads and sand roads. Different from most existing methods in which the motion control problem is decoupled into lateral control and longitudinal control. In this work, a coupling controller is designed for solving the motion control problem of autonomous driving system. Experiments in the real-world rugged mountain road and sand road environment have been conducted to demonstrate the high-precision performance and efficiency of the proposed motion controller.

Dynamic vehicle detection and tracking is crucial for self-driving in urban environments. The main problem of the previous beam-model-based algorithms is that they cannot detect and track dynamic vehicles that are occluded by other objects. In this paper, we develop a novel dynamic vehicle detection and tracking algorithm to solve this problem for our autonomous land vehicle (ALV), which is equipped with a Velodyne LIDAR and a GPS-aid inertial navigation system. For detection, our improved two-dimensional virtual scan is presented to detect the potential dynamic vehicles with a scan differencing operation. Then, for each potential dynamic vehicle, a novel likelihood-field-based vehicle measurement model is proposed to weight its possible poses. Finally, our newly modified scaling series algorithm and the importance sampling technique are adopted to estimate the initial pose and the corresponding velocity for each vehicle, respectively. The scaling series algorithm coupled with a Bayesian filter (SSBF) was previously used to handle the tactile localization problem in static background scenes. For tracking dynamic vehicles, we improve the SSBF by adding the ego-motion compensation so that the improved algorithm is able to update the pose and velocity for each vehicle in dynamic background scenes. Both the quantitative and qualitative experimental results validate the performance of our dynamic vehicle detection and tracking algorithm on the KITTI datasets and the Velodyne data collected by our ALV in dynamic urban environments.

Long range curb detection is crucial for an Autonomous Land Vehicle (ALV) navigation in urban environments. This paper presents a novel curb detection algorithm which can detect the curbs up to 50 meters away with Velodyne LIDAR. Instead of building a Digital Elevation Map (DEM) and utilizing geometric features (like normal direction) to extract candidate curb points, we take each scan line of Velodyne LIDAR as a processing unite directly. Some feature points, which are extracted from individual scan lines, are selected as the initial curb points by the distance criterion and Hough Transform (HT). Eventually, iterative Gaussian Process Regression (GPR), which utilizes the above initial curb points as the initial seeds, is exploited to represent both the curved and straight-line curb model. In order to verify the effectiveness of our algorithm quantitatively, 2934 Velodyne scans are collected in various urban scenes with our ALV, and 566 of them are labelled manually1. Our algorithm is also compared with two other curb detection techniques. The experimental results on the dataset show promising performance.

This paper presents a practical trajectory planning framework towards fully autonomous driving in urban environments. Firstly, based on the behavioral decision commands, a reference path is extracted from the digital map using the LIDAR-based localization information. The reference path is refined and interpolated via a nonlinear optimization algorithm and a parametric algorithm, respectively. Secondly, the trajectory planning task is decomposed into spatial path planning and velocity profile planning. A closed-form algorithm is employed to generate a rich set of kinematically-feasible spatial path candidates within the curvilinear coordinate framework. At the same time, the velocity planning algorithm is performed with considering safety and smoothness constraints. The trajectory candidates are evaluated by a carefully developed objective function. Subsequently, the best collision-free and dynamically-feasible trajectory is selected and executed by the trajectory tracking controller. We implemented the proposed trajectory planning strategy on our test autonomous vehicle in the realistic urban traffic scenarios. Experimental results demonstrated its capability and efficiency to handle a variety of driving situations, such as lane keeping, lane changing, vehicle following, and static and dynamic obstacles avoiding, while respecting traffic regulations.

In this paper, we propose to fuse the LIDAR and monocular image in the framework of conditional random field to detect the road robustly in challenging scenarios. LIDAR points are aligned with pixels in image by cross calibration. Then boosted decision tree based classifiers are trained for image and point cloud respectively. The scores of the two kinds of classifiers are treated as the unary potentials of the corresponding pixel nodes of the random field. The fused conditional random field can be solved efficiently with graph cut. Extensive experiments tested on KITTI-Road benchmark show that our method reaches the state-of-the-art.

Object Recognition is an essential component for Autonomous Land Vehicle (ALV) navigation in urban environments. This paper presents a thorough evaluation of the performance of some state of the art global descriptors on the public Sydney Urban Objects Dataset1, which was collected in the Central Business District of Sydney. These descriptors are Bounding Box descriptor, Histogram of Local Point Level descriptor, Hierarchy descriptor, and Spin Image (SI). We also propose a novel Global Fourier Histogram (GFH) descriptor. Experimental results on the public data set show that GFH descriptor turns out to be one of the best global descriptors for the object recognition in urban environments, and the results on the data collected by our own ALV in urban environments also demonstrate its usefulness.

Curb detection is an important research topic for unmanned ground vehicle (UGV) navigation. In this paper, a new curb detection method is proposed using a 2D laser range finder in a campus environment. Firstly, a local Digital Elevation Map (DEM) is built with 2D sequential laser range finder data and vehicle state information. Then, the curb candidate points are extracted considering the moving direction of the vehicle in the local DEM. Finally, the 1D Gaussian process regression is firstly used for curb detection, and the initial training curb data are obtained online by the extracted straight curbs. The proposed method has been verified in different scenes with the real vehicle platform, and it can detect the straight and curved curbs robustly in a campus environment.

To address the path tracking problem of autonomous land vehicle, a new vehicle-road model named “Ribbon Model” is constructed under the constraints of road width and vehicle geometry structure. A new vehicle-road evaluation algorithm is developed based on this model, and new path tracking controller is designed. The difficulties of preview distance selection and parameters tuning with speed of pure following controller are avoided in this controller. Performance of the novel method is verified by simulation and vehicle experiments.

Long range road intersection detection is crucial for localization and local path planning of autonomous vehicle in urban environments. In this paper, a new long-range road intersection detection approach for autonomous vehicle equipped with 3D LIDAR is presented. The approach first analyzes the admissible space in front of the autonomous vehicle, and then a virtual 3D LIDAR is placed in the admissible space 20 meters away from the vehicle. Finally the beam model of range finders and an improved toe-finding algorithm for virtual 3D LIDAR is used to find the road intersection. Experiments are carried out at the autonomous vehicle in campus, and results show the promising performance of the presented method.