One key challenge for multi-task Reinforcement learning (RL) in practice is the absence of task specifications. Robust RL has been applied to deal with task ambiguity but may result in over-conservative policies. To balance the worst-case (robustness) and average performance, we propose Group Distributionally Robust Markov Decision Process (GDR-MDP), a flexible hierarchical MDP formulation that encodes task groups via a latent mixture model. GDR-MDP identifies the optimal policy that maximizes the expected return under the worst-possible qualified belief over task groups within an ambiguity set. We rigorously show that GDR-MDP’s hierarchical structure improves distributional robustness by adding regularization to the worst possible outcomes. We then develop deep RL algorithms for GDR-MDP for both value-based and policy-based RL methods. Extensive experiments on Box2D control tasks, MuJoCo benchmarks, and Google football platforms show that our algorithms outperform classic robust training algorithms across diverse environments in terms of robustness under belief uncertainties. Demos are available on our project page (https://sites.google.com/view/gdr-rl/home).

Reasoning about the future behavior of other agents is critical to safe robot navigation. The multiplicity of plausible futures is further amplified by the uncertainty inherent to agent state estimation from data, including positions, velocities, and semantic class. Forecasting methods, however, typically neglect class uncertainty, conditioning instead only on the agent's most likely class, even though perception models often return full class distributions. To exploit this information, we present HAICU, a method for heterogeneous-agent trajectory forecasting that explicitly incorporates agents' class probabilities. We additionally present PUP, a new challenging real-world autonomous driving dataset, to investigate the im-pact of Perceptual Uncertainty in Prediction. It contains chal-lenging crowded scenes with unfiltered agent class probabilities that reflect the long-tail of current state-of-the-art perception systems. We demonstrate that incorporating class probabilities in trajectory forecasting significantly improves performance in the face of uncertainty, and enables new forecasting capabilities such as counterfactual predictions.

Video captioning is a challenging task that requires a deep understanding of visual scenes. State-of-the-art methods generate captions using either scene-level or object-level information but without explicitly modeling object interactions. Thus, they often fail to make visually grounded predictions, and are sensitive to spurious correlations. In this paper, we propose a novel spatio-temporal graph model for video captioning that exploits object interactions in space and time. Our model builds interpretable links and is able to provide explicit visual grounding. To avoid unstable performance caused by the variable number of objects, we further propose an object-aware knowledge distillation mechanism, in which local object information is used to regularize global scene features. We demonstrate the efficacy of our approach through extensive experiments on two benchmarks, showing our approach yields competitive performance with interpretable predictions.

In autonomous driving, accurately estimating the state of surrounding obstacles is critical for safe and robust path planning. However, this perception task is difficult, particularly for generic obstacles/objects, due to appearance and occlusion changes. To tackle this problem, we propose an end-to-end deep learning framework for LIDAR-based flow estimation in bird's eye view (BeV). Our method takes consecutive point cloud pairs as input and produces a 2-D BeV flow grid describing the dynamic state of each cell. The experimental results show that the proposed method not only estimates 2-D BeV flow accurately but also improves tracking performance of both dynamic and static objects.

Traffic simulators are important tools in autonomous driving development. While continuous progress has been made to provide developers more options for modeling various traffic participants, tuning these models to increase their behavioral diversity while maintaining quality is often very challenging. This paper introduces an easily-tunable policy generation algorithm for autonomous driving agents. The proposed algorithm balances diversity and driving skills by leveraging the representation and exploration abilities of deep reinforcement learning via a distinct policy set selector. Moreover, we present an algorithm utilizing intrinsic rewards to widen behavioral differences in the training. To provide quantitative assessments, we develop two trajectory-based evaluation metrics which measure the differences among policies and behavioral coverage. We experimentally show the effectiveness of our methods on several challenging intersection scenes.

We tackle the problem of Human Locomotion Forecasting, a task for jointly predicting the spatial positions of several keypoints on human body in the near future under an egocentric setting. In contrast to the previous work that aims to solve either the task of pose prediction or trajectory forecasting in isolation, we propose a framework to unify these two problems and address the practically useful task of pedestrian locomotion prediction in the wild. Among the major challenges in solving this task is the scarcity of annotated egocentric video datasets with dense annotations for pose, depth, or egomotion. To surmount this difficulty, we use state-of-the-art models to generate (noisy) annotations and propose robust forecasting models that can learn from this noisy supervision. We present a method to disentangle the overall pedestrian motion into easier to learn subparts by uti-lizing a pose completion and a decomposition module. The completion module fills in the missing key-point annotations and the decomposition module breaks the cleaned locomotion down to global (trajectory) and local (pose keypoint movements). Further, with Quasi RNN as our backbone, we propose a novel hierarchical trajectory forecasting network that utilizes low-level vision domain specific signals like egomotion and depth to predict the global trajectory. Our method leads to state-of-the-art results for the prediction of human locomotion in the egocentric view.

Vehicle taillight recognition is an important application for automated driving, especially for intent prediction of ado vehicles and trajectory planning of the ego vehicle. In this work, we propose an end-to-end deep learning framework to recognize taillights, i.e. rear turn and brake signals, from a sequence of images. The proposed method starts with a Convolutional Neural Network (CNN) to extract spatial features, and then applies a Long Short-Term Memory network (LSTM) to learn temporal dependencies. Furthermore, we integrate attention models in both spatial and temporal domains, where the attention models learn to selectively focus on both spatial and temporal features. Our method is able to outperform the state of the art in terms of accuracy on the UC Merced Vehicle Rear Signal Dataset, demonstrating the effectiveness of attention models for vehicle taillight recognition.

This paper proposes the Permutation Matrix Track Association (PMTA) algorithm to support track-to-track, multi-sensor data fusion for multiple targets in an autonomous driving system. In this system, measurement data from different sensor modalities (LIDAR, radar, and vision) is processed by object trackers operating on each sensor modality independently to create the tracks of the objects. The proposed approach fuses the object track lists from each tracker, first by associating the tracks within each track list, followed by a state estimation (filtering) step. The eventual output is the unified tracks of the objects provided for further autonomous driving processing, such as path and motion planning. The permutation matrix track association (PMTA) algorithm considers both spatial and temporal information to associate object tracks from different sensor modalities. Experimental results show that the proposed approach improves not only the performance of the multipletarget track-to-track fusion, but also stability and robustness in the resulting speed control and decision making in the autonomous driving system.

More than 70% of consumer mobile Internet traffic will be mobile video transmissions by 2019. The development of wireless video transmission technologies has been boosted by the rapidly increasing demand of video streaming applications. Although more and more videos are delivered for video analysis (e.g., object detection/tracking and action recognition), most existing wireless video transmission schemes are developed to optimize human perception quality and are suboptimal for video analysis. In mobile surveillance networks, a cloud server collects videos from multiple moving cameras and detects suspicious persons in all camera views. Camera mobility in smartphones or dash cameras implies that video is to be uploaded through bandwidth-limited and error-prone wireless networks, which may cause quality degradation of the decoded videos and jeopardize the performance of video analyses. In this paper, we propose an effective rate-allocation scheme for multiple moving cameras in order to improve human detection (content) performance. Therefore, the optimization criterion of the proposed rate-allocation scheme is driven by quality of content (QoC). Both video source coding and application layer forward error correction coding rates are jointly optimized. Moreover, the proposed rate-allocation problem is formulated as a convex optimization problem and can be efficiently solved by standard solvers. Many simulations using High Efficiency Video Coding standard compression of video sequences and the deformable part model object detector are carried, and results demonstrate the effectiveness and favorable performance of our proposed QoC-driven scheme under different pedestrian densities and wireless conditions.

In a video surveillance system with a single static camera, tracking results of moving persons can be effectively used for camera self-calibration. However, the current methods need to depend on robustness of both tracking and segmentation procedures. RANSAC has been widely used to remove outliers in finding the vertical vanishing point and the horizon line, but the performance is degraded when the proportion of outliers is high. Last but not least, all of them require excessive simplifications in the algorithmic procedures resulting in increasing reprojection error. In this paper, a robust segmentation and tracking system is applied to provide accurate estimation of head and foot locations of moving persons. The noise in the computation of vanishing points is handled by mean shift clustering and Laplace linear regression through convex optimization. We also propose to use the estimation of distribution algorithm (EDA) to search for the local optimal solution for camera calibration that minimizes average reprojection error on the ground plane, while relaxing the assumptions on camera parameters. Promising evaluations of the performance of our proposed method on real scenes are presented.

This paper proposes a robust ground-moving-platform-based human tracking system, which effectively integrates visual simultaneous localization and mapping (V-SLAM), human detection, ground plane estimation, and kernel-based tracking techniques. The proposed system systematically detects humans from recorded video frames of a moving camera and tracks the humans in the V-SLAM-inferred 3-D space via a tracking-by-detection scheme. To efficiently associate the detected human frame by frame, we propose a novel human tracking framework, combining the constrained-multiple-kernel tracking and the estimated 3-D information (depth), to globally optimize the data association between consecutive frames. By taking advantage of the appearance model and 3-D information, the proposed system not only achieves high effectiveness but also well handles occlusion in the tracking. Experimental results show the favorable performance of the proposed system, which efficiently tracks humans in a camera equipped on a ground-moving platform such as a dash camera and an unmanned ground vehicle.

In this paper, we propose an innovative human tracking algorithm, which efficiently integrates the deformable part model (DPM) into the multiple-kernel based tracking using a moving camera. By representing each part model of a DPM detected human as a kernel, the proposed algorithm iteratively mean-shift the kernels (i.e., part models) based on color appearance and histogram of gradient (HOG) features. More specifically, the color appearance features, in terms of kernel histogram, are used for tracking each body part from one frame to the next, the deformation cost provided by DPM detector is further used to constrain the movement of each body kernel based on the HOG features. The proposed deformable multiple-kernel (DMK) tracking algorithm takes advantage of not only low computation owing to the kernelbased tracking, but also robustness of the DPM detector. Experimental results have shown the favorable performance of the proposed algorithm, which can successfully track human using a moving camera more accurately under different scenarios.

Nowadays, more and more videos are transmitted for video analytics purposes rather than human perceptions. In mobile surveillance networks, a cloud server collects videos delivered from multiple moving cameras and detects suspicious people in all the camera views. However, all the videos recorded by moving cameras such as phone or dash cameras are uploaded through bandwidth-limited wireless networks. Therefore, videos are required to be encoded with high compression ratio to satisfy the total data rate constraint, which may affect the video analyses (e.g., human detection/tracking and action recognition, etc.) performance due to the degraded video decoding qualities at the server side. In this paper, we propose an effective content-driven video source coding rate allocation scheme, which can improve the human detection success rate in mobile surveillance networks under a total data rate constraint. The proposed scheme allocates appropriate amount of data rate to each moving camera based on the corresponding content information (i.e., human detection results). A model of human detection accuracy based on object area and video quality is provided. The rate allocation problem is formulated as a convex optimization problem and can be solved by standard solvers. Simulations with real video sequences demonstrate the effectiveness of our proposed scheme.

In this paper, we propose a novel model-based vehicle localization approach on the basis of surveillance cameras. The proposed approach regards each patch of the 3-D vehicle model as a kernel, and tracks the kernels under certain constrains facilitated by the 3-D geometry of the vehicle model. Meanwhile, a kernel density estimator is designed to well fit the 3-D vehicle model during tracking. With elegant application of the constrained multiple-kernel tracking facilitated with the 3-D vehicle model, the vehicles are able to be tracked efficiently and located precisely. The proposed approach achieves high effectiveness in the tracking and localization by taking advantage of the color similarity and shape fitness. Experimental results have shown the favorable performance of the proposed approach, in several scenarios, which efficiently tracks vehicles while maintaining the knowledge of 3-D geometry of the tracked vehicles.

In this paper, we propose a new framework to track on-road pedestrians across multiple driving recorders. The framework is built upon the results of tracking under a single driving recorder. More specifically, we treat the problem as a multi-label classification task and determine whether a specific pedestrian belongs to one or several cameras' field of views by considering association likelihood of the tracked pedestrians . The likelihood is calculated based on the pedestrians' motion cues and appearance features, which are necessarily transformed via brightness transfer functions obtained by some available spatially overlapping views for compensating diversity of the cameras. When a pedestrian is leaving a camera's field of view, the proposed framework predicts and interpolates its possible moving trajectories, facilitated by open map service which can provide routing information. Experimental results show the robustness and effectiveness of the proposed framework in tracking pedestrians across several recorded driving videos. Moreover, based on the GPS locations, we can also reconstruct a 3-D visualization on a 3-D virtual real-world environment, so as to show the dynamic scenes of the recorded videos.

We present a self-organized and scalable multiple-camera tracking system that tracks human across the cameras with nonoverlapping views. Given the GPS locations of uncalibrated cameras, the system automatically detects the existence of camera link within the camera network based on the routing information provided by Google Maps. The connected zones in any pair of directly-connected cameras are identified based on the feature matching between the camera's view and Google Street View. The camera link model is further estimated by an unsupervised learning scheme. Finally, multiple-camera tracking is performed. Thanks to the unsupervised pairwise learning and tracking in our system, the camera network is self-organized, and our proposed system is able to be scaled up efficiently when more cameras are added into the network.

In this paper, we propose a novel vehicle tracking system under a surveillance camera. The proposed system tracks vehicles by using constrained multiple-kernel, facilitated with Kalman filtering, to continuously update the position and the orientation of the moving vehicles. To further reliably track vehicles under partial occlusion or even total occlusion, our tracking algorithm also systematically builds 3-D vehicle model, from which the license plate region is identified and a self-similarity descriptor is further used for low-resolution license plate matching. Experimental results have shown the favorable performance of the proposed system, which can successfully track vehicles under serious occlusion while maintaining the knowledge of 3-D geometry of the tracked vehicles.

In this paper, we propose a novel vehicle tracking system under a surveillance camera. The proposed system tracks vehicles by using constrained multiple-kernel facilitated with Kalman filtering, and then continuously updates the position and the orientation by adopting a systematically built 3-D vehicle model in an evolutionary computing framework. The proposed system can thus successfully track vehicles under occlusion as facilitated by the obtained 3-D geometry of vehicles. Experimental results have shown the favorable performance of the proposed system, which can successfully track vehicles while maintaining the knowledge of 3-D vehicle geometry.