Modeling multi-modal high-level intent is important for ensuring diversity in trajectory prediction. Existing approaches explore the discrete nature of human intent before predicting continuous trajectories, to improve accuracy and support explainability. However, these approaches often assume the intent to remain fixed over the prediction horizon, which is problematic in practice, especially over longer horizons. To overcome this limitation, we introduce HYPER, a general and expressive hybrid prediction framework that models evolving human intent. By modeling traffic agents as a hybrid discrete-continuous system, our approach is capable of predicting discrete intent changes over time. We learn the probabilistic hybrid model via a maximum likelihood estimation problem and leverage neural proposal distributions to sample adaptively from the exponentially growing discrete space. The overall approach affords a better trade-off between accuracy and coverage. We train and validate our model on the Argoverse dataset, and demonstrate its effectiveness through comprehensive ablation studies and comparisons with state-of-the-art models.

Language allows humans to build mental models that interpret what is happening around them resulting in more accurate long-term predictions. We present a novel trajectory prediction model that uses linguistic intermediate representations to forecast trajectories, and is trained using trajectory samples with partially-annotated captions. The model learns the meaning of each of the words without direct per-word supervision. At inference time, it generates a linguistic description of trajectories which captures maneuvers and interactions over an extended time interval. This generated description is used to refine predictions of the trajectories of multiple agents. We train and validate our model on the Argoverse dataset, and demonstrate improved accuracy results in trajectory prediction. In addition, our model is more interpretable: it presents part of its reasoning in plain language as captions, which can aid model development and can aid in building confidence in the model before deploying it.

When predicting trajectories of road agents, motion predictors often approximate the future distribution by a limited number of samples. This constraint requires the predictors to generate samples that best support the task given task specifications. However, existing predictors are often optimized and evaluated via task-agnostic measures without accounting for the use of predictions in downstream tasks, and thus could result in sub-optimal task performance. In this paper, we propose a task-informed motion prediction model that better supports the tasks through its predictions by jointly reasoning about prediction accuracy and the utility of the downstream tasks during training. The task utility function is commonly used to evaluate task performance. It does not require the full task information, but rather a specification of the utility of the task, resulting in predictors that are tailored to different downstream tasks. We demonstrate our approach on two use cases of common decision making tasks and their utility functions, in the context of autonomous driving and parallel autonomy. Experiment results show that our predictor produces accurate predictions that improve the task performance by a large margin in both tasks when compared to task-agnostic baselines on the Waymo Open Motion dataset.

Predicting the motion of a driver’s vehicle is crucial for advanced driving systems, enabling detection of potential risks towards shared control between the driver and automation systems. In this paper, we propose a variational neural network approach that predicts future driver trajectory distributions for the vehicle based on multiple sensors.Our predictor generates both a conditional variational distribution of future trajectories, as well as a confidence estimate for different time horizons. Our approach allows us to handle inherently uncertain situations, and reason about information gain from each input, as well as combine our model with additional predictors, creating a mixture of experts.We show how to augment the variational predictor with a physics-based predictor, and based on their confidence estimations, improve overall system performance. The resulting combined model is aware of the uncertainty associated with its predictions, which can help the vehicle autonomy to make decisions with more confidence. The model is validated on real-world urban driving data collected in multiple locations. This validation demonstrates that our approach improves the prediction error of a physics-based model by 25% while successfully identifying the uncertain cases with 82% accuracy.

Motion planning for high degree of freedom (DOF) robots is not an easy task, and often requires optimization in a high dimensional space. Still, a generic motion planner using a single cost function for optimization may not be optimal over a number of different tasks with various task specific constraints. In this paper, we present a motion planning system that utilizes both easy to communicate human preferences and dimensionality reduction to handle these issues. Joint trajectories with human preference costs are projected into the null space of the task space, which helps make the resulting optimization simpler and more reliable. In addition, we apply the dimensionality reduction for the optimization, which significantly lowers the computational load. The suggested controller has been successfully used in the DARPA Robotics Challenge (DRC) Finals to handle a number of manipulation tasks.

Major rule changes for the RoboCup Humanoid League in 2015 pose significant vision and locomotion challenges for disambiguating similarly colored objects and navigating soft terrain. These significant changes highlight the need for applying general purpose humanoid robotics approaches that can handle abrupt environment modifications, and we utilize the general purpose THOR (Tactical Hazardous Operations Robot) series of robot from the recent DARPA Robotics Challenge (DRC). Specific techniques for vision, kicking and autonomy complement software developed for robust deployments in the DRC. In this paper, we present these soccer playing techniques, which were validated in the Humanoid AdultSize league in Hefei. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Developing a reliable humanoid robot that operates in uncharted real-world environments is a huge challenge for both hardware and software. Commensurate with the technology hurdles, the amount of time and money required can also be prohibitive barriers. This paper describes Team THOR's approach to overcoming such barriers for the 2013 DARPA Robotics Challenge (DRC) Trials. We focused on forming modular components - in both hardware and software - to allow for efficient and cost effective parallel development. The robotic hardware consists of standardized and general purpose actuators and structural components. These allowed us to successfully build the robot from scratch in a very short development period, modify configurations easily and perform quick field repair. Our modular software framework consists of a hybrid locomotion controller, a hierarchical arm controller and a platform-independent operator interface. These modules helped us to keep up with hardware changes easily and to have multiple control options to suit various situations. We validated our approach at the DRC Trials where we fared very well against robots many times more expensive.