Predicting the future behavior of road users is one of the most challenging and important problems in autonomous driving. Applying deep learning to this problem requires fusing heterogeneous world state in the form of rich perception signals and map information, and inferring highly multi-modal distributions over possible futures. In this paper, we present MultiPath++, a future prediction model that achieves state-of-the-art performance on popular benchmarks. MultiPath++ improves the MultiPath architecture [34] by revisiting many design choices. The first key design difference is a departure from dense image-based encoding of the input world state in favor of a sparse encoding of heterogeneous scene elements: MultiPath++ consumes compact and efficient polylines to describe road features, and raw agent state information directly (e.g., position, velocity, acceleration). We propose a context-aware fusion of these elements and develop a reusable multi-context gating fusion component. Second, we reconsider the choice of pre-defined static anchors, and develop a way to learn latent anchor embeddings end-to-end in the model. Lastly, we explore ensembling and output aggregation techniques—common in other ML domains—and find effective variants for our probabilistic multimodal output representation. We perform an extensive ablation on these design choices, and show that our proposed model achieves state-of-the-art performance on the Argoverse Motion Forecasting Competition [10] and the Waymo Open Dataset Motion Prediction Challenge [13].

Existing commercial driver assistance systems, including automatic braking systems and lane-keeping systems, may monitor the state of the vehicle or the environment to determine whether the systems should intervene. However, the state of the human driver is not typically included in the decision making process. In this paper, we propose to use hidden mode stochastic hybrid systems to model the interaction between the human driver and the vehicle. We show that by monitoring the human behavior as well as the vehicle state, we can infer the human state and enhance the quality of decision making in a driver assistance system. The resulting control policy is obtained by solving an optimal planning problem of the proposed hidden mode hybrid system. The policy can automatically balance the decision making about when to give warning to the driver and when to actually intervene in the control of the vehicle.

This paper proposes a navigation algorithm that considers the states of humans and robots in order to achieve harmonious coexistence between them. A robot navigation in the presence of humans and other robots is rarely considered in the field of robotics. When navigating through a space filled with humans and robots with different functions, a robot should not only pay attention to obstacle avoidance and goal seeking, it should also take into account whether it interferes with other people or robots. To deal with this problem, we propose several harmonious rules, which guarantee a safe and smooth navigation in a human-robot environment. Based on these rules, a practical navigation method-human-centered sensitive navigation (HCSN)-is proposed. HCSN considers the fact that both humans and robots have sensitive zones, depending on their security regions or on a human's psychological state. We model these zones as various sensitive fields with priorities, whereby robots tend to yield socially acceptable movements.

This paper proposes a navigation algorithm that considers the states of humans and other robots in order to achieve harmonious coexistence between robots and humans. When navigating through humans and robots with different functions, a robot should not only pay attention to obstacle avoidance and goal seeking, it should also take care of whether it interferes with other people or robots. To deal with this problem, we propose several harmonious rules, which guarantee a safe and smooth navigation in multi-human and multi-robot (MHMR) environment. Based on those rules, a practical navigation method-human- centered sensitive navigation (HCSN)-is proposed. HCSN considers the fact that both humans and robots have sensitive zones depending on their security regions or on psychological feeling of people. We model these zones as various sensitive fields with priorities, whereby robots tend to yield socially acceptable movements.