How can I improve my performance here?Figure 1: After completing a lap, the driver pauses the video at a specific segment and asks ApexTrainer for feedback.The system generates context-aware suggestions based on the referenced location.

Customizing autonomous vehicles to align with user preferences while ensuring safety may significantly impact their adoption. Collecting user preference data by asking a large number of comparison questions can be demanding. In this work, we use active learning along with temporal logic descriptions of constraints to enable safe learning of preferences with a reduced number of questions. We take a Bayesian inference approach combined with Weighted Signal Temporal Logic (WSTL), resulting in a WSTL formula that can rank signals based on user preferences and be used for correct-and-custom-by-construction control synthesis. Our method is practical for formulas and signals with various complexity since we compute STL-related values offline. We provide an upper bound for the number of answers in disagreement with user answers. We demonstrate the performance of our method both on synthetic data and by human subject experiments in an immersive driving simulator. We consider two driving scenarios, one involving a vehicle approaching a pedestrian crossing and the other with an overtake maneuver. Our results over synthetic experiments with ground truth weight valuation show that our query selection algorithm converges faster than random query selection. Human subject study results show an average agreement of 94% with user answers during training, and 79% during validation (which increases to 86% when restricted to high confidence results).

Personalization of autonomous vehicles (AVs) may significantly increase acceptance. In particular, we hypothesize that the similarity of an AV's driving style compared to a user's driving style, the level of aggressiveness of the driving style, and other subjective factors (e.g., personality) will have a major impact on user's willingness to use the AV. In this work, we 1) develop a data-driven approach to personalize driving style and calibrate the level of aggressiveness and 2) investigate the subjective factors that impact user preference. Across two human subject studies (n = 54), we demonstrate that our approach can mimic the driving styles and tune the level of aggressiveness. Second, we leverage our framework to investigate the factors that impact homophily. We demonstrate that our approach generates driving styles objectively ($p < .001$) and subjectively ($p = .002$) consistent with end-user styles ($p < .001$) and can effectively isolate and modulate a dimension of style (i.e., aggressiveness) ($p < .001$). Furthermore, we find that personality ($p < .001$), perceived similarity ($p < .001$), and high-velocity driving style ($p = .0031$) significantly modulate the effect of homophily.

In this work, we propose a safety-guaranteed personalization for autonomous vehicles by incorporating Signal Temporal Logic (STL) into preference learning problem. We propose a new variant of STL called Parametric Weighted Signal Temporal Logic with a new quantitative semantics, namely weighted robustness. Given a set of pairwise preferences, and by using gradient-based optimization methods, we learn a set of valuations for weights that reflect preferences such that preferred ones have greater weighted robustness value than their non-preferred matches. Traditional STL formulas fail to incorporate preferences due its complex nature. Our initial results with data from a human-subject on an intersection with stop sign driving scenario, in which the participant is asked their preferred driving behavior from pairs of vehicle trajectories, indicate that we can learn a new weighted STL formula that captures preferences while also encoding correctness.

We present a novel approach for generating plausible verbal interactions between virtual human-like agents and user avatars in shared virtual environments. Sense-Plan-Ask, or SPA, extends prior work in propositional planning and natural language processing to enable agents to plan with uncertain information, and leverage question and answer dialogue with other agents and avatars to obtain the needed information and complete their goals. The agents are additionally able to respond to questions from the avatars and other agents using natural-language enabling real-time multi-agent multi-avatar communication environments.Our algorithm can simulate tens of virtual agents at interactive rates interacting, moving, communicating, planning, and replanning. We find that our algorithm creates a small runtime cost and enables agents to complete their goals more effectively than agents without the ability to leverage natural-language communication. We demonstrate quantitative results on a set of simulated benchmarks and detail the results of a preliminary user-study conducted to evaluate the plausibility of the virtual interactions generated by SPA. Overall, we find that participants prefer SPA to prior techniques in 84% of responses including significant benefits in terms of the plausibility of natural-language interactions and the positive impact of those interactions.

We present a real-time algorithm to infer the intention of a user's avatar in a virtual environment shared with multiple human-like agents. Our algorithm applies the Bayesian Theory of Mind approach to make inferences about the avatar's hidden intentions based on the observed proxemics and gaze-based cues. Our approach accounts for the potential irrationality in human behavior, as well as the dynamic nature of an individual's intentions. The inferred intent is used to guide the response of the virtual agent and generate locomotion and gaze-based behaviors. Our overall approach allows the user to actively interact with tens of virtual agents from a first-person perspective in an immersive setting. We systematically evaluate our inference algorithm in controlled multi-agent simulation environments and highlight its ability to reliably and efficiently infer the hidden intent of a user's avatar even under noisy conditions. We quantitatively demonstrate the performance benefits of our approach in terms of reducing false inferences, as compared to a prior method. The results of our user evaluation show that 68.18% of participants reported feeling more comfortable in sharing the virtual environment with agents simulated with our algorithm as compared to a prior inference method, likely as a direct result of significantly fewer false inferences and more plausible responses from the virtual agents.

We present an interactive algorithm to generate plausible movements for human-like agents interacting with other agents or avatars in a virtual environment. Our approach takes into account high-dimensional human motion constraints and bio-mechanical constraints to compute collision-free trajectories for each agent. We present a novel full-body movement constrained-velocity computation algorithm that can easily be combined with many existing motion synthesis techniques. Compared to prior local navigation methods, our formulation reduces artefacts that arise in dense scenarios and close interactions, and results in smoother and plausible locomotive behaviors. We have evaluated the benefits of our new algorithm in single-agent and multi-agent environments. We investigated the perception of a single agent's movements in dense scenarios and observed that our algorithm has a strong positive effect on the perceived quality of the simulation. Our approach also allows the user to interact with the agents from a first-person perspective in immersive settings. We conducted a study to investigate the perception of such avatar-agent interactions, and found that interactions generated using our approach lead to an increase in the user's sense of co-presence.

We present AutonoVi, a novel algorithm for autonomous vehicle navigation that supports dynamic maneuvers and integrates traffic constraints and norms. Our approach is based on optimization-based maneuver planning that supports dynamic lane-changes, swerving, and braking in all traffic scenarios and guides the vehicle to its goal position. We take into account various traffic constraints, including collision avoidance with other vehicles, pedestrians, and cyclists using control velocity obstacles. We use a data-driven approach to model the vehicle dynamics for control and collision avoidance. Furthermore, our trajectory computation algorithm takes into account traffic rules and behaviors, such as stopping at intersections and stoplights, based on an arc-spline representation. We have evaluated our algorithm in a simulated environment and tested its interactive performance in urban and highway driving scenarios with tens of vehicles, pedestrians, and cyclists. These scenarios include jaywalking pedestrians, sudden stops from high speeds, safely passing cyclists, a vehicle suddenly swerving into the roadway, and high-density traffic where the vehicle must change lanes to progress more effectively.

Abstract Current 3D capture and modeling technology can rapidly generate highly photo‐realistic 3D avatars of human subjects. However, while the avatars look like their human counterparts, their movements often do not mimic their own due to existing challenges in accurate motion capture and retargeting. A better understanding of factors that influence the perception of biological motion would be valuable for creating virtual avatars that capture the essence of their human subjects. To investigate these issues, we captured 22 subjects walking in an open space. We then performed a study where participants were asked to identify their own motion in varying visual representations and scenarios. Similarly, participants were asked to identify the motion of familiar individuals. Unlike prior studies that used captured footage with simple “point‐light” displays, we rendered the motion on photo‐realistic 3D virtual avatars of the subject. We found that self‐recognition was significantly higher for virtual avatars than with point‐light representations. Users were more confident of their responses when identifying their motion presented on their virtual avatar. Recognition rates varied considerably between motion types for recognition of others, but not for self‐recognition. Overall, our results are consistent with previous studies that used recorded footage and offer key insights into the perception of motion rendered on virtual avatars.

We present a novel algorithm for real-time collision-free navigation between elliptical agents. Each robot or agent is represented using a tight-fitting 2D ellipse in the plane. We extend the reciprocal velocity obstacle formulation by using conservative linear approximations of ellipses and derive sufficient conditions for collision-free motion based on low-dimensional linear programming. We use precomputed Minkowski Sum approximations for real-time and conservative collision avoidance in large multi-agent environments. Finally, we present efficient techniques to update the orientation to compute collision-free trajectories. Our algorithm can handle thousands of elliptical agents in real-time on a single core and provides significant speedups over prior algorithms for elliptical agents. We compare the runtime performance and behavior with circular agents on different benchmarks.

We present an adaptive data-driven algorithm for interactive crowd simulation. Our approach combines realistic trajectory behaviors extracted from videos with synthetic multi-agent algorithms to generate plausible simulations. We use statistical techniques to compute the movement patterns and motion dynamics from noisy 2D trajectories extracted from crowd videos. These learned pedestrian dynamic characteristics are used to generate collision-free trajectories of virtual pedestrians in slightly different environments or situations. The overall approach is robust and can generate perceptually realistic crowd movements at interactive rates in dynamic environments. We also present results from preliminary user studies that evaluate the trajectory behaviors generated by our algorithm.

We present a novel interactive approach, PedVR, to generate plausible behaviors for a large number of virtual humans, and to enable natural interaction between the real user and virtual agents. Our formulation is based on a coupled approach that combines a 2D multi-agent navigation algorithm with 3D human motion synthesis. The coupling can result in plausible movement of virtual agents and can generate gazing behaviors, which can considerably increase the believability. We have integrated our formulation with the DK-2 HMD and demonstrate the benefits of our crowd simulation algorithm over prior decoupled approaches. Our user evaluation suggests that the combination of coupled methods and gazing behavior can considerably increase the behavioral plausibility.

We present a multi-agent simulation algorithm to compute the trajectories and full-body motion of human-like agents. Our formulation uses a coupled approach that combines 2D collision-free navigation with high-DOF human motion simulation using a behavioral finite state machine. In order to generate plausible pedestrian motion, we use a closed-loop hierarchical planner that satisfies dynamic stability, biomechanical, and kinematic constraints, and is tightly integrated with multi-agent navigation. Furthermore, we use motion capture data to generate natural looking human motion. The overall system is able to generate plausible motion with upper and lower body movements and avoid collisions with other human-like agents. We highlight its performance in indoor and outdoor scenarios with tens of human-like agents.