This paper presents a system for enabling real-time synthesis of whole-body locomotion and manipulation policies for real-world legged robots. Motivated by recent advancements in robot simulation, we leverage the efficient parallelization capabilities of the MuJoCo simulator on a multi-core CPU to achieve fast sampling over the robot state and action trajectories. Our results show surprisingly effective real-world locomotion and manipulation capabilities with a very simple control strategy. We demonstrate our approach on several hardware and simulation experiments: robust locomotion over flat and uneven terrains, climbing over a box whose height is comparable to the robot, and pushing a box to a goal position. To our knowledge, this is the first successful deployment of whole-body sampling-based MPC on real-world legged robot hardware. Experiment videos and code can be found at: whole-body-mppi.github.io.

Modern non-linear model-based controllers require an accurate physics model and model parameters to be able to control mobile robots at their limits. Also, due to surface slipping at high speeds, the friction parameters may continually change (like tire degradation in autonomous racing), and the controller may need to adapt rapidly. Many works derive a task-specific robot model with a parameter adaptation scheme that works well for the task but requires a lot of effort and tuning for each platform and task. In this work, we design a full model-learning-based controller based on meta pretraining that can very quickly adapt using few-shot dynamics data to any wheel-based robot with any model parameters, while also reasoning about model uncertainty. We demonstrate our results in small-scale numeric simulation, the large-scale Unity simulator, and on a medium-scale hardware platform with a wide range of settings. We show that our results are comparable to domain-specific well-engineered controllers, and have excellent generalization performance across all scenarios.

Ensuring safety via safety filters in real-world robotics presents significant challenges, particularly when the system dynamics is complex or unavailable. To handle this issue, learning-based safety filters recently gained popularity, which can be classified as model-based and model-free methods. Existing model-based approaches requires various assumptions on system model (e.g., control-affine), which limits their application in complex systems, and existing model-free approaches need substantial modifications to standard RL algorithms and lack versatility. This paper proposes a simple, plugin-and-play, and effective model-free safety filter learning framework. We introduce a novel reward formulation and use Q-learning to learn Q-value functions to safeguard arbitrary task specific nominal policies via filtering out their potentially unsafe actions. Due to its model-free nature and simplicity, our framework can be seamlessly integrated with various RL algorithms. We validate the proposed approach through simulations on double integrator and Dubin's car systems and demonstrate its effectiveness in real-world experiments with a soft robotic limb.

Recent works in the robot learning community have successfully introduced generalist models capable of controlling various robot embodiments across a wide range of tasks, such as navigation and locomotion. However, achieving agile control, which pushes the limits of robotic performance, still relies on specialist models that require extensive parameter tuning. To leverage generalist-model adaptability and flexibility while achieving specialist-level agility, we propose AnyCar, a transformer-based generalist dynamics model designed for agile control of various wheeled robots. To collect training data, we unify multiple simulators and leverage different physics backends to simulate vehicles with diverse sizes, scales, and physical properties across various terrains. With robust training and real-world fine-tuning, our model enables precise adaptation to different vehicles, even in the wild and under large state estimation errors. In real-world experiments, AnyCar shows both few-shot and zero-shot generalization across a wide range of vehicles and environments, where our model, combined with a sampling-based MPC, outperforms specialist models by up to 54%. These results represent a key step toward building a foundation model for agile wheeled robot control. AnyCar is fully open-source to support further research.

Varying dynamics pose a fundamental difficulty when deploying safe control laws in the real world. Safety Index Synthesis (SIS) deeply relies on the system dynamics and once the dynamics change, the previously synthesized safety index becomes invalid. In this work, we show the real-time efficacy of Safety Index Adaptation (SIA) in varying dynamics. SIA enables real-time adaptation to the changing dynamics so that the adapted safe control law can still guarantee 1) forward invariance within a safe region and 2) finite time convergence to that safe region. This work employs SIA on a packagecarrying quadruped robot, where the payload weight changes in real-time. SIA updates the safety index when the dynamics change, e.g., a change in payload weight, so that the quadruped can avoid obstacles while achieving its performance objectives. Numerical study provides theoretical guarantees for SIA and a series of hardware experiments demonstrate the effectiveness of SIA in real-world deployment in avoiding obstacles under varying dynamics.

Safety is one of the most crucial challenges of autonomous driving vehicles, and one solution to guarantee safety is to employ an additional control revision module after the planning backbone. Control Barrier Function (CBF) has been widely used because of its strong mathematical foundation on safety. However, the incompatibility with heterogeneous perception data and incomplete consideration of traffic scene elements make existing systems hard to be applied in dynamic and complex real-world scenarios. In this study, we introduce a generalized control revision method for autonomous driving safety, which adopts both vectorized perception and occupancy grid map as inputs and comprehensively models multiple types of traffic scene constraints based on a new proposed barrier function. Traffic elements are integrated into one unified framework, decoupled from specific scenario settings or rules. Experiments on CARLA, SUMO, and OnSite simulator prove that the proposed algorithm could realize safe control revision under complicated scenes, adapting to various planning backbones, road topologies, and risk types. Physical platform validation also verifies the real-world application feasibility.

We present Look-Back and Look-Ahead Adaptive Model Predictive Control (LLA-MPC), a real-time adaptive control framework for autonomous racing that addresses the challenge of rapidly changing tire-surface interactions. Unlike existing approaches requiring substantial data collection or offline training, LLA-MPC employs parallelization over a bank of models for rapid adaptation with no training. It integrates two key mechanisms: a look-back window that uses recent vehicle behavior to optimize the model used in a look-ahead stage for trajectory optimization and control. The optimized model and its associated parameters are then incorporated into an adaptive path planner to optimize reference racing paths in real time. Experiments across diverse racing scenarios demonstrate that LLA-MPC outperforms state-of-the-art methods in adaptation speed and handling, even during sudden friction transitions. Its learning-free, computationally efficient design enables rapid adaptation, making it ideal for high-speed autonomous racing in multi-surface environments.

Reinforcement learning (RL) agents need to explore their environment to learn optimal behaviors and achieve maximum rewards. However, exploration can be risky when training RL directly on real systems, while simulation-based training introduces the tricky issue of the sim-to-real gap. Recent approaches have leveraged safety filters, such as control barrier functions (CBFs), to penalize unsafe actions during RL training. However, the strong safety guarantees of CBFs rely on a precise dynamic model. In practice, uncertainties always exist, including internal disturbances from the errors of dynamics and external disturbances such as wind. In this work, we propose a novel safe RL framework built on a robust CBF, where the discrepancy between the nominal and true dynamic models is quantified through a combination of disturbance observation and residual model learning. We demonstrate our results on the Safety-gym benchmark for Point and Car robots on all tasks where we can outperform state-of-the-art approaches that use only residual model learning or a disturbance observer (DOB). We further validate the efficacy of our framework using a physical F1/10 racing car.Videos: https://sites.google.com/view/res-dob-cbf-rl

Offline reinforcement learning (RL) holds promise as a means to learn high-reward policies from a static dataset, without the need for further environment interactions. However, a key challenge in offline RL lies in effectively stitching portions of suboptimal trajectories from the static dataset while avoiding extrapolation errors arising due to a lack of support in the dataset. Existing approaches use conservative methods that are tricky to tune and struggle with multi-modal data or rely on noisy Monte Carlo return-to-go samples for reward conditioning. In this work, we propose a novel approach that leverages the expressiveness of latent diffusion to model in-support trajectory sequences as compressed latent skills. This facilitates learning a Q-function while avoiding extrapolation error via batch-constraining. The latent space is also expressive and gracefully copes with multi-modal data. We show that the learned temporally-abstract latent space encodes richer task-specific information for offline RL tasks as compared to raw state-actions. This improves credit assignment and facilitates faster reward propagation during Q-learning. Our method demonstrates state-of-the-art performance on the D4RL benchmarks, particularly excelling in long-horizon, sparse-reward tasks.

Autonomous racing is a challenging problem, as the vehicle needs to operate at the friction or handling limits in order to achieve minimum lap times. Autonomous race cars require highly accurate perception, state estimation, planning, and control. Adding to this complexity is the need to accurately identify vehicle model parameters governing lateral tire slip effects, which can evolve over time due to factors such as tire wear and tear. Current approaches to this problem typically either propose offline model identification methods or rely on initial parameters within a narrow range (typically within 15-20% of the actual values). However, these approaches fall short in accounting for significant changes in tire models that can occur during actual races, particularly when pushing the vehicle to its handling limits. We present a unified framework that not only learns the tire model in real time from collected data but also adapts the model to environmental changes, even when the model parameters exhibit substantial deviations. The friction estimation, obtained as a byproduct from the learning results, facilitates the selection of the optimal racing line from a library for adaptive speed planning. We validate our approach through testing in simulators, encompassing a 1:43 scale race car and a full-size car, and also through experiments with a physical F1/10 autonomous race car.

This paper presents a novel approach to modeling human driving behavior, designed for use in evaluating autonomous vehicle control systems in a simulation environments. Our methodology leverages a hierarchical forward-looking, risk-aware estimation framework with learned parameters to generate human-like driving trajectories, accommodating multiple driver levels determined by model parameters. This approach is grounded in multimodal trajectory prediction, using a deep neural network with LSTM-based social pooling to predict the trajectories of surrounding vehicles. These trajectories are used to compute forward-looking risk assessments along the ego vehicle’s path, guiding its navigation. Our method aims to replicate human driving behaviors by learning parameters that emulate human decision-making during driving. We ensure that our model exhibits robust generalization capabilities by conducting simulations, employing real-world driving data to validate the accuracy of our approach in modeling human behavior. The results reveal that our model effectively captures human behavior, showcasing its versatility in modeling human drivers in diverse highway scenarios.

Significant progress has been made in training multimodal trajectory forecasting models for autonomous driving. However, effectively integrating these models with downstream planners and model-based control approaches is still an open problem. Although these models have conventionally been evaluated for open-loop prediction, we show that they can be used to parameterize autoregressive closed-loop models without retraining. We consider recent trajectory prediction approaches which leverage learned anchor embeddings to predict multiple trajectories, finding that these anchor embeddings can parameterize discrete and distinct modes representing high-level driving behaviors. We propose to perform fully reactive closed-loop planning over these discrete latent modes, allowing us to tractably model the causal interactions between agents at each step. We validate our approach on a suite of more dynamic merging scenarios, finding that our approach avoids the frozen robot problem which is pervasive in conventional planners. Our approach also outperforms the previous state-of-the-art in CARLA on challenging dense traffic scenarios when evaluated at realistic speeds.

A critical goal of autonomy and artificial intelligence is enabling autonomous robots to rapidly adapt in dynamic and uncertain environments. Classic adaptive control and safe control provide stability and safety guarantees but are limited to specific system classes. In contrast, policy adaptation based on reinforcement learning (RL) offers versatility and generalizability but presents safety and robustness challenges. We propose SafeDPA, a novel RL and control framework that simultaneously tackles the problems of policy adaptation and safe reinforcement learning. SafeDPA jointly learns adaptive policy and dynamics models in simulation, predicts environment configurations, and fine-tunes dynamics models with few-shot real-world data. A safety filter based on the Control Barrier Function (CBF) on top of the RL policy is introduced to ensure safety during real-world deployment. We provide theoretical safety guarantees of SafeDPA and show the robustness of SafeDPA against learning errors and extra perturbations. Comprehensive experiments on (1) classic control problems (Inverted Pendulum), (2) simulation benchmarks (Safety Gym), and (3) a real-world agile robotics platform (RC Car) demonstrate great superiority of SafeDPA in both safety and task performance, over state-of-the-art baselines. Particularly, SafeDPA demonstrates notable generalizability, achieving a 300% increase in safety rate compared to the baselines, under unseen disturbances in real-world experiments.

This work presents a novel Learning Model Predictive Control (LMPC) strategy for autonomous racing at the handling limit that can iteratively explore and learn unknown dynamics in high-speed operational domains. We start from existing LMPC formulations and modify the system dynamics learning method. In particular, our approach uses a nominal, global, nonlinear, physics-based model with a local, linear, data-driven learning of the error dynamics. We conducted experiments in simulation and on 1/10th scale hardware, and deployed the proposed LMPC on a full-scale autonomous race car used in the Indy Autonomous Challenge (IAC) with closed loop experiments at the Putnam Park Road Course in Indiana, USA. The results show that the proposed control policy exhibits improved robustness to parameter tuning and data scarcity. Incremental and safety-aware exploration toward the limit of handling and iterative learning of the vehicle dynamics in high-speed domains is observed both in simulations and experiments.

Cooperative planning of Connected and Autonomous vehicles (CAVs) is a promising way to reshape the intelligent transportation system, and planning under scenarios mixed with human-driven vehicles is one of the critical challenges. Many existing studies have proposed vehicle group planning methods towards mixed traffic scenes. By regarding the reward of all CAVs as a unified entity, the overall average driving performance was improved. However, one limitation is that during the collective planning process, individual demands are sacrificed for lack of considering agent personalized preferences. To balance the group common benefit and individual diversity, this paper proposes a novel cooperative multi-vehicle planning method combining collective decision-making and individual preference evaluation. First, a bi-level collective planning framework is designed including region-driven behavior selection and conflict-free trajectory generation. To further consider personalized features, an individual preference evaluation system is established based on Social Value Orientation. Then the evaluation results are merged into the group planning process so that vehicles can generate various decisions according to personal demands. Experimental results show that the driving personalization levels of safety, time efficiency, and comfort are increased by 1.28%, 3.78%, and 8.80%, correspondingly.

Decision-making is one of the critical modules of autonomous driving, and how to evaluate its performance precisely remains a challenge. Although some current evaluation methods have considered multiple aspects of driving experience and been able to give a comprehensive evaluation result, the emphasis on individual personalization diversity is still limited. Existing methods mainly fit the evaluation model to an average human model, neglecting the individual demand features. In this paper, we propose a novel integrated decision-making evaluation method considering individual personalization diversity. First, we build an integrated evaluation model to represent the average human evaluation. Based on the four fundamental single-factor model including safety, time efficiency, comfort, and energy consumption, a segmental linear model is applied to combine various elements into one unified framework. Then a personalized weight fluctuation mechanism is proposed which adjusts the relative term weights in the integrated model dynamically according to the users’ preferences. Finally, a corresponding online individual demand distribution estimation method is designed to assess human personal diversity. The experiments on the D2E dataset prove that the proposed evaluation system can better adapt to individual preference diversity, reducing the evaluation mean absolute error by 7.40%. The higher the degree of personalization for the users, the greater the improvement this method can generate.

Delays endanger the safety of autonomous systems functioning in the rapidly changing environments of autonomous driving and high-speed racing. Unfortunately, the consideration of delays is often overlooked during controller design or learning-enabled controller training phases prior to deployment in the physical world. This paper systematically and comprehensively addresses both the computation delay arising from nonlinear optimization for control and other inevitable delays caused by actuators. First, we propose a new filtering approach to adaptively estimate the time-variant computation delay. Second, we model actuation dynamics for steering delay. Third, all the constrained optimization is realized in a robust tube model predictive controller. In terms of application merits, our approach is a novel design for a standalone delay-aware controller; in addition, our approach can also serve as a delay compensator for an existing controller. Video (https://youtu.be/nURl_HTW_Mo) and code (https://github.com/dvij542/Delay-aware-Robust-Tube-MPC) are available.

In this work, we present a novel risk-aware decentralized Control Barrier Function (CBF)-based controller for multi-agent systems. The proposed decentralized controller is composed based on pairwise agent responsibility shares (a percentage), calculated from the risk evaluation of each individual agent faces in a multi-agent interaction environment. With our proposed CBF-inspired risk evaluation framework, the responsibility portions between pairwise agents are dynamically updated based on the relative risk they face. Our method allows agents with lower risk to enjoy a higher level of freedom in terms of a wider action space, and the agents exposed to higher risk are constrained more tightly on action spaces, and are therefore forced to proceed with caution.

Multi-agent Reinforcement Learning (MARL) has been increasingly used in safety-critical applications but has no safety guarantees, especially during training. In this paper, we propose dynamic shielding, a novel decentralized MARL framework to ensure safety in both training and deployment phases. Our framework leverages Shield, a reactive system running in parallel with the reinforcement learning algorithm to monitor and correct agents' behavior. In our algorithm, shields dynamically split and merge according to the environment state in order to maintain decentralization and avoid conservative behaviors while enjoying formal safety guarantees. We demonstrate the effectiveness of MARL with dynamic shielding in the mobile navigation scenario.

Prior work has looked at applying reinforcement learning (RL) approaches to autonomous driving scenarios, but the safety of the algorithm is often compromised due to instability or the presence of ill-defined reward functions. With the use of control barrier functions embedded into the RL policy, we arrive at safe policies to optimize the performance of the autonomous driving vehicle through the advantage of a safety layer over the RL methods to ease the design of reward functions. However, control barrier functions need a good approximation of the model of the system. We use probabilistic control barrier functions [4] to account for model uncertainty. Our Safety-Assured Policy Optimization - Ramp Merging (SAPO-RM) algorithm is implemented online in the CARLA [1] Simulator and offline on the US I-80 dataset extracted from the NGSIM Database provided by NHTSA [2]. We further test the algorithm and perform ablation studies of it on the US-101 and exi-D datasets to compare the approaches. The proposed algorithm can also be applied to other driving scenarios by changing the reward and safety constraints.

Autonomous agents (robots) face tremendous challenges while interacting with heterogeneous human agents in close proximity. One of these challenges is that the autonomous agent does not have an accurate model tailored to the specific human that the autonomous agent is interacting with, which could sometimes result in inefficient human-robot interaction and suboptimal system dynamics. Developing an online method to enable the autonomous agent to learn information about the human model is therefore an ongoing research goal. Existing approaches position the robot as a passive learner in the environment to observe the physical states and the associated human response. This passive design, however, only allows the robot to obtain information that the human chooses to exhibit, which sometimes doesn't capture the human's full intention. In this work, we present an online optimization-based probing procedure for the autonomous agent to clarify its belief about the human model in an active manner. By optimizing an information radius, the autonomous agent chooses the action that most challenges its current conviction. This procedure allows the autonomous agent to actively probe the human agents to reveal information that's previously unavailable to the autonomous agent. With this gathered information, the autonomous agent can interactively influence the human agent for some designated objectives. Our main contributions include a coherent theoretical framework that unifies the probing and influence procedures and two case studies in autonomous driving that show how active probing can help to create better participant experience during influence, like higher efficiency or less perturbations.

Decentralized control schemes are increasingly favored in various domains that involve multi-agent systems due to the need for computational efficiency as well as general applicability to large-scale systems. However, in the absence of an explicit global coordinator, it is hard for distributed agents to determine how to efficiently interact with others. In this paper, we present a risk-aware decentralized control framework that provides guidance on how much relative responsibility share (a percentage) an individual agent should take to avoid collisions with others while moving efficiently without direct communications. We propose a novel Control Barrier Function (CBF)-inspired risk measurement to characterize the aggregate risk agents face from potential collisions under motion uncertainty. We use this measurement to allocate responsibility shares among agents dynamically and develop risk-aware decentralized safe controllers. In this way, we are able to leverage the flexibility of robots with lower risk to improve the motion flexibility for those with higher risk, thus achieving improved collective safety. We demonstrate the validity and efficiency of our proposed approach through two examples: ramp merging in autonomous driving and a multi-agent position-swapping game.

Detecting when road maps change is useful for autonomous vehicles to drive safely and legally, for city planners to make more educated decisions, and web maps to better serve consumers. Many public vehicles drive around the city on a regular basis and collect road data for security and safety purposes through dash cams, yet few cities and companies have considered this as a data source for city monitoring. We present an automatic method and system for crosswalk change detection at city intersections using a monocular camera on a city bus and analyze longitudinal results over the course of a year. Using images recorded by a bus two years ago as reference, multiple city intersections are reconstructed, fitted for ground planes, and labelled for crosswalks. Subsequent images from the bus are imported and processed to detect if changes have occurred since intersections were first seen by first localizing current images with respect to the reference images, detecting for crosswalks, and computing detection overlaps in the bird’s-eye-view. Our method makes improvements upon baseline methods by checking for crosswalk visibility and localization errors, is able to generate results typically seen by using more expensive LiDAR sensors, and has been successfully deployed live for one month.

Control Barrier Functions have been extensively studied to ensure guaranteed safety during inter-robot interactions. In this paper, we introduce the Parametric-Control Barrier Function (Parametric-CBF), a novel variant of the traditional Control Barrier Function to extend its expressivity in describing different safe behaviors among heterogeneous robots. A parametric-CBF based framework is presented to enable the ego robot to model the neighboring robots behavior and further improve the coordination efficiency during interaction while enjoying formally provable safety guarantees. We demonstrate the usage of Parametric-CBF in behavior prediction and adaptive safe control in the ramp merging scenario.

Impressive results in natural language processing (NLP) based on the Transformer neural network architecture have inspired researchers to explore viewing offline reinforcement learning (RL) as a generic sequence modeling problem. Recent works based on this paradigm have achieved state-of-the-art results in several of the mostly deterministic offline Atari and D4RL benchmarks. However, because these methods jointly model the states and actions as a single sequencing problem, they struggle to disentangle the effects of the policy and world dynamics on the return. Thus, in adversarial or stochastic environments, these methods lead to overly optimistic behavior that can be dangerous in safety-critical systems like autonomous driving. In this work, we propose a method that addresses this optimism bias by explicitly disentangling the policy and world models, which allows us at test time to search for policies that are robust to multiple possible futures in the environment. We demonstrate our method’s superior performance on a variety of autonomous driving tasks in simulation.

In this paper, we present a novel convex optimization approach to address the minimum-time speed planning problem over a fixed path with dynamic obstacle constraints and point-wise speed and acceleration constraints. The contributions of this paper are three-fold. First, we formulate the speed planning as an iterative convex optimization problem based on space discretization. Our formulation allows imposing dynamic obstacle constraints and point-wise speed and acceleration constraints simultaneously. Second, we propose a modified vertical cell decomposition method to handle dynamic obstacles. It divides the freespace into channels, where each channel represents a homotopy of free paths and defines convex constraints for dynamic obstacles. Third, we demonstrate significant improvement over previous work on speed planning for typical driving scenarios such as following, merging, and crossing.

Traversing intersections is a challenging problem for autonomous vehicles, especially when the intersections do not have traffic control. Recently deep reinforcement learning has received massive attention due to its success in dealing with autonomous driving tasks. In this work, we address the problem of traversing unsignalized intersections using a novel curriculum for deep reinforcement learning. The proposed curriculum leads to: 1) A faster training process for the reinforcement learning agent, and 2) Better performance compared to an agent trained without curriculum. Our main contribution is two-fold: 1) Presenting a unique curriculum for training deep reinforcement learning agents, and 2) demonstrating the performance improvement using the proposed curriculum in the unsignalized intersection traversal task. The framework expects processed observations of the surroundings from the perception system of the autonomous vehicle. We test our method in the CommonRoad motion planning simulator on T-intersections and four-way intersections.

To efficiently generate safe trajectories for an autonomous vehicle in dynamic environments, a layered motion planning method with decoupled path and speed planning is widely used. This paper studies speed planning, which mainly deals with dynamic obstacle avoidance given a planned path. The main challenges lie in the optimization in a non-convex space and the trade-off between safety, comfort, and efficiency. First, this work proposes to conduct a search in second-order derivative space for generating a comfort-optimal reference trajectory. Second, by combining abstraction and refinement, an algorithm is proposed to construct a convex feasible space for optimization. Finally, a piecewise Bézier polynomial optimization approach with trapezoidal corridors is presented, which theoretically guarantees safety and significantly enlarges the solution space compared with the existing rectangular corridors-based approach. We validate the efficiency and effectiveness of the proposed approach in simulations.

A control barrier functions-based quadratic programming (CBF-QP) method has emerged as a controller synthesis tool to assure safety of autonomous systems owing to the appealing safe forward invariant set. However, the provable safety relies on a precisely described dynamic model, which is not always available in practice. Recent works leverage learning to compensate model uncertainty for a CBF controller. However, these approaches based on reinforcement learning or episodic learning are limited to dealing with time-invariant uncertainty. Also, the reinforcement learning approach learns the uncertainty offline, while episodic learning only updates the controller after a batch of data is available by the end of an episode. Instead, we propose a novel tuning extreme learning machine (tELM)-based CBF controller that can compensate time-variant and time-invariant model uncertainty adaptively in an online manner. We validate our approach's effectiveness in a simulation of an Adaptive Cruise Control (ACC) system.

With the advancement of affordable self-driving vehicles using complicated nonlinear optimization but limited computation resources, computation time becomes a matter of concern. Other factors such as actuator dynamics and actuator command processing cost also unavoidably cause delays. In high-speed scenarios, these delays are critical to the safety of a vehicle. Recent works consider these delays individually, but none unifies them all in the context of autonomous driving. Moreover, recent works inappropriately consider computation time as a constant or a large upper bound, which makes the control either less responsive or over-conservative. To deal with all these delays, we present a unified framework by 1) modeling actuation dynamics, 2) using robust tube model predictive control, and 3) using a novel adaptive Kalman filter without assuming a known process model and noise covariance, which makes the controller safe while minimizing conservativeness. On the one hand, our approach can serve as a standalone controller; on the other hand, our approach provides a safety guard for a high-level controller, which assumes no delay. This can be used for compensating the sim-to-real gap when deploying a black-box learning-enabled controller trained in a simplistic environment without considering delays for practical vehicle systems.

With the increasing need for safe control in the domain of autonomous driving, model-based safety-critical control approaches are widely used, especially Control Barrier Function (CBF)-based approaches. Among them, Exponential CBF (eCBF) is particularly popular due to its realistic applicability to high-relative-degree systems. However, for most of the optimization-based controllers utilizing CBF-based constraints, solution feasibility is a common issue arising from potential conflict among different constraints. Moreover, how to incorporate uncertainty into the eCBF-based constraints in high-relative-degree systems to account for safety remains an open challenge. In this paper, we present a novel approach to extend an eCBF-based safe critical controller to a probabilistic setting to handle potential motion uncertainty from system dynamics. More importantly, we leverage an optimization-based technique to provide a solution feasibility guarantee in run time, while ensuring probabilistic safety. Lane changing and intersection handling are demonstrated as two use cases, and experiment results are provided to show the effectiveness of the proposed approach.

With the increasing emphasis on the safe autonomy for robots, model-based safe control approaches such as Control Barrier Functions have been extensively studied to ensure guaranteed safety during inter-robot interactions. In this paper, we introduce the Parametric Control Barrier Function (Parametric-CBF), a novel variant of the traditional Control Barrier Function to extend its expressivity in describing different safe behaviors among heterogeneous robots. Instead of assuming cooperative and homogeneous robots using the same safe controllers, the ego robot is able to model the neighboring robots’ underlying safe controllers through different Parametric-CBFs with observed data. Given learned parametric-CBF and proved forward invariance, it provides greater flexibility for the ego robot to better coordinate with other heterogeneous robots with improved efficiency while enjoying formally provable safety guarantees. We demonstrate the usage of Parametric-CBF in behavior prediction and adaptive safe control in the ramp merging scenario from the applications of autonomous driving. Compared to traditional CBF, Parametric-CBF has the advantage of capturing varying drivers’ characteristics given richer description of robot behavior in the context of safe control. Numerical simulations are given to validate the effectiveness of the proposed method.

Fish-eye cameras have become essential sensors in intelligent vehicles. Due to its unique projection principle, a fish-eye camera can provide a large field of view. Benefiting from this special feature, fish-eye cameras have rich applications in intelligent vehicles. However, dataset and distortion problems are still challenges when applying fish-eye cameras in reality. This work introduces the projection principle of fish-eye cameras, and four classic fish-eye image representation models are presented. Then, the typical fish-eye datasets are presented, including real collected data and virtually generated data. Through the organization and summarization of the relevant studies, we demonstrate various applications of fish-eye cameras in intelligent vehicles, e.g., object detection and tracking, image segmentation, mapping and localization, and around-view monitoring. These works design various strategies to exploit the advantages of fish-eye cameras and prevent image distortion problems, showing the broad application prospects of such cameras. Finally, we discuss the development tendencies of intelligent vehicle applications involving fish-eye cameras.

Safe local motion planning for autonomous driving in dynamic environments requires forecasting how the scene evolves. Practical autonomy stacks adopt a semantic object-centric representation of a dynamic scene and build object detection, tracking, and prediction modules to solve forecasting. However, training these modules comes at an enormous human cost of manually annotated objects across frames. In this work, we explore future freespace as an alternative representation to support motion planning. Our key intuition is that it is important to avoid straying into occupied space regardless of what is occupying it. Importantly, computing ground-truth future freespace is annotation-free. First, we explore freespace forecasting as a self-supervised learning task. We then demonstrate how to use forecasted freespace to identify collision-prone plans from off-the-shelf motion planners. Finally, we propose future freespace as an additional source of annotation-free supervision. We demonstrate how to integrate such supervision into the learning-based planners. Experimental results on nuScenes and CARLA suggest both approaches lead to a significant reduction in collision rates.1

Comprehensive depth information from surrounding scenes is important for perception in autonomous driving and robots. Sparse LIDAR sensors give a low-density point cloud of the environment, but are more affordable than their high-density counterparts. In this paper, we propose a novel sensor fusion architecture for sparse LIDAR depth completion. Instead of the traditional end-to-end neural network-based algorithm, we formulate depth completion as a Linear Inverse Problem (LIP) with a multi-modal proximal operator. This sensor fusion architecture allows a better signal prior and finds the unique optimal solution to the LIP. Instead of learning a unified network for the sparse input which treats pixels evenly, the proposed architecture guarantees both the data consistency and smoothness of the predicted depth map. To demonstrate the performance of our algorithm, we benchmark on the simulation dataset TartanAir, and the real indoor NYUdepthv2 and real outdoor KITTI datasets. Our proposed method outperforms previous methods and uses fewer parameters in both indoor and outdoor datasets.

Recently, autonomous driving has made substantial progress in addressing the most common traffic scenarios like intersection navigation and lane changing. However, most of these successes have been limited to scenarios with well-defined traffic rules and require minimal negotiation with other vehicles. In this paper, we introduce a previously unconsidered, yet everyday, high-conflict driving scenario requiring negotiations between agents of equal rights and priorities. There exists no centralized control structure and we do not allow communications. Therefore, it is unknown if other drivers are willing to cooperate, and if so to what extent. We train policies to robustly negotiate with opposing vehicles of an unobservable degree of cooperativeness using multi-agent reinforcement learning (MARL). We propose Discrete Asymmetric Soft Actor-Critic (DASAC), a maximum- entropy off-policy MARL algorithm allowing for centralized training with decentralized execution. We show that using DASAC we are able to successfully negotiate and traverse the scenario considered over 99% of the time. Our agents are robust to an unknown timing of opponent decisions, an unobservable degree of cooperativeness of the opposing vehicle, and previously unencountered policies. Furthermore, they learn to exhibit human-like behaviors such as defensive driving, anticipating solution options and interpreting the behavior of other agents.

Autonomous vehicles face tremendous challenges while interacting with human drivers in different kinds of scenarios. Developing control methods with safety guarantees while performing interactions with uncertainty is an ongoing research goal. In this paper, we present a real-time safe control framework using bi-level optimization with Control Barrier Function (CBF) that enables an autonomous ego vehicle to interact with human-driven cars in ramp merging scenarios with a consistent safety guarantee. In order to explicitly address motion uncertainty, we propose a novel extension of control barrier functions to a probabilistic setting with provable chance-constrained safety and analyze the feasibility of our control design. The formulated bi-level optimization framework entails first choosing the ego vehicle's optimal driving style in terms of safety and primary objective, and then minimally modifying a nominal controller in the context of quadratic programming subject to the probabilistic safety constraints. This allows for adaptation to different driving strategies with a formally provable feasibility guarantee for the ego vehicle's safe controller. Experimental results are provided to demonstrate the effectiveness of our proposed approach.

For autonomous vehicles, effective behavior planning is crucial to ensure safety of the ego car. In many urban scenarios, it is hard to create sufficiently general heuristic rules, especially for challenging scenarios that some new human drivers find difficult. In this work, we propose a behavior planning structure based on reinforcement learning (RL) which is capable of performing autonomous vehicle behavior planning with a hierarchical structure in simulated urban environments. Application of the hierarchical structure [1] allows the various layers of the behavior planning system to be satisfied. Our algorithms can perform better than heuristic-rule-based methods for elective decisions such as when to turn left between vehicles approaching from the opposite direction or possible lane-change when approaching an intersection due to lane blockage or delay in front of the ego car. Such behavior is hard to evaluate as correct or incorrect, but some aggressive expert human drivers handle such scenarios effectively and quickly. On the other hand, compared to traditional RL methods, our algorithm is more sample-efficient, due to the use of a hybrid reward mechanism and heuristic exploration during the training process. The results also show that the proposed method converges to an optimal policy faster than traditional RL methods.

Trajectory planning in dynamic environments can be decomposed into two sub-problems: 1) planning a path to avoid static obstacles, 2) then planning a speed profile to avoid dynamic obstacles. This is also called path-speed decomposition. In this work, we present a novel approach to solve the first sub-problem, motion planning with static obstacles. From an optimization perspective, motion planning for autonomous vehicles can be viewed as non-convex constrained nonlinear optimization, which requires a good enough initial guess to start and is often sensitive to algorithm parameters. We formulate motion planning as convex spline optimization. The convexity of the formulated problem makes it able to be solved fast and reliably, while guaranteeing a global optimum. We then reorganize the constrained spline optimization into a recurrent formulation, which further reduces the computational time to be linear in the optimization horizon size. The proposed method can be applied to both trajectory generation and motion planning problems. Its effectiveness is demonstrated in challenging scenarios such as tight lane changes and sharp turns.

Ego-motion estimation is a key requirement for the simultaneous localization and mapping (SLAM) problem. The traditional pipeline goes through feature extraction, feature matching and pose estimation, whose performance depends on the manually designed features. In this paper, we are motivated by the strong performance of deep learning methods in other computer vision and robotics tasks. We replace hand-crafted features with a neural network and directly estimate the relative pose between two adjacent scans from a LiDAR sensor using ENCODE: a dEep poiNt Cloud ODometry nEtwork. Firstly, a spherical projection of the input point cloud is performed to acquire a multi-channel vertex map. Then a multi-layer network backbone is applied to learn the abstracted features and a fully connected layer is adopted to estimate the 6-DoF ego-motion. Additionally, a map-to-map optimization module is applied to update the local poses and output a smooth map. Experiments on multiple datasets demonstrate that the proposed method achieves the best performance in comparison to state-of-the-art methods and is capable of providing accurate poses with low drift in various kinds of scenarios.

This work proposes a novel framework for motion planning using trajectory optimization for autonomous driving. First, a two-phase behavioral policy maker (BPM) is proposed as a high-level decision maker to mimic human-like driving style by avoiding unnecessary tasks and early lane changes. Second, a comprehensive study on iterative adaptive weight tuning functions has been done to limit manual weight tuning in the Constrained Iterative Linear Quadratic Regulator (CILQR) motion planner. Third, a jerk-minimized CILQR is presented to ensure the comfort and safety of passengers by generating smooth trajectories. The simulation results show efficiency, safety, and comfort of generated trajectories.

Interactive behavior is important for autonomous driving vehicles, especially for scenarios like ramp merging which require significant social interaction between autonomous driving vehicles and human-driven cars. This paper enhances our previous Probabilistic Graphical Model (PGM) merging control model for the interactive behavior of autonomous driving vehicles. To better estimate the interactive behavior for autonomous driving cars, a Factor Graph (FG) is used to describe the dependency among observations and estimate other cars’ intentions. Real trajectories are used to approximate the model instead of human-designed models or cost functions. Forgetting factors and a Gaussian Mixture Model (GMM) are also applied in the intention estimation process for stabilization, interpolation and smoothness. The advantage of the factor graph is that the relationship between its nodes can be described by self-defined functions, instead of probabilistic relationships as in PGM, giving more flexibility. Continuity of GMM also provides higher accuracy than the previous discrete speed transition model. The proposed method enhances the overall performance of intention estimation, in terms of collision rate and average distance between cars after merging, which means it is safer and more efficient.

How autonomous vehicles and human drivers share public transportation systems is an important problem, as fully automatic transportation environments are still a long way off. Understanding human drivers’ behavior can be beneficial for autonomous vehicle decision making and planning, especially when the autonomous vehicle is surrounded by human drivers who have various driving behaviors and patterns of interaction with other vehicles. In this paper, we propose an LSTM-based trajectory prediction method for human drivers which can help the autonomous vehicle make better decisions, especially in urban intersection scenarios. Meanwhile, in order to collect human drivers’ driving behavior data in the urban scenario, we describe a system called UrbanFlow which includes the whole procedure from raw bird’s-eye view data collection via drone to the final processed trajectories. The system is mainly intended for urban scenarios but can be extended to be used for any traffic scenarios.

Modern high-definition LIDAR is expensive for commercial autonomous driving vehicles and small indoor robots. An affordable solution to this problem is fusion of planar LIDAR with RGB images to provide a similar level of perception capability. Even though state-of-the-art methods provide approaches to predict depth information from limited sensor input, they are usually a simple concatenation of sparse LIDAR features and dense RGB features through an end-to-end fusion architecture. In this paper, we introduce an inductive late-fusion block which better fuses different sensor modalities inspired by a probability model. The proposed demonstration and aggregation network propagates the mixed context and depth features to the prediction network and serves as a prior knowledge of the depth completion. This late-fusion block uses the dense context features to guide the depth prediction based on demonstrations by sparse depth features. In addition to evaluating the proposed method on benchmark depth completion datasets including NYUDepthV2 and KITTI, we also test the proposed method on a simulated planar LIDAR dataset. Our method shows promising results compared to previous approaches on both the benchmark datasets and simulated dataset with various 3D densities.

Learning-enabled components have been widely deployed in autonomous systems. However, due to the weak interpretability and the prohibitively high complexity of large-scale machine learning models such as neural networks, reliability has been a crucial concern for safety-critical autonomous systems. This work proposes an online monitor called Reach-Flow for fault prevention of waypoint-following tasks for self-driving cars. It mainly consists of two components: (a) an online verification tool which conservatively checks the safety of the system behavior in the near future, and (b) a fallback controller which steers the system back to a desired state when the system is potentially unsafe. We implement ReachFlow in a self-driving racing car governed by a reinforcement learning-based controller. We demonstrate the effectiveness by rigorously verifying a safe waypoint-following control and providing a fallback control for an unsafe situation in which a large deviation from the planned path is predicted.

Constrained Iterative Linear Quadratic Regulator (CILQR), a variant of ILQR, has been recently proposed for motion planning problems of autonomous vehicles to deal with constraints such as obstacle avoidance and reference tracking. However, the previous work considers either deterministic trajectories or persistent prediction for target dynamical obstacles. The other drawback is lack of generality - it requires manual weight tuning for different scenarios. In this paper, two significant improvements are achieved. Firstly, a two-stage uncertainty-aware prediction is proposed. The short-term prediction with safety guarantee based on reachability analysis is responsible for dealing with extreme maneuvers conducted by target vehicles. The long-term prediction leveraging an adaptive least square filter preserves the long-term optimality of the planned trajectory since using reachability only for long-term prediction is too pessimistic and makes the planner over-conservative. Secondly, to allow a wider coverage over different scenarios and to avoid tedious parameter tuning case by case, this paper designs a scenario-based analytical function taking the states from the ego vehicle and the target vehicle as input, and carrying weights of a cost function as output. It allows the ego vehicle to execute multiple behaviors (such as lane-keeping and overtaking) under a single planner. We demonstrate safety, effectiveness, and real-time performance of the proposed planner in simulations.

Behavioral decision making is an important aspect of autonomous vehicles (AV). In this work, we propose a behavior planning structure based on hierarchical reinforcement learning (HRL) which is capable of performing autonomous vehicle planning tasks in simulated environments with multiple sub-goals. In this hierarchical structure, the network is capable of 1) learning one task with multiple sub-goals simultaneously; 2) extracting attentions of states according to changing sub-goals during the learning process; 3) reusing the well-trained network of sub-goals for other tasks with the same sub-goals. A hybrid reward mechanism is designed for different hierarchical layers in the proposed HRL structure. Compared to traditional RL methods, our algorithm is more sample-efficient, since its modular design allows reusing the policies of sub-goals across similar tasks for various transportation scenarios. The results show that the proposed method converges to an optimal policy faster than traditional RL methods.

Imitation learning provides a way to automatically construct a controller by mimicking human behavior from data. For safety-critical systems such as autonomous vehicles, it can be problematic to use controllers learned from data because they cannot be guaranteed to be collision-free. Recently, a method has been proposed for learning a multi-mode hybrid automaton cruise controller (MOHA). Besides being accurate, the logical nature of this model makes it suitable for formal verification. In this paper, we demonstrate this capability using the SpaceEx hybrid model checker as follows. We develop an automated tool to translate the automaton model into constraints and equations required by SpaceEx. We then verify that a pure MOHA controller is not collision-free. By adding a safety state based on headway in time, a rule that human drivers should follow anyway, we do obtain a provably safe cruise control. Moreover, the safe controller remains more humanlike than existing cruise controllers.

Several key scenarios, such as intersection navigation, lane changing, and ramp merging, are active areas of research in autonomous driving. In order to properly navigate these scenarios, autonomous vehicles must implicitly negotiate with human drivers. Prior work in driving behaviors presents reinforcement learning as a promising technique, as it can leverage data as well as the underlying decision-making structure of driving with interaction. We apply a hierarchical approach to decision-making, where we train a high-level policy using reinforcement learning, and execute the policy's output on a low-level controller. This hierarchical structure helps increase the policy's overall safety, and allows the learning component to be agnostic to the low-level control scheme. We validate our approach on a simulation using real-world highway data and find improved results compared to prior work in ramp merging.

Perception is an essential task for self-driving cars, but most perception tasks are usually handled independently. We propose a unified neural network named DLT-Net to detect drivable areas, lane lines, and traffic objects simultaneously. These three tasks are most important for autonomous driving, especially when a high-definition map and accurate localization are unavailable. Instead of separating tasks in the decoder, we construct context tensors between sub-task decoders to share designate influence among tasks. Therefore, each task can benefit from others during multi-task learning. Experiments show that our model outperforms the conventional multi-task network in terms of the task-wise accuracy and the overall computational efficiency, in the challenging BDD dataset.

Interactive driving is challenging but essential for autonomous cars in dense traffic or urban areas. Proper interaction requires understanding and prediction of future trajectories of all neighboring cars around a target vehicle. Current solutions typically assume a certain distribution or stochastic process to approximate human-driven cars' behaviors. To relax this assumption, a Recurrent Meta Induction Network (RMIN) framework is developed. The original Conditional Neural Process (CNP) on which this is based does not consider the sequence of the conditions, due to the permutation invariance requirements for stochastic processes. However, the sequential information is important for the driving behavior estimation. Therefore, in the proposed method, a recurrent neural cell replaces the original demonstration sub-net. The behavior estimation is conditioned on the historical observations for all related cars, including the target car and its surrounding cars. The method is applied to predict the lane change trajectory of a target car in dense traffic areas. The proposed method achieves better results than previous methods and thanks to the meta-learning framework, it can use a smaller dataset, putting fewer demands on autonomous driving data collection.

Performing safe and efficient lane changes is a crucial feature for creating fully autonomous vehicles. Recent advances have demonstrated successful lane following behavior using deep reinforcement learning, yet the interactions with other vehicles on-road for lane changes are rarely considered. In this paper, we design a hierarchical Deep Reinforcement Learning (DRL) algorithm to learn lane change behaviors in dense traffic. By breaking down overall behavior to sub-policies, faster and safer lane change actions can be learned. We also apply temporal and spatial attention to the DRL architecture, which helps the vehicle focus more on surrounding vehicles and leads to smoother lane change behavior. We conduct our experiments in the TORCS simulator and the results outperform the state-of-the-art deep reinforcement learning algorithm in various lane change scenarios.

A safe and robust on-road navigation system is a crucial component of achieving fully automated vehicles. NVIDIA recently proposed an End-to-End algorithm that can directly learn steering commands from raw pixels of a front camera by using one convolutional neural network. In this paper, we leverage auxiliary information aside from raw images and design a novel network structure, called Auxiliary Task Network (ATN), to help boost the driving performance while maintaining the advantage of minimal training data and an End-to-End training method. In this network, we introduce human prior knowledge into vehicle navigation by transferring features from image recognition tasks. Image semantic segmentation is applied as an auxiliary task for navigation. We consider temporal information by introducing an LSTM module and optical flow to the network. Finally, we combine vehicle kinematics with a sensor fusion step. We discuss the benefits of our method over state-of-the-art visual navigation methods both in the Udacity simulation environment and on the real-world Comma.ai dataset.

Cooperative driving behavior is essential for driving in traffic, especially for ramp merging, lane changing or navigating intersections. Autonomous vehicles should also man- age these situations by behaving cooperatively and naturally. In this paper, we enhance our previous learning-based method to efficiently estimate other vehicles' intentions and interact with them in ramp merging scenarios, without over-the-air communication between vehicles. The proposed approach inherits our previous Probabilistic grahpical Model (PGM) and distance- keeping framework. Real driving trajectories are used to learn transition models in the PGM. Thus, besides the structure of the PGM, our method does not require human-designed reward or cost functions. The PGM-based intention estimation is followed by an off-the-shelf distance-keeping model to generate proper acceleration/deceleration controls. The PGM plays a plug-in role in our self-driving framework. The new model eliminates two assumptions in the previous model: 1) a fixed merging point for all merging agents, which is hard to determine before the merging vehicles make the merge; 2) Perfect velocity measurement, which requires sophisticated perception systems. We validate the performance of our method both on real merging data and using a designed merging strategy in simulation, and show significant improvements compared with previous methods. Parameter design is also discussed by experiments. The new method is computationally efficient, and exhibits better robustness against sensing uncertainty.

We address the problem of learning autonomous driving behaviors in urban intersections using deep reinforcement learning (DRL). DRL has become a popular choice for creating autonomous agents due to its success in various tasks. However, as the problems tackled become more complex, the number of training iterations necessary increase drastically. Curriculum learning has been shown to reduce the required training time and improve the performance of the agent, but creating an optimal curriculum often requires human handcrafting. In this work, we learn a policy for urban intersection crossing using DRL and introduce a method to automatically generate the curriculum for the training process from a candidate set of tasks. We compare the performance of the automatically generated curriculum (AGC) training to those of randomly generated sequences and show that AGC can significantly reduce the training time while achieving similar or better performance.

Predicting lane-changing intentions has long been a very active area of research in the autonomous driving community. However, most of the literature has focused on individual vehicles and did not consider both the neighbor information and the accumulated effects of vehicle history trajectories when making the predictions. We propose to apply a surrounding-aware LSTM algorithm for predicting the intention of a vehicle to perform a lane change that takes advantage of both vehicle past trajectories and their neighbor's current states. We trained the model on real-world lane changing data and were able to show in simulation that these two components can lead not only to higher accuracy, but also to earlier lane-changing prediction time, which plays an important role in potentially improving the autonomous vehicle's overall performance.

The effective detection of curbs is fundamental and crucial for the navigation of a self-driving car. This paper presents a real-time curb detection method that automatically segments the road and detects its curbs using a 3D-LiDAR sensor. The point cloud data of the sensor are first processed to distinguish on-road and off-road areas. A sliding-beam method is then proposed to segment the road by using the off-road data. A curb-detection method is finally applied to obtain the position of curbs for each road segments. The proposed method is tested on the data sets acquired from the self-driving car of laboratory of VeCaN at Tongji University. Off-line experiments demonstrate the accuracy and robustness of the proposed method, i.e., the average recall, precision and their harmonic mean are all over 80%. Online experiments demonstrate the real-time capability for autonomous driving as the average processing time for each frame is only around 12 ms.

Nowadays, self-driving cars are being applied to more complex urban scenarios including intersections, merging ramps or lane changes. It is, therefore, important for self-driving cars to behave socially with human-driven cars. In this paper, we focus on generating the lane change behavior for self-driving cars: perform a safe and effective lane change behavior once a lane-change command is received. Our method bridges the gap between higher-level behavior commands and the trajectory planner. There are two challenges in the task: 1) Analyzing the surrounding vehicles' mutual effects from their trajectories. 2) Estimating the proper lane change start point and end point according to the analysis of surrounding vehicles. We propose a learning-based approach to understand surrounding traffic and make decisions for a safe lane change. Our contributions and advantages of the approach are: 1 Considers the behavior generator as a continuous function in Reproducing Kernel Hilbert Space (RKHS) which contains a family of behavior generators; 2 Constructs the behavior generator function in RKHS by non-parametric regressions on training data; 3 Takes past trajectories of all related surrounding cars as input to capture mutual interactions and output continuous values to represent behaviors. Experimental results show that the proposed approach is able to generate feasible and human-like lane-change behavior (represented by start and end points) in multi-car environments. The experiments also verified that our suggested kernel outperforms the ones which were used in a previous method.

Cooperative driving behavior is essential for driving in traffic, especially for ramp merging, lane changing or navigating intersections. Autonomous vehicles should also manage these situations by behaving cooperatively and naturally. In this paper, we present a novel learning-based method to efficiently estimate other vehicles' intentions and interact with them in ramp merging scenarios, without over-the-air communication between vehicles. The intention estimate is generated from a Probabilistic Graphical Model (PGM) which organizes historical data and latent intentions and determines predictions. Real driving trajectories are used to learn transition models in the PGM. Thus, besides the structure of the PGM, our method does not require human-designed reward or cost functions. The PGM-based intention estimation is followed by an off-the-shelf ACC distance keeping model to generate proper acceleration/deceleration commands. The PGM plays a plug-in role in our self-driving framework [1]. We validate the performance of our method both on real merging data and using a designed merging strategy in simulation, and show significant improvements compared with previous methods. Parameter design is also discussed by experiments. The new method is computationally efficient, and does not require acceleration information about other vehicles, which is hard to read directly from sensors mounted on the autonomous vehicle.

The detection of surrounding vehicles is an essential task in autonomous driving, which has been drawing enormous attention recently. When using laser scanners, L-Shape fitting is a key step for model-based vehicle detection and tracking, which requires thorough investigation and comprehensive research. In this paper, we formulate the L-Shape fitting as an optimization problem. An efficient search based method is then proposed to find the optimal solution. Our method does not rely on laser scan sequence information and therefore supports convenient data fusion from multiple laser scanners; it is efficient and involves very few parameters for tuning; the approach is also flexible to suit various fitting demands with different fitting criteria. On-road experiments with production-grade laser scanners have demonstrated the effectiveness and robustness of our approach.

In a hierarchical motion planning system for urban autonomous driving, it is a common practice to separate tactical reasoning from the lower-level trajectory planning. This separation makes it difficult to achieve robust maneuver-based tactical reasoning, which is intrinsically linked to trajectory planning. We therefore propose a planning method that automatically discovers tactical maneuver patterns, and fuses pattern reasoning and sampling-based trajectory planning. The results demonstrate enhanced planning feasibility, coherency and scalability.

In this paper, we develop a motion planner for on-road autonomous swerve maneuvers that is capable of learning passengers' individual driving styles. It uses a hybrid planning approach that combines sampling-based graph search and vehicle model-based evaluation to obtain a smooth trajectory plan. To automate the parameter tuning process, as well as to reflect individual driving styles, we further adapt inverse reinforcement learning techniques to distill human driving patterns from maneuver demonstrations collected from different individuals. We found that the proposed swerve planner and its learning routine can approximate a good variety of maneuver demonstrations. However, due to the underlying stochastic nature of human driving, more data are needed in order to obtain a more generative swerve model.

Trajectory planning methods for on-road autonomous driving are commonly formulated to optimize a Single Objective calculated by accumulating Multiple Weighted Feature terms (SOMWF). Such formulation typically suffers from the lack of planning tunability. Two main causes are the lack of physical intuition and relative feature prioritization due to the complexity of SOMWF, especially when the number of features is big. This paper addresses this issue by proposing a framework with multiple tunable phases of planning, along with two novel techniques: Optimization-free trajectory smoothing/nudging. Sampling-based trajectory search with cascaded ranking.

This paper presents the autonomous tracking and following of a marine vessel by an Unmanned Surface Vehicle in the presence of dynamic obstacles while following the International Regulations for Preventing Collisions at Sea (COLREGS) rules. The motion prediction for the target vessel is based on Monte-Carlo sampling of dynamically feasible and collision-free paths with fuzzy weights, leading to a predicted path resembling anthropomorphic driving behavior. This prediction is continuously optimized for a particular target by learning the necessary parameters for a 3-degree-of-freedom model of the vessel and its maneuvering behavior from its path history without any prior knowledge. The path planning for the USV with COLREGS is achieved on a grid-based map in a single stage by incorporating A* path planning with Artificial Terrain Costs for dynamically changing obstacles. Various scenarios for interaction, including multiple civilian and adversarial vessels, are handled by the planner with ease. The effectiveness of the algorithms has been demonstrated both in representative simulations and on-water experiments.

This paper investigates real-time on-road motion planning algorithms for autonomous passenger vehicles (APV) in urban environments, and propose a computationally efficient planning formulation. Two key properties, tunability and stability, are emphasized when designing the proposed planner. The main contributions of this paper are: · A computationally efficient decoupled space-time trajectory planning structure. · The formulation of optimization-free elastic-band-based path planning and speed-constraint-based temporal planning routines with pre-determined runtime. · Identification of continuity problems with previous cost-based planners that cause tunability and stability issues.

We propose a robust object tracking algorithm for distance keeping. Taking advantage of a context-based region of interest, we are able to maximize the performance of each sensor, and reduce the computation time since we only focus on the targets inside the region. Tracking targets in road coordinates enables finding the distance-keeping target on any curved road, while a commercial Adaptive Cruise Control (ACC) system works best on straight roads. We demonstrate that the overall performance of the proposed algorithm is better than that of a commercial ACC system. The distance-keeping target can either be used for lane following for a standalone ACC system or an autonomous vehicle. Our object tracking algorithm can also be extended to find the target of interest for lane changing or ramp merging for an autonomous vehicle.

We present a motion planning framework for autonomous on-road driving considering both the uncertainty caused by an autonomous vehicle and other traffic participants. The future motion of traffic participants is predicted using a local planner, and the uncertainty along the predicted trajectory is computed based on Gaussian propagation. For the autonomous vehicle, the uncertainty from localization and control is estimated based on a Linear-Quadratic Gaussian (LQG) framework. Compared with other safety assessment methods, our framework allows the planner to avoid unsafe situations more efficiently, thanks to the direct uncertainty information feedback to the planner. We also demonstrate our planner's ability to generate safer trajectories compared to planning only with a LQG framework.

Prior autonomous navigation systems focused on the demonstration of the technological feasibility. But as the technology evolves, improving user experience through learning expert's or individual's driving pattern emerges as a promising research direction. As a first step toward this goal, we investigate methods to learn from human demonstrations in urban scenarios without any environmental disturbances (traffic-free). We propose a path model that generates a reference path with smooth and peak-value-reduced curvature, and a parameterized speed model to be fitted by human driving data. Model parameters are then learned through regression methods, and certain statistical human driving patterns are revealed. The learned model is then evaluated by comparing the generated plan with the collected data by the same human driver.

In this paper, we propose a novel planning framework that can greatly improve the level of intelligence and driving quality of autonomous vehicles. A reference planning layer first generates kinematically and dynamically feasible paths assuming no obstacles on the road, then a behavioral planning layer takes static and dynamic obstacles into account. Instead of directly commanding a desired trajectory, it searches for the best directives for the controller, such as lateral bias and distance keeping aggressiveness. It also considers the social cooperation between the autonomous vehicle and surrounding cars. Based on experimental results from both simulation and a real autonomous vehicle platform, the proposed behavioral planning architecture improves the driving quality considerably, with a 90.3% reduction of required computation time in representative scenarios.

An approach for formally verifying the safety of automated vehicles is proposed. Due to the uniqueness of each traffic situation, we verify safety online, i.e., during the operation of the vehicle. The verification is performed by predicting the set of all possible occupancies of the automated vehicle and other traffic participants on the road. In order to capture all possible future scenarios, we apply reachability analysis to consider all possible behaviors of mathematical models considering uncertain inputs (e.g., sensor noise, disturbances) and partially unknown initial states. Safety is guaranteed with respect to the modeled uncertainties and behaviors if the occupancy of the automated vehicle does not intersect that of other traffic participants for all times. The applicability of the approach is demonstrated by test drives with an automated vehicle at the Robotics Institute at Carnegie Mellon University.

Robotic vehicles have become a critical tool for studying the under-sampled coastal ocean. This has led to new paradigms in scientific discovery. The combination of agility, reactivity, and persistent presence makes autonomous robots ideal for targeted sampling of elusive, episodic events such as algal blooms. In order to achieve this goal, they need to be deployed at the right place and time. To that end, we have designed and will soon deploy a shore-based event recognition technology to continuously monitor remote sensing imagery for algal blooms as targets for robotic field experiments. A Support Vector Machine underlies a field-tested decision support system which scientists will consult prior to deploying robots in the coastal ocean. Our aim is to target oceanographic field experiments for evaluation and verification.

On-road motion planning for autonomous vehicles is in general a challenging problem. Past efforts have proposed solutions for urban and highway environments individually. We identify the key advantages/shortcomings of prior solutions, and propose a novel two-step motion planning system that addresses both urban and highway driving in a single framework. Reference Trajectory Planning (I) makes use of dense lattice sampling and optimization techniques to generate an easy-to-tune and human-like reference trajectory accounting for road geometry, obstacles and high-level directives. By focused sampling around the reference trajectory, Tracking Trajectory Planning (II) generates, evaluates and selects parametric trajectories that further satisfy kinodynamic constraints for execution. The described method retains most of the performance advantages of an exhaustive spatiotemporal planner while significantly reducing computation.

“Socially cooperative driving” is an integral part of our everyday driving, hence requiring special attention to imbue the autonomous driving with a more natural driving behavior. In this paper, an intention-integrated Prediction- and Cost function-Based algorithm (iPCB) framework is proposed to enable an autonomous vehicle to perform cooperative social behavior. An intention estimator is developed to extract the probability of surrounding agents' intentions in real time. Then for each candidate strategy, a prediction engine considering the interaction between host and surrounding agents is used to predict future scenarios. A cost function-based evaluation is applied to compute the cost for each scenario and select the decision corresponding to the lowest cost. The algorithm was tested in simulation on an autonomous vehicle cooperating with vehicles merging from freeway entrance ramps with 10,000 randomly generated scenarios. Compared with approaches that do not take social behavior into account, the iPCB algorithm shows a 41.7% performance improvement based on the chosen cost functions.

We present an autonomous driving research vehicle with minimal appearance modifications that is capable of a wide range of autonomous and intelligent behaviors, including smooth and comfortable trajectory generation and following; lane keeping and lane changing; intersection handling with or without V2I and V2V; and pedestrian, bicyclist, and workzone detection. Safety and reliability features include a fault-tolerant computing system; smooth and intuitive autonomous-manual switching; and the ability to fully disengage and power down the drive-by-wire and computing system upon E-stop. The vehicle has been tested extensively on both a closed test field and public roads.

In this paper, an efficient real-time autonomous driving motion planner with trajectory optimization is proposed. The planner first discretizes the plan space and searches for the best trajectory based on a set of cost functions. Then an iterative optimization is applied to both the path and speed of the resultant trajectory. The post-optimization is of low computational complexity and is able to converge to a higher-quality solution within a few iterations. Compared with the planner without optimization, this framework can reduce the planning time by 52% and improve the trajectory quality. The proposed motion planner is implemented and tested both in simulation and on a real autonomous vehicle in three different scenarios. Experiments show that the planner outputs high-quality trajectories and performs intelligent driving behaviors.

As robotic software systems become larger and more complex, it is increasingly important to reuse existing software components to control development costs. For a broad class of relatively simple components, ranging from sensor and actuation interfaces to many simple perception and navigation algorithms, this can be a reasonably straightforward process, and many excellent frameworks have been developed in recent years that support reuse of such components in novel systems. However, there is also a class of more advanced software components, such as for modeling and interacting with complex environments, for which reuse can be a much more challenging problem. In particular, many advanced robotic algorithms can be highly sensitive to the perception and actuation capabilities of the specific robots they are deployed on, in turn requiring significant and invasive modifications to accommodate the specific capabilities of a different robotic system. This work examines the nature of this sensitivity and proposes a novel design methodology for isolating a stable, reusable "core" algorithm from any platform-specific enhancements, or "supplemental" effects. Modern software engineering techniques are used to encapsulate these supplemental effects separately from the core algorithm, allowing platform-specific details to be accommodated by modular substitution instead of direct modification of the "core" component. This methodology is experimentally evaluated on existing software for autonomous driving behaviors, yielding useful insights into the creation of highly adaptable robotic software components.

We present a motion planner for autonomous highway driving that adapts the state lattice framework pioneered for planetary rover navigation to the structured environment of public roadways. The main contribution of this paper is a search space representation that allows the search algorithm to systematically and efficiently explore both spatial and temporal dimensions in real time. This allows the low-level trajectory planner to assume greater responsibility in planning to follow a leading vehicle, perform lane changes, and merge between other vehicles. We show that our algorithm can readily be accelerated on a GPU, and demonstrate it on an autonomous passenger vehicle.

In this paper, a point-based Markov Decision Process (QMDP) algorithm is used for robust single-lane autonomous driving behavior control under uncertainties. Autonomous vehicle decision making is modeled as a Markov Decision Process (MDP), then extended to a QMDP framework. Based on MDP/QMDP, three kinds of uncertainties are taken into account: sensor noise, perception constraints and surrounding vehicles' behavior. In simulation, the QMDP-based reasoning framework makes the autonomous vehicle perform with differing levels of conservativeness corresponding to different perception confidence levels. Road tests also indicate that the proposed algorithm helps the vehicle in avoiding potentially unsafe situations under these uncertainties. In general, the results indicate that the proposed QMDP-based algorithm makes autonomous driving more robust to limited sensing ability and occasional sensor failures.

In the mobile robotics literature, there is little formal discussion of reliability and failure. Moreover, current work focuses more on the assessment of existing robots. In contrast, our work predicts the impact on reliability on robotic missions. In our previous work, we presented a quantitative analysis to predict the probability of robot failure during a mission and use this to compare the performance of different robot team configurations. In order to comprehensively characterize robot failure, we proposed a taxonomy system which divides planetary robotic missions into three classes and showed how the taxonomy can be used as a framework to explore the reliability characteristics of each mission class. In this paper, we define and simulate common mission scenarios for each class in the taxonomy system and attempt to extract general reliability trends and mission characteristics for given robot and environment parameters. Our results show that, for comparable mission scopes with a fixed budget, exploration-type missions have maximum mission success probability for smaller team sizes than is the case for construction-type missions.

In this paper, a prediction- and cost function-based algorithm (PCB) is proposed to implement robust freeway driving in autonomous vehicles. A prediction engine is built to predict the future microscopic traffic scenarios. With the help of a human-understandable and representative cost function library, the predicted traffic scenarios are evaluated and the best control strategy is selected based on the lowest cost. The prediction- and cost function-based algorithm is verified using the simulator of the autonomous vehicle Boss from the DARPA Urban Challenge 2007. The results of both case tests and statistical tests using PCB show enhanced performance of the autonomous vehicle in performing distance keeping, lane selecting and merging on freeways.

This paper proposes a multi-level collaborative driving framework (MCDF) for human-autonomous vehicle interaction. There are three components in MCDF; the mission-behavior-motion block diagram, the functionality-module relationship and the human participation level table. Through integration of the three components, a human driver can cooperate with the vehicle's intelligence to achieve better driving performance, robustness and safety. The MCDF is successfully implemented in TROCS, a real-time autonomous vehicle control system developed by the Tartan Racing Team for the 2007 DARPA Urban Challenge. The performance of MCDF is analyzed and a preliminary test in TROCS simulation mode shows that MCDF is effective in improving the driving performance.

The reliability of individual team members has a substantial and complex influence on the success of multirobot missions. When one robot fails, other robots must be retasked to complete the tasks that were assigned to the failed robot. This in turn increases the likelihood of these other robots failing, since they have more work to do. Existing multirobot task allocation systems consider robot failures only after the fact-by replanning after a failure occurs. We hypothesize that it should be important to consider robot reliabilities when generating an initial plan. In this paper we test this hypothesis in the context of the multirobot exploration problem. We take a simple exhaustive planner and compare the plan it chooses against the optimal plan that takes into account robot failures and the backup plans that occur after failure. Our results show that for this problem domain, making an initial plan without regards to individual robot reliabilities results in choosing a suboptimal plan most of the time, and that the difference in mission performance between the chosen plan and the optimal plan is usually substantial. In brief, in order to successfully plan we must 'plan to fail'.

This paper describes a multi-robot science exploration software architecture and system called the telesupervised adaptive ocean sensor fleet (TAOSF). TAOSF supervises and coordinates a group of robotic boats, the OASIS platforms, to enable in situ study of phenomena in the ocean/atmosphere interface, as well as on the ocean surface and sub-surface. The OASIS platforms are extended-deployment autonomous ocean surface vessels, whose development is funded separately by the National Oceanic and Atmospheric Administration (NOAA). TAOSF allows a human operator to effectively supervise and coordinate multiple robotic assets using a multi-level autonomy control architecture, where the operating mode of the vehicles ranges from autonomous control to teleoperated human control. TAOSF increases data-gathering effectiveness and science return while reducing demands on scientists for robotic asset tasking, control, and monitoring. The first field application chosen for TAOSF is the characterization of Harmful Algal Blooms (HABs). We discuss the overall TAOSF architecture, describe field tests conducted under controlled conditions using rhodamine dye as a HAB simulant, present initial results from these tests, and outline the next steps in the development of TAOSF.

We describe an autonomous robotic software subsystem for managing mission execution and discrete traffic interaction in the 2007 DARPA Urban Challenge. Its role is reviewed in the context of the software system that controls ldquoBossrdquo, Tartan Racingpsilas winning entry in the competition. Design criteria are presented, followed by the application of software design principles to derive an architecture well suited to the rigors of developing complex robotic systems. Combined with a discussion of robust behavioral algorithms, the designpsilas effectiveness is highlighted in its ability to manage complex autonomous driving behaviors while remaining adaptable to the systempsilas evolving capabilities.

The exploration problem is a central issue in mobile robotics. A complete coverage is not practical if the environment is large with a few small hotspots, and the sampling cost is high. So, it is desirable to build robot teams that can coordinate to maximize sampling at these hotspots while minimizing resource costs, and consequently learn more accurately about properties of such environmental phenomena. An important issue in designing such teams is the exploration strategy. The contribution of this paper is in the evaluation of an adaptive exploration strategy called adaptive cluster sampling (ACS), which is demonstrated to reduce the resource costs (i.e., mission time and energy consumption) of a robot team, and yield more information about the environment by directing robot exploration towards hotspots. Due to the adaptive nature of the strategy, it is not obvious how the sampled data can be used to provide unbiased, low-variance estimates of the properties. This paper therefore discusses how estimators that are Rao-Blackwellized can be used to achieve low error. This paper also presents the first analysis of the characteristics of the environmental phenomena that favor the ACS strategy and estimators. Quantitative experimental results in a mineral prospecting task simulation show that our approach is more efficient in exploration by yielding more minerals and information with fewer resources and providing more precise mineral density estimates than previous methods.

This paper describes efficient utilization of human time by two means: prioritization of human tasks and maximizing multirobot team size. We propose an efficient scheduling algorithm for multirobot supervisory control that helps complete a mission faster. The proposed algorithm is superior to existing algorithms by prioritizing human tasks such that robots can regain autonomous control sooner. In simulations of a multirobot area surveying problem, we show that the rate of area coverage is much higher using our algorithm compared to first-in-first-out. We also show that the use of different scheduling algorithms can affect the maximum number of robots a human can manage on a team. Another significant finding related to maximum team size is that the size is always the same or higher than an often-cited estimate known as fan-out [5]. Since fan-out is derived from an ideal, average case, simulations show that the upper bound on team size is higher than that predicted by the fan-out equation. Fan-out is actually a lower bound on the maximum team size for any practical situation (i.e., where task lengths and periodicity may vary or when robots are heterogeneous).

In January 2004, NASA began a bold enterprise to return to the Moon, and with the technologies and expertise gained, press on to Mars. The underlying Vision for Space Exploration calls for a sustained and affordable human and robotic program to explore the solar system and beyond; to conduct human expeditions to Mars after successfully demonstrating sustained human exploration missions on the Moon. The approach is to "send human and robotic explorers as partners, leveraging the capabilities of each where most useful." Human-robot interfacing technologies for this approach are required at readiness levels above any available today. In this paper, we describe the HRI aspects of a robot supervision architecture we are developing under NASA's auspices, based on the authors' extensive experience with field deployment of ground, underwater, lighter-than-air, and inspection autonomous and semi-autonomous robotic vehicles and systems.

One reason given for the use of multirobot systems is that many cheap robots are more reliable than one expensive robot. To date, however, there has been no quantitative analysis to support this assertion. This paper presents the first quantitative support for the argument that larger teams of less-reliable robots can perform certain missions more reliably than smaller teams of more-reliable robots. Our results show that for short missions, in fact, a team of four robots can provide greater mission reliability than a team of two robots, even when the individual robots in the team of four have reliability that is an order of magnitude lower. These results suggest that considerable cost reductions can be achieved for some missions by choosing larger teams of less-reliable robots over smaller teams of more-reliable robots

Occupancy grid mapping algorithms assume that grid block values are independently distributed. However, most environments of interest contain spatial patterns that are better characterized by models that capture dependencies among grid blocks. To account for such dependencies, we model the environment as a pairwise Markov random field. We specify a belief propagation-based mapping algorithm that takes these dependencies into account when estimating a map. To demonstrate the potential benefits of this approach, we simulate a simple multi-robot minefield mapping scenario. Minefields contain spatial dependencies since some landmine configurations are more likely than others, and since clutter, which causes false alarms, can be concentrated in certain regions and completely absent in others. Our belief propagation-based approach outperforms conventional occupancy grid mapping algorithms in the sense that better maps can be obtained with significantly fewer robot measurements. The belief propagation algorithm requires a modest amount of increased computation, but we contend that in applications where significant energy and time expenditure is associated with robot movement and active sensing, the reduction in the required number of samples justified the increased computation.

This work describes a method for observing maneuvering targets using a group of mobile robots equipped with video cameras. These robots are part of a team of small-size (7/spl times/7/spl times/7 cm) robots configured from modular components that collaborate to accomplish a given task. The cameras seek to observe the target while facing it as much as possible from their respective viewpoints. This work considers the problem of scheduling and maneuvering the cameras based on the evaluation of their current positions in terms of how well can they maintain a frontal view of the target. We describe our approach, which distributes the task among several robots and avoids extensive energy consumption on a single robot. We explore the concept in simulation and present results.

Deployed vision systems often encounter image variations poorly represented in their training data. While observing their environment, such vision systems obtain unlabeled data that could be used to compensate for incomplete training. In order to exploit these relatively cheap and abundant unlabeled data we present a family of algorithms called /spl lambda/MEEM. Using these algorithms, we train an appearance-based people detection model. In contrast to approaches that rely on a large number of manually labeled training points, we use a partially labeled data set to capture appearance variation. One can both avoid the tedium of additional manual labeling and obtain improved detection performance by augmenting a labeled training set with unlabeled data. Further, enlarging the original training set with new unlabeled points enables the update of detection models after deployment without human intervention. To support these claim we show people detection results, and compare our performance to a purely generative expectation maximization-based approach to learning over partially labeled data.

The effectiveness of multirobot learning in achieving optimal, cooperative solutions is potentially affected by various factors having to do with the nature and configuration of the robots and the nature and configuration of the robots and the nature of the learning entities. Varying one factor wrongly may lead to undesirable results. There is no reported work on how systematically to set up these factors. In this paper, we methodically test the effect of varying four common factors (reward scope, global information delay, diversity of robots, and number of robots) in a decentralized multirobot system, first in simulation and then on real robots. The results show that two of these factors, reward scope and global information delay, if set up incorrectly, can prevent optimal, cooperative solutions.

Learning can be an effective way for robot systems to deal with dynamic environments and changing task conditions. However, popular single-robot learning algorithms based on discounted rewards, such as Q learning, do not achieve cooperation (i.e., purposeful division of labor) when applied to task-level multirobot systems. A task-level system is defined as one performing a mission that is decomposed into subtasks shared among robots. We demonstrate the superiority of average-reward-based learning such as the Monte Carlo algorithm for task-level multirobot systems, and suggest an explanation for this superiority.

The objective of the CyberScout project is to develop an autonomous surveillance and reconnaissance system using a network of all-terrain vehicles. We focus on two facets of this system: 1) vision for surveillance and 2) autonomous navigation and dynamic path planning. In the area of vision-based surveillance, we have developed robust, efficient algorithms to detect, classify, and track moving objects of interest (person, people, or vehicle) with a static camera. Adaptation through feedback from the classifier and tracker allow the detector to use grayscale imagery, but perform as well as prior color-based detectors. We have extended the detector using scene mosaicing to detect and index moving objects when the camera is panning or tilting. The classification algorithm performs well with coarse inputs, has unparalleled rejection capabilities, and can flag novel moving objects. The tracking algorithm achieves highly accurate (96%) frame-to-frame correspondence for multiple moving objects in cluttered scenes by determining the discriminant relevance of object features. We have also developed a novel mission coordination architecture, CPAD (Checkpoint/Priority/Action Database), which performs path planning via checkpoint and dynamic priority assignment, using statistical estimates of the environment's motion structure. The motion structure is used to make both preplanning and reactive behaviors more efficient by applying global context. This approach is more computationally efficient than centralized approaches and exploits robot cooperation in dynamic environments better than decoupled approaches.

To focus on the research issues surrounding collaborative behavior in multiple mobile-robotic systems, a great amount of low-level infrastructure is required. To facilitate our on-going research into multi-robot systems, we have developed RAVE, a software framework that provides a real and virtual environment for running and managing multiple heterogeneous mobile-robot systems. This framework simplifies the implementation and development of collaborative robotic systems by providing the following capabilities: the ability to run systems off-line in simulation, user-interfaces for observing and commanding simulated and real robots, transparent transference of simulated robot programs to real robots, the ability to have simulated robots interact with real robots, and the ability to place virtual sensors on real robots to augment or experiment with their performance.

The postural stiffness of the human arm has previously been estimated by displacing the hand from a series of equilibrium positions and correlating the resultant displacements and restoring forces. We extend this experimental methodology to include measurement of dynamic components of impedance. The stiffness-damping-mass characteristics are represented numerically as matrices and graphically as ellipses characterized by size, shape, and orientation. The latter depict the predominant nonrotational component of the impedance force fields. The results suggest: (1) joint damping is related to both joint stiffness and joint inertia; and (2) two-joint impedances, i.e., impedances associated with muscles connected across both the elbow and shoulder joints, play a relatively smaller role in damping than in stiffness. The ability to modulate stiffness in the face of initial static bias forces, i.e., "loading", is also examined. We observe regular shifts in the human arm endpoint's "spring center" corresponding to the bias force directions and magnitudes.>