Effective physical human-robot interaction requires systems that are not only adaptable to user preferences but also transparent about their actions. This paper introduces BRIDGE, a system for bidirectional human-robot communication in physical assistance. Our method allows users to modify a robot's planned trajectory---position, velocity, and force---in real time using natural language. We utilize a large language model (LLM) to interpret any trajectory modifications implied by user commands in the context of the planned motion and conversation history. Importantly, our system provides verbal feedback in response to the user, either assuring any resulting changes or posing a clarifying question. We evaluated our method in a user study with 18 older adults across three assistive tasks, comparing BRIDGE to an ablation without verbal feedback and a baseline. Results show that participants successfully used the system to modify trajectories in real time. Moreover, the bidirectional feedback led to significantly higher ratings of interactivity and transparency, demonstrating that the robot's verbal response is critical for a more intuitive user experience. Videos and code can be found on our project website.

Achieving robotic touch that is accepted and trusted by human users requires a clear understanding of touch parameters and how they influence perceptions of safety and comfort. We explore such parameters through a participatory experiment and generate design guidelines for robotic touch, involving 20 participants in designing, evaluating, and iterating robotic parameters for instrumental touch using a robotic arm equipped with a robotic hand. Through our user study, we find that parameters such as robot’s speed, force, and motion, contact surface material, and the robot hand pose impact users’ perception of safety, comfort, intuitiveness, efficiency, effectiveness, trust, and sense of agency. We also find that the choice of robotic parameters is task-dependent and body-part dependent, and that these parameters inter-correlate with each other. Based on these findings, we provide design guidelines for generating user-friendly robotic touch interactions in instrumental touch contexts.

Planning contact-rich interactions for multi-finger manipulation is challenging due to the high-dimensionality and hybrid nature of dynamics. Recent advances in data-driven methods have shown promise, but are sensitive to the quality of training data. Combining learning with classical methods like trajectory optimization and search adds additional structure to the problem and domain knowledge in the form of constraints, which can lead to outperforming the data on which models are trained. We present Diffusion-Informed Probabilistic Contact Search (DIPS), which uses an A* search to plan a sequence of contact modes informed by a diffusion model. We train the diffusion model on a dataset of demonstrations consisting of contact modes and trajectories generated by a trajectory optimizer given those modes. In addition, we use a particle filter-inspired method to reason about variability in diffusion sampling arising from model error, estimating likelihoods of trajectories using a learned discriminator. We show that our method outperforms ablations that do not reason about variability and can plan contact sequences that outperform those found in training data across multiple tasks. We evaluate on simulated tabletop card sliding and screwdriver turning tasks, as well as the screwdriver task in hardware to show that our combined learning and planning approach transfers to the real world.

Robots that can physically interact with humans in a safe manner have the potential to revolutionize application domains like home assistance and nursing care. However, to become long-term companions, such robots must learn user-specific preferences and adapt their behaviors in real time. We propose a Constrained Partially Observable Markov Decision Process framework for modeling human safety preferences over representative variables like force, velocity, and proximity. These variables are modeled as adaptive linear constraints, with a belief over their upper bounds that is updated online based on noisy human feedback. By modeling the belief as phase dependent, the model captures varying preferences across different task phases. The robot then solves a hierarchical optimization to select actions that respect both the learned constraints and robot motion limits. Our method does not require offline training data and can be applied directly to diverse physical interaction tasks and operation modes (tele-operated or autonomous). A pilot study shows that our approach effectively learns user preferences and improves perceived safety while reducing user effort compared to baselines.

When using a tool, the grasps used for picking it up, reposing, and holding it in a suitable pose for the desired task could be distinct. Therefore, a key challenge for autonomous in-hand tool manipulation is finding a sequence of grasps that facilitates every step of the tool use process while continuously maintaining force closure and stability. Due to the complexity of modeling the contact dynamics, reinforcement learning (RL) techniques can provide a solution in this continuous space subject to highly parameterized physical models. However, these techniques impose a trade-off in adaptability and data efficiency. At test time the tool properties, desired trajectory, and desired application forces could differ substantially from training scenarios. Adapting to this necessitates more data or computationally expensive online policy updates. In this work, we apply the principles of discrete dynamic programming (DP) to augment RL performance with domain knowledge. Specifically, we first design a computationally simple approximation of our environment. We then demonstrate in physical simulation that performing tree searches (i.e., lookaheads) and policy rollouts with this approximation can improve an RL-derived grasp sequence policy with minimal additional online computation. Additionally, we show that pretraining a deep RL network with the DP-derived solution to the discretized problem can speed up policy training.

This paper presents a hierarchical framework for planning and control of in-hand manipulation of a rigid object involving grasp changes using fully-actuated multifin-gered robotic hands. While the framework can be applied to the general dexterous manipulation, we focus on a more complex definition of in-hand manipulation, where at the goal pose the hand has to reach a grasp suitable for using the object as a tool. The high level planner determines the object trajectory as well as the grasp changes, i.e. adding, removing, or sliding fingers, to be executed by the low-level controller. While the grasp sequence is planned online by a learning-based policy to adapt to variations, the trajectory planner and the low-level controller for object tracking and contact force control are exclusively model-based to robustly realize the plan. By infusing the knowledge about the physics of the problem and the low-level controller into the grasp planner, it learns to successfully generate grasps similar to those generated by model-based optimization approaches, obviating the high computation cost of online running of such methods to account for variations. By performing experiments in physics simulation for realistic tool use scenarios, we show the success of our method on different tool-use tasks and dexterous hand models. Additionally, we show that this hybrid method offers more robustness to trajectory and task variations compared to a model-based method.

In this paper a dynamic model is proposed for a wheeled mobile robot (WMR) that incorporates a novel slip characterization and is used to study navigation under hard control constraints. Many available dynamic models of WMR assume that either the wheels roll without slipping or has longitudinal slip only. As a result they do not necessarily capture the true dynamics of WMRs. The model proposed in this paper considers a two wheeled differential drive mobile robot moving on a flat surface with possibly non-zero lateral and longitudinal slipping. A novel slip characterizing model is introduced and defining parameters are estimated via a set of experiments. Compared with several existing models in the literature simulation results show that trajectories predicted by the proposed model agree better with experimental data. The introduced dynamics and the corresponding slip model coupled with a novel direction search scheme were then used to ascertain the reachable states under constrained control. The scheme involves a finite dimensional search procedure which eliminates having to deal with two point boundary value problems resulting from classic optimal control formulations. The results of the paper can be easily applied to the design of control schemes for WMRs under realistic surface topography and control input constraints.