The performance of optimization-based robot motion planning algorithms is highly dependent on the initial solutions, commonly obtained by running a sampling-based planner to obtain a collision-free path. However, these methods can be slow in high-dimensional and complex scenes and produce nonsmooth solutions. Given previously solved path-planning problems, it is highly desirable to learn their distribution and use it as a prior for new similar problems. Several works propose utilizing this prior to bootstrap the motion planning problem, either by sampling initial solutions from it, or using its distribution in a maximum-a-posterior formulation for trajectory optimization. In this work, we introduce motion planning diffusion (MPD), an algorithm that learns trajectory distribution priors with diffusion models. These generative models have shown increasing success in encoding multimodal data and have desirable properties for gradient-based motion planning, such as cost guidance. Given a motion planning problem, we construct a cost function and sample from the posterior distribution using the learned prior combined with the cost function gradients during the denoising process. Instead of learning the prior on all trajectory waypoints, we propose learning a lower dimensional representation of a trajectory using linear motion primitives, particularly B-spline curves. This parametrization guarantees that the generated trajectory is smooth, can be interpolated at higher frequencies, and needs fewer parameters than a dense waypoint representation. We demonstrate the results of our method ranging from simple 2-D to more complex tasks using a 7-DOF robot arm manipulator. In addition to learning from simulated data, we also use human demonstrations on a real-world pick-and-place task. The experiment results show that diffusion models are strong priors for encoding multimodal trajectory distributions for optimization-based motion planning.

Abstract Research on multi-agent interaction involving humans is still in its infancy. Most approaches have focused on environments with collaborative human behavior or a small, defined set of situations. When deploying robots in human-inhabited environments in the future, the diversity of interactions surpasses the capabilities of pre-trained collaboration models. ”Coexistence” environments, characterized by agents with varying or partially aligned objectives, present a unique challenge for robotic collaboration. Traditional reinforcement learning methods fall short in these settings. These approaches lack the flexibility to adapt to changing agent counts or task requirements without undergoing retraining. Moreover, existing models do not adequately support scenarios where robots should exhibit helpful behavior toward others without compromising their primary goals. To tackle this issue, we introduce a novel framework that decomposes interaction and task-solving into separate learning problems and blends the resulting policies at inference time using a goal inference model for task estimation. We create impact-aware agents and linearly scale the cost of training agents with the number of agents and available tasks. To this end, a weighting function blending action distributions for individual interactions with the original task action distribution is proposed. To support our claims we demonstrate that our framework scales in task and agent count across several environments and considers collaboration opportunities when present. The new learning paradigm opens the path to more complex multi-robot, multi-human interactions.

Interactive Reinforcement Learning (IntRL) with human advice has shown great potential for human-guided self-improvement of robots and can accelerate learning compared to traditional Reinforcement Learning. However, most existing approaches assume perfectly correct advice or partially incorrect advice is only accounted for by assessing trustworthiness of human advice equally across all states. This can lead to problems in practical scenarios, where human advice might be inaccurate in some states but still useful in others. We propose a novel IntRL algorithm that handles state-dependent unreliable action advice by computing a trust estimate for both human advice and the robot’s own policy. We use three indicators to assess trustworthiness of human advice, namely consistency of advice, retrospective optimality, and a multi-modal human uncertainty classifier based on behavioral cues. For estimating the state-dependent trust in the robot’s policy, we compare five different methods. Evaluations in gridworlds with simulated advice show that our approach significantly outperforms a state-independent baseline. Robotic experiments with perceptual uncertainty and advice from 26 participants confirm the usefulness of the included human uncertainty classification as an indicator for unreliable advice. Additionally, we show that our approach is more robust to incorrect advice compared to a state-independent computation of trust in the policy.

The performance of optimization-based robot motion planning algorithms is highly dependent on the initial solutions, commonly obtained by running a sampling-based planner to obtain a collision-free path. However, these methods can be slow in high-dimensional and complex scenes and produce nonsmooth solutions. Given previously solved path-planning problems, it is highly desirable to learn their distribution and use it as a prior for new similar problems. Several works propose utilizing this prior to bootstrap the motion planning problem, either by sampling initial solutions from it, or using its distribution in a maximum-a-posterior formulation for trajectory optimization. In this work, we introduce motion planning diffusion (MPD), an algorithm that learns trajectory distribution priors with diffusion models. These generative models have shown increasing success in encoding multimodal data and have desirable properties for gradient-based motion planning, such as cost guidance. Given a motion planning problem, we construct a cost function and sample from the posterior distribution using the learned prior combined with the cost function gradients during the denoising process. Instead of learning the prior on all trajectory waypoints, we propose learning a lower dimensional representation of a trajectory using linear motion primitives, particularly B-spline curves. This parametrization guarantees that the generated trajectory is smooth, can be interpolated at higher frequencies, and needs fewer parameters than a dense waypoint representation. We demonstrate the results of our method ranging from simple 2-D to more complex tasks using a 7-DOF robot arm manipulator. In addition to learning from simulated data, we also use human demonstrations on a real-world pick-and-place task. The experiment results show that diffusion models are strong priors for encoding multimodal trajectory distributions for optimization-based motion planning.

This article presents a method for learning well-coordinated Human-Robot Interaction (HRI) from Human-Human Interactions (HHI). We devise a hybrid approach using Hidden Markov Models (HMMs) as the latent space priors for a Variational Autoencoder to model a joint distribution over the interacting agents. We leverage the interaction dynamics learned from HHI to learn HRI and incorporate the conditional generation of robot motions from human observations into the training, thereby predicting more accurate robot trajectories. The generated robot motions are further adapted with Inverse Kinematics to ensure the desired physical proximity with a human, combining the ease of joint space learning and accurate task space reachability. For contact-rich interactions, we modulate the robot's stiffness using HMM segmentation for a compliant interaction. We verify the effectiveness of our approach deployed on a Humanoid robot via a user study. Our method generalizes well to various humans despite being trained on data from just two humans. We find that users perceive our method as more human-like, timely, and accurate and rank our method with a higher degree of preference over other baselines. We additionally show the ability of our approach to generate successful interactions in a more complex scenario of Bimanual Robot-to-Human Handovers.

Designing interactions for human-AI teams (HATs) can be challenging due to an AI agent’s potential autonomy. Previous work suggests that higher autonomy does not always improve team performance, and situation-dependent autonomy adaptation might be beneficial. However, there is a lack of systematic empirical evaluations of such autonomy adaptation in human-AI interaction. Therefore, we propose a cooperative task in a simulated shared workspace to investigate effects of fixed levels of AI autonomy and situation-dependent autonomy adaptation on team performance and user satisfaction. We derive adaptation rules for AI autonomy from previous work and a pilot study. We implement these rule for our main experiment and find that team performance was best when humans collaborated with an agent adjusting its autonomy based on the situation. Additionally, users rated this agent highest in terms of perceived intelligence. From these results, we discuss the influence of varying autonomy degrees on HATs in shared workspaces.

In a question-and-answer setting, the respondent is often not only communicating the requested information but also indicating their confidence in the answer through various behavioral cues. Humans excel at interpreting these cues and monitoring the uncertainty of other persons. Being able to detect human uncertainty in human-robot interactions in a similar way can enable future robotic systems to better recognize uncertain and error-prone human input. Additionally, automatic human uncertainty detection can enhance the responsiveness of robots to the user in moments of uncertainty by providing help or clarification. While there is some work on uncertainty detection based on a single modality, only a few works focus on multi-modal uncertainty detection. Even fewer works explore how human uncertainty manifests through behavioral cues in human-robot interactions. In this work, we analyze occurrences of behavioral cues related to self-reported uncertainty on experimental data from 27 participants across two decision-making tasks. Additionally, in the first task, we varied if participants interacted with a human or a robot. On the recorded data, we extract features accessible via a webcam and a microphone and train a multi-modal classifier. Experimental evaluation of our developed classifier shows that it significantly outperforms third-person annotators in accuracy and F1 score. Humans report feeling less observed when responding to a robot compared to a human. Nevertheless, we found that the behavioral differences did not significantly affect the performance of our proposed uncertainty classification.

Learning priors on trajectory distributions can help accelerate robot motion planning optimization. Given previously successful plans, learning trajectory generative models as priors for a new planning problem is highly desirable. Prior works propose several ways on utilizing this prior to bootstrapping the motion planning problem. Either sampling the prior for initializations or using the prior distribution in a maximum-a-posterior formulation for trajectory optimization. In this work, we propose learning diffusion models as priors. We then can sample directly from the posterior trajectory distribution conditioned on task goals, by leveraging the inverse denoising process of diffusion models. Furthermore, diffusion has been recently shown to effectively encode data multi-modality in high-dimensional settings, which is particularly well-suited for large trajectory dataset. To demonstrate our method efficacy, we compare our proposed method - Motion Planning Diffusion - against several baselines in simulated planar robot and 7-dof robot arm manipulator environments. To assess the generalization capabilities of our method, we test it in environments with previously unseen obstacles. Our experiments show that diffusion models are strong priors to encode high-dimensional trajectory distributions of robot motions. https://sites.google.com/view/mp-diffusion

Extracting modular segments from raw video demonstrations of high-level actions is important to understand the underlying building blocks for different tasks in human-robot interaction. While (data-hungry) supervised learning approaches for Action Segmentation show good performance when the underlying segments are predefined, their performance degrades when unseen actions are introduced on-the-go as new data samples are scarce. In this regard, Zero-and Few-Shot Learning approaches have shown good performance in generalizing to unseen examples. In Action Segmentation, where each frame needs to be labeled, annotating new data even for a few tasks can become tedious as the number of tasks scale. In this work, we propose Interactive Iterative Improvement $(I^{3})$ for Few-Shot Action Segmentation, a Semi-Supervised Interactive Meta-Learning approach for Zero-Shot Learning on unlabeled videos and Few-Shot Learning on small amounts of labeled videos. $I^{3}$ consists of a Prototypical Network model for frame-wise prediction coupled with a Hidden-Semi-Markov-Model to prevent over-segmentation. The model is iteratively improved in an interactive manner through users’ annotations provided via a webinterface. This is done in a task-agnostic manner that, in theory, can be reused for a number of different actions. Our model provides sequentially accurate segmentations using only a limited amount of labeled data which shows the efficacy of our learning approach. A lower edit distance compared to baselines indicates a lower number of required user edits making it well suited for non-expert users to smoothly provide annotations enabling them to have more control over the learned model.

Assistive robots interacting with people promise to increase quality of life and productivity in households, caregiving, or industry settings. Importantly, the quality of such interactions crucially depends on the intuitive ease and reliability of humans being able to request the robot’s assistance. Thus, the ability to detect a human’s Intention for Interaction (IFI) is beneficial for human-robot interaction across multiple application domains. However, existing works that detect IFIs often focus on single tasks, contexts, or interactions or limit their data collection to invariability in human positions. In contrast, here we aim for a more task-independent IFI detection. We record natural human behavior in an experimental setup with a two-armed robot that includes different tasks and interactions, and different positions and orientations of the human towards the robot. We collected audio and RGB-D data from 21 human subjects in the proposed experimental setup resulting in overall 405 IFIs. Using head orientation, shoulder orientation, distance, speech activity recognition, and hotword detection as features, we trained multimodal probabilistic classifiers. We compare feature fusion and decision fusion using the Bayesian fusion method Independent Opinion Pool. The resulting multimodal classifiers can detect task-independent IFIs from natural human behavior with an F1 score of up to 0.81. Overall, we show that good IFI detection can be achieved by modularly combining individual classifiers probabilistically.

This paper introduces the first workshop on “Empowering Interactive Robots by Learning Through Multimodal Feedback Channel” organized at the 23rd ACM International Conference on Multimodal Interaction in Montreal, Canada. This workshop aims to bring together researcher from different disciplines to stimulate a discussion on how multimodality can become key to the emerging field of interactive machine learning.

The majority of learning from demonstration approaches do not address suboptimal demonstrations or cases when drastic changes in the environment occur after the demonstrations were made. For example, in real teleoperation tasks, the demonstrations provided by the user are often suboptimal due to interface and hardware limitations. In tasks involving co-manipulation and manipulation planning, the environment often changes due to unexpected obstacles rendering previous demonstrations invalid. This paper presents a reinforcement learning algorithm that exploits the use of relevance functions to tackle such problems. This paper introduces the Pearson correlation as a measure of the relevance of policy parameters in regards to each of the components of the cost function to be optimized. The method is demonstrated in a static environment where the quality of the teleoperation is compromised by the visual interface (operating a robot in a three-dimensional task by using a simple 2D monitor). Afterward, we tested the method on a dynamic environment using a real 7-DoF robot arm where distributions are computed online via Gaussian Process regression.

Assistive robots can potentially improve the quality of life and personal independence of elderly people by supporting everyday life activities. To guarantee a safe and intuitive interaction between human and robot, human intentions need to be recognized automatically. As humans communicate their intentions multimodally, the use of multiple modalities for intention recognition may not just increase the robustness against failure of individual modalities but especially reduce the uncertainty about the intention to be recognized. This is desirable as particularly in direct interaction between robots and potentially vulnerable humans a minimal uncertainty about the situation as well as knowledge about this actual uncertainty is necessary. Thus, in contrast to existing methods, in this work a new approach for multimodal intention recognition is introduced that focuses on uncertainty reduction through classifier fusion. For the four considered modalities speech, gestures, gaze directions and scene objects individual intention classifiers are trained, all of which output a probability distribution over all possible intentions. By combining these output distributions using the Bayesian method Independent Opinion Pool [1] the uncertainty about the intention to be recognized can be decreased. The approach is evaluated in a collaborative human-robot interaction task with a 7-DoF robot arm. The results show that fused classifiers, which combine multiple modalities, outperform the respective individual base classifiers with respect to increased accuracy, robustness, and reduced uncertainty.

Challenging tasks in unstructured environments require robots to learn complex models. Given a large amount of information, learning multiple simple models can offer an efficient alternative to a monolithic complex network. Training multiple models-that is, learning their parameters and their responsibilities-has been shown to be prohibitively hard as optimization is prone to local minima. To efficiently learn multiple models for different contexts, we thus develop a new algorithm based on expectation maximization (EM). In contrast to comparable concepts, this algorithm trains multiple modules of paired forward-inverse models by using the prediction errors of both forward and inverse models simultaneously. In particular, we show that our method yields a substantial improvement over only considering the errors of the forward models on tasks where the inverse space contains multiple solutions.