Interactive Reinforcement Learning (IntRL) with human advice has shown great potential for human-guided self-improvement of robots and can accelerate learning compared to traditional Reinforcement Learning. However, most existing approaches assume perfectly correct advice or partially incorrect advice is only accounted for by assessing trustworthiness of human advice equally across all states. This can lead to problems in practical scenarios, where human advice might be inaccurate in some states but still useful in others. We propose a novel IntRL algorithm that handles state-dependent unreliable action advice by computing a trust estimate for both human advice and the robot’s own policy. We use three indicators to assess trustworthiness of human advice, namely consistency of advice, retrospective optimality, and a multi-modal human uncertainty classifier based on behavioral cues. For estimating the state-dependent trust in the robot’s policy, we compare five different methods. Evaluations in gridworlds with simulated advice show that our approach significantly outperforms a state-independent baseline. Robotic experiments with perceptual uncertainty and advice from 26 participants confirm the usefulness of the included human uncertainty classification as an indicator for unreliable advice. Additionally, we show that our approach is more robust to incorrect advice compared to a state-independent computation of trust in the policy.

In a question-and-answer setting, the respondent is often not only communicating the requested information but also indicating their confidence in the answer through various behavioral cues. Humans excel at interpreting these cues and monitoring the uncertainty of other persons. Being able to detect human uncertainty in human-robot interactions in a similar way can enable future robotic systems to better recognize uncertain and error-prone human input. Additionally, automatic human uncertainty detection can enhance the responsiveness of robots to the user in moments of uncertainty by providing help or clarification. While there is some work on uncertainty detection based on a single modality, only a few works focus on multi-modal uncertainty detection. Even fewer works explore how human uncertainty manifests through behavioral cues in human-robot interactions. In this work, we analyze occurrences of behavioral cues related to self-reported uncertainty on experimental data from 27 participants across two decision-making tasks. Additionally, in the first task, we varied if participants interacted with a human or a robot. On the recorded data, we extract features accessible via a webcam and a microphone and train a multi-modal classifier. Experimental evaluation of our developed classifier shows that it significantly outperforms third-person annotators in accuracy and F1 score. Humans report feeling less observed when responding to a robot compared to a human. Nevertheless, we found that the behavioral differences did not significantly affect the performance of our proposed uncertainty classification.

Extracting modular segments from raw video demonstrations of high-level actions is important to understand the underlying building blocks for different tasks in human-robot interaction. While (data-hungry) supervised learning approaches for Action Segmentation show good performance when the underlying segments are predefined, their performance degrades when unseen actions are introduced on-the-go as new data samples are scarce. In this regard, Zero-and Few-Shot Learning approaches have shown good performance in generalizing to unseen examples. In Action Segmentation, where each frame needs to be labeled, annotating new data even for a few tasks can become tedious as the number of tasks scale. In this work, we propose Interactive Iterative Improvement $(I^{3})$ for Few-Shot Action Segmentation, a Semi-Supervised Interactive Meta-Learning approach for Zero-Shot Learning on unlabeled videos and Few-Shot Learning on small amounts of labeled videos. $I^{3}$ consists of a Prototypical Network model for frame-wise prediction coupled with a Hidden-Semi-Markov-Model to prevent over-segmentation. The model is iteratively improved in an interactive manner through users’ annotations provided via a webinterface. This is done in a task-agnostic manner that, in theory, can be reused for a number of different actions. Our model provides sequentially accurate segmentations using only a limited amount of labeled data which shows the efficacy of our learning approach. A lower edit distance compared to baselines indicates a lower number of required user edits making it well suited for non-expert users to smoothly provide annotations enabling them to have more control over the learned model.

Assistive robots interacting with people promise to increase quality of life and productivity in households, caregiving, or industry settings. Importantly, the quality of such interactions crucially depends on the intuitive ease and reliability of humans being able to request the robot’s assistance. Thus, the ability to detect a human’s Intention for Interaction (IFI) is beneficial for human-robot interaction across multiple application domains. However, existing works that detect IFIs often focus on single tasks, contexts, or interactions or limit their data collection to invariability in human positions. In contrast, here we aim for a more task-independent IFI detection. We record natural human behavior in an experimental setup with a two-armed robot that includes different tasks and interactions, and different positions and orientations of the human towards the robot. We collected audio and RGB-D data from 21 human subjects in the proposed experimental setup resulting in overall 405 IFIs. Using head orientation, shoulder orientation, distance, speech activity recognition, and hotword detection as features, we trained multimodal probabilistic classifiers. We compare feature fusion and decision fusion using the Bayesian fusion method Independent Opinion Pool. The resulting multimodal classifiers can detect task-independent IFIs from natural human behavior with an F1 score of up to 0.81. Overall, we show that good IFI detection can be achieved by modularly combining individual classifiers probabilistically.