A major appeal of learning from demonstrations or imitation learning (IL) in robotics is that it learns a policy directly from lay users. However, Lay users may inadvertently provide erroneous demonstrations that lead to learning of policies that are inaccurate and hence, unsafe for humans and/or robot. This paper makes two contributions in the endeavour of recognizing human errors in demonstrations and thereby helping to learn a safe IL policy. First, we created a dataset – Layman V1.0 – with 15 lay users who provided a total of 1200 demonstrations for three simulated tasks – Lift, Can and Square in the simulated Robosuite environment – and two real robot tasks with a Sawyer robot, using a custom designed Android app for tele-operation. Second, we propose a framework named Behavior Cloning for Error Detection (BED) to autonomously detect and discard erroneous demonstrations from a demonstration pool. Our method uses a Behavior Cloning method as self-supervised technique and assigns binary weight to each demonstration based on its inconsistencies with the rest of the demonstrations. We show the effectiveness of this framework in detecting incorrect demonstrations in the Layman V1.0 dataset. We further show that state-of-the-art (SOTA) policy learners learns a better policy when bad demonstrations, identified through the proposed framework, are removed from the training pool. Dataset and Codes are available in https://github.com/AssistiveRoboticsUNH/bed

This paper proposes a novel autonomous dynamic system (ADS) based controller for trajectory learning from demonstration (LfD). We call our method Learning Stable Dynamics via Iterative Quadratic Programming (LSD-IQP). LSD-IQP learns an energy function and an ADS from demonstrations via semi-infinite quadratic programming. Energy function constraints are imposed on the learned ADS to ensure convergence to a single goal position. Unlike other energy-based methods, LSD-IQP allows the energy function to have both local maximums and saddle points. This flexibility enables LSD-IQP to learn a broader class of motions compared to other ADS-based controllers. We demonstrate the capabilities of LSD-IQP via several experiments, including: 1) learning handwritten symbols and comparing the swept error area to several other ADS methods 2) learning a pick-and-place task with novel goal positions for a robot, and 3) learning a point to point motion in the presence of a non-convex obstacle for a robot.

In this paper, we propose a novel network architecture for visual imitation learning that exploits neural radiance fields (NeRFs) and key-point correspondence for self-supervised visual motor policy learning. The proposed network architecture incorporates a dynamic system output layer for policy learning. Combining the stability and goal adaption properties of dynamic systems with the robustness of keypoint-based correspondence yields a policy that is invariant to significant clutter, occlusions, lighting conditions changes, and spatial variations in goal configurations. Experiments on multiple manipulation tasks show that our method outperforms comparable visual motor policy learning methods on both in-distribution and out-of-distribution scenarios when using a small number of training samples.

We are interested in learning from demonstration (LfD) that can both learn and execute a trajectory and evaluate the quality of a previously unseen trajectory in the domain of assistive robotics. To this end, we propose a novel continuous inverse optimal control (IOC) formulation that simultaneously learns an optimal time-invariant controller and an evaluation metric from human demonstrations. We assume that the expert’s objective function is a weighted combination of physically meaningful basis objective functions. The evaluation metric is derived from the learned expert’s objective function. The benefit of this approach is twofold: 1) the controller can be optimized with respect to the learned evaluation metric and subject to the robot’s dynamic limitations and 2) the evaluation metric can evaluate the quality of a demonstrated trajectory. We validate our approach with two experiments in a robot guided therapy setting: 1) evaluating demonstrated exercises with the learned metric and 2) reproducing both unconstrained trajectories and trajectories subject to the robot’s dynamic constraints.

Imitation learning (IL) frameworks in robotics typically assume that a domain expert's demonstration always contains a correct way of doing the task. Despite its theoretical convenience, this assumption has limited practical values for an IL-powered robot in real world. There are many reasons for an expert in the real world to provide demonstrations that may contain incorrect or potentially unsafe way of doing a task. In order for IL-powered robots to work in the real world, IL frameworks need to detect such adversarial demonstrations and not learn from them. This paper proposes an IL framework that can autonomously detect and remove adversarial demonstrations, if they exist in the demonstration set, as it directly learns a task policy from the expert. The proposed framework that we term Robust Maximum Entropy behavior cloning (R-MaxEnt) learns a stochastic model that maps states to actions. In doing so, R-MaxEnt solves a minmax problem that leverages the entropy of the model to assign weights to different demonstrations while assigning poor weights to adversarial samples. Our empirical results show that R-MaxEnt outperforms the existing IL approaches in both real and simulated robotics tasks.

This paper proposes a dynamic system based learning from demonstration approach to teach a robot activities of daily living. The approach takes inspiration from human movement literature to formulate trajectory learning as an optimal control problem. We assume a weighted combination of basis objective functions is the true objective function for a demonstrated motion. We derive basis objective functions analogous to those in human movement literature to optimize the robot's motion. This method aims to naturally adapt the learned motion in different situations. To validate our approach, we learn motions from two categories: 1) commonly prescribed therapeutic exercises and 2) tea making. We show the reproduction accuracy of our method and compare torque requirements to the dynamic motion primitive for each motion, with and without an added load.

High-level human activities often have rich temporal structures that determine the order in which atomic actions are executed. We propose the Temporal Context Graph (TCG), a temporal reasoning model that integrates probabilistic inference with Allen's interval algebra, to capture these temporal structures. TCGs are capable of modeling tasks with cyclical atomic actions and consisting of sequential and parallel temporal relations. We present Learning from Demonstration as the application domain where the use of TCGs can improve policy selection and address the problem of perceptual aliasing. Experiments validating the model are presented for learning two tasks from demonstration that involve structured human-robot interactions. The source code for this implementation is available at https://github.com/AssistiveRoboticsUNH/TCG.

A major goal of learning from demonstration is task generalization via observation of a teacher. In this paper, we propose a novel framework for learning motion from a single demonstration. Our approach reconstructs the demonstrated trajectory's phase space curve via a linear piece-wise regression method. We approximate dynamics of trajectory segments with linear time invariant equations, each yielding closed form solutions. We show convergence to desired phase space states via an energy-based analysis. The robustness of the model is evaluated on a robot for a sequential trajectory task. Additionally, we show the advantages that the phase space model has over the dynamic motion primitive for a kinematic based task.

This paper presents a framework to learn the reward function underlying high-level sequential tasks from demonstrations. The purpose of reward learning, in the context of learning from demonstration (LfD), is to generate policies that mimic the demonstrator's policies, thereby enabling imitation learning. We focus on a human-robot interaction (HRI) domain where the goal is to learn and model structured interactions between a human and a robot. Such interactions can be modeled as a partially observable Markov decision process (POMDP) where the partial observability is caused by uncertainties associated with the ways humans respond to different stimuli. The key challenge in finding a good policy in such a POMDP is determining the reward function that was observed by the demonstrator. Existing inverse reinforcement learning (IRL) methods for POMDPs are computationally very expensive and the problem is not well understood. In comparison, IRL algorithms for Markov decision process (MDP) are well defined and computationally efficient. We propose an approach of reward function learning for high-level sequential tasks from human demonstrations where the core idea is to reduce the underlying POMDP to an MDP and apply any efficient MDPIRL algorithm. Our extensive experiments suggest that the reward function learned this way generates POMDP policies that mimic the policies of the demonstrator well.

There are many human-robot interaction (HRI) tasks that are highly structured and follow a certain temporal sequence. Learning such tasks from demonstrations requires understanding the underlying rules governing the interactions. This involves identifying and generalizing the key spatial and temporal features of the task and capturing the high-level relationships among them. Despite its crucial role in sequential task learning, temporal reasoning is often ignored in existing learning from demonstration (LFD) research. This paper proposes a holistic LFD framework that learns the underlying temporal structure of sequential HRI tasks. The proposed Temporal-Reasoning-based LFD (TR-LFD) framework relies on an automated spatial reasoning layer to identify and generalize relevant spatial features, and a temporal reasoning layer to analyze and learn the high-level temporal structure of a HRI task. We evaluate the performance of this framework by learning a well-explored task in HRI research: robot-mediated autism intervention. The source code for this implementation is available at https://github.com/AssistiveRoboticsUNH/TR-LFD.

Extracting a set of generalizable rules that govern the dynamics of complex, high-level interactions between humans based only on observations is a high-level cognitive ability. Mastery of this skill marks a significant milestone in the human developmental process. A key challenge in designing such an ability in autonomous robots is discovering the relationships among discriminatory features. Identifying features in natural scenes that are representative of a particular event or interaction (i.e. »discriminatory features») and then discovering the relationships (e.g., temporal/spatial/spatio-temporal/causal) among those features in the form of generalized rules are non-trivial problems. They often appear as a »chicken-and-egg» dilemma. This paper proposes an end-to-end learning framework to tackle these two problems in the context of learning generalized, high-level rules of human interactions from structured demonstrations. We employed our proposed deep reinforcement learning framework to learn a set of rules that govern a behavioral intervention session between two agents based on observations of several instances of the session. We also tested the accuracy of our framework with human subjects in diverse situations.

We present a learning from demonstration (LfD) framework that uses a deep recurrent Q-network (DRQN) to learn how to deliver a behavioral intervention (BI) from demonstrations performed by a human. The trained DRQN enables a robot to deliver a similar BI in an autonomous manner. BIs are highly structured procedures wherein children with developmental delays/disorders (e.g. autism, ADHD, etc.) are trained to perform new behaviors and life-skills. Mounting anecdotal evidence from human-robot interaction (HRI) research has shown that BI benefits from the use of robots as a delivery tool. Most of the HRI research on robot-based intervention relies on tele-operated robots. However, the need for autonomy has become increasingly evident, especially when it comes to the real-world deployment of these systems. The few studies that have used autonomy in robot-based BI relied on hand-picked features of the environment in order to trigger correct robot actions. Additionally, none of these automated architectures attempted to learn the BI from human demonstrations, though this appears to be the most natural way of learning. This paper represents the first attempt to design a robot that uses LfD to learn BI. We generate a model then correctly predict appropriate actions with greater than 80% accuracy. To the best of our knowledge, this is the first attempt to employ DRQN within an LfD framework to learn high level reasoning embedded in human actions and behaviors simply from observations.

We have designed an augmented reality (AR) game for children with cerebral palsy (CP) to perform home-based neurorehabilitation. A Myo armband detects electromyographic (EMG) signals and accelerometer data from the arm, and a trained classifier determines whether the neuromotor performance of the arm satisfies the expectation of the exercise. The user can move a virtual object only through therapist-prescribed motor movement. The user completes the exercise by moving the virtual object to some targets displayed in the glass.

Human-robot interaction involving the failure of autonomous robots is not yet well understood. We conducted two online surveys with a total of 1200 participants in which people assessed situations where an autonomous robot experienced different kinds of failure. This information was used to construct a measurement scale of people's reaction to failure where positive values correspond with increasingly positive reactions and negative values with negative reactions. We then used this scale to compare different kinds of failure situations, including the severity of the failures, the context risk involved, and the effectiveness of different kinds of recovery strategies. We found evidence that the effectiveness of recovery strategies depends on the task, context, and severity of failure.

The paper proposes a learning from demonstration (LfD) framework which will enable children with motor disabilities to perform neuromotor rehabilitation exercises at home- and community- settings. LfD, a popular robot learning paradigm, has traditionally been used to teach embodied robots different skills through demonstrations by lay users. In this paper, we propose a novel application of LfD in the health-care domain. The goal of the proposed LfD framework is to learn standard rehabilitation exercises from a therapist's demonstration during a patient's clinic visit and assist the patient to perform the exercises at home through demonstrating (using a 3D avatar) different steps of the exercise. Motion information and EMG signals of a patient are used to train a Markov Decision Process (MDP) model with different steps of the exercise from real-time demonstrations. The MDP model then tracks the progress of a patient as (s)he performs the exercise at home and provides prompts if there is any error or missed steps. The MDP model also allows quantitative evaluation of a patient's performance and improvements over time, a highly desirable property of any home-based rehabilitation system.

A significant amount of robotics research over the past decade has shown that many children with autism spectrum disorders (ASD) have a strong interest in robots and robot toys, concluding that robots are potential tools for the therapy of individuals with ASD. However, clinicians, who have the authority to approve robots in ASD therapy, are not convinced about the potential of robots. One major reason is that the research in this domain does not have a strong focus on the efficacy of robots. Robots in ASD therapy are end-user oriented technologies, the success of which depends on their demonstrated efficacy in real settings. This paper focuses on measuring the efficacy of robots in ASD therapy and, based on the data from a feasibility study, shows that the human-robot interaction (HRI) metrics commonly used in this research domain might not be sufficient.

A common characteristics of the computational models of visual attention is they execute the two modes of visual attention (visual exploration and visual search) separately. This makes a visual attention model unsuitable for real-world robotic applications. This paper focuses on integrating visual exploration and visual search in a common framework of visual attention and the challenges resulting from such integration. It proposes a visual attention-oriented speech-based human robot interaction framework which helps a robot to switch back and-forth between the two modes of visual attention. A set of experiments are presented to demonstrate the performance of the proposed framework.

Turn-taking is fundamental to the way humans engage in information exchange, but robots currently lack the turn-taking skills required for natural communication. In order to bring effective turn-taking to robots, we must first understand the underlying processes in the context of what is possible to implement. We describe a data collection experiment with an interaction format inspired by “Simon says,” a turn-taking imitation game that engages the channels of gaze, speech, and motion. We analyze data from 23 human subjects interacting with a humanoid social robot and propose the principle of minimum necessary information (MNI) as a factor in determining the timing of the human response.We also describe the other observed phenomena of channel exclusion, efficiency, and adaptation. We discuss the implications of these principles and propose some ways to incorporate our findings into a computational model of turn-taking.

Visual attention is one of the major requirements for a robot to serve as a cognitive companion for human. The robotic visual attention is mostly concerned with overt attention which accompanies head and eye movements of a robot. In this case, each movement of the camera head triggers a number of events, namely transformation of the camera and the image coordinate systems, change of content of the visual field, and partial appearance of the stimuli. All of these events contribute to the reduction in probability of meaningful identification of the next focus of attention. These events are specific to overt attention with head movement and, therefore, their effects are not addressed in the classical models of covert visual attention. This paper proposes a Bayesian model as a robot-centric solution for the overt visual attention problem. The proposed model, while taking inspiration from the primates visual attention mechanism, guides a robot to direct its camera toward behaviorally relevant and/or visually demanding stimuli. A particle filter implementation of this model addresses the challenges involved in overt attention with head movement. Experimental results demonstrate the performance of the proposed model.

The interaction between a robot and a human becomes meaningful when the robotic agent possesses some level of human-like cognition. This paper proposes an attention-based approach for multi-modal HRI. The core of the proposed approach is a bio-inspired artificial model of visual attention which enables a robot to focus on the visually salient and/or behaviorally relevant stimuli in the surrounding. The attention model provides the human partner with the opportunity to manipulate the attention behavior of the robot through natural speech command. Similarly the robot is able to manipulate the attention of the human partner using its actuators. Thus the bio-inspired visual attention mechanism along with the sensors and actuators enables the robot to establish joint attention with the human partner. Formation of this joint attention is the basis for further human-robot interaction. Experimental results validate different aspects of the proposed attention-based HRI framework.

This paper argues that the object- and space-based modes of visual attention can be naturally integrated in a common mathematical framework. In an earlier work we have proposed a mathematical model of visual attention for robotic system exploiting the knowledge of visual attention mechanism of the primates. This paper investigates on the validity of the proposed model for robotic systems through experimentation on a real robot. The paper sheds light on a number of real world issues involved with the design of visual attention system for physically embodied robots and explains how the proposed Bayesian model of visual attention addresses these issues. The object- and space-based modes of visual attention are naturally integrated in the model and is reflected in the sequential Monte Carlo implementation of the model on a real robot.

This paper presents a real world application of fuzzy logic and Genetic algorithm (GA) in mobile robotics. It proposes a novel method of integrating fuzzy logic and GA to solve the Simultaneous Localization And Mapping (SLAM) problem of mobile robots. The proposed algorithm, termed as Fuzzy-Evolutionary SLAM, solves the global optimization problem of SLAM where the objective function measures the quality of a robot's pose in accommodating a local map into a partially developed global map of the environment. The search for the optimal robot's pose is performed by a GA. Knowledge on the problem domain is preprocessed by a fuzzy logic system and allows the GA to evolve within a specified region of the search space. It helps to speed-up the GA based search. The proposed algorithm processes data in an incremental fashion and follows essentially no assumption about the environment. Experimental results validate the performance of the proposed algorithm.

This paper presents a novel algorithm for simultaneous localization and mapping (SLAM) of mobile robots. The proposed algorithm, termed as evolutionary SLAM, is based on an island model genetic algorithm (IGA). The IGA searches for the most probable map(s) such that the underlying robot's pose(s) provide a robot with the best localization information. The correspondence problem in SLAM is solved by exploiting the property of natural selection, to support only better performing individuals to survive. The algorithm does not follow any explicit heuristics for loop closing, rather maintains multiple hypotheses to solve the loop closing problem. The algorithm processes sensor data incrementally and therefore, has the capability to work online. Experimental results in different indoor environments validate the robustness of the proposed algorithm

This paper proposes a novel algorithm combining fuzzy logic (FL) and genetic algorithm (GA) for concurrent mapping and localization (CML) of mobile robot. First, CML is formulated as a multidimensional informed search problem. The search is performed to detect a robot pose which can best accommodate the recent sensor scan in the currently available map. A fuzzy set theoretic approach is used to predict a sample based representation of the state space of possible robot poses and a GA is designed to find out the globally optimal solution from the predicted pose space. The GA evaluates the fitness of poses based on the sensory information and drives the generation gradually towards the globally optimal solution even when the fuzzy prediction is inaccurate. The best fit solution as decided by GA offers the most likely continuation of the currently available map. Experiment on synthetic and real data illustrates the robustness of the algorithm.