Learning from Demonstration (LfD) empowers robots to acquire new skills through human demonstrations, making it feasible for everyday users to teach robots. However, the success of learning and generalization heavily depends on the quality of these demonstrations. Consistency is often used to indicate quality in LfD, yet the factors that define this consistency remain underexplored. In this article, we evaluate a comprehensive set of motion data characteristics to determine which consistency measures best predict learning performance. By ensuring demonstration consistency prior to training, we enhance models’ predictive accuracy and generalization to novel scenarios. We validate our approach with two user studies involving participants with diverse levels of robotics expertise. In the first study ( N = 24), users taught a PR2 robot to perform a button-pressing task in a constrained environment, while in the second study ( N = 30), participants trained an UR5 robot on a pick-and-place task. Results show that demonstration consistency significantly impacts success rates in both learning and generalization, with 70% and 89% of task success rates in the two studies predicted using our consistency metrics. Moreover, our metrics estimate generalized performance success rates with 76% and 91% accuracy. These findings suggest that our proposed measures provide an intuitive, practical way to assess demonstration data quality before training, without requiring expert data or algorithm-specific modifications. Our approach offers a systematic way to evaluate demonstration quality, addressing a critical gap in LfD by formalizing consistency metrics that enhance the reliability of robot learning from human demonstrations.

Learning from Demonstration (LfD) is a framework that allows lay users to easily program robots. However, the efficiency of robot learning and the robot’s ability to generalize to task variations hinge upon the quality and quantity of the provided demonstrations. Our objective is to guide human teachers to provide more effective demonstrations, thus facilitating efficient robot learning. To achieve this, we propose to use a measure of uncertainty, namely task-related information entropy , as a criterion for suggesting informative demonstration examples to human teachers to improve their teaching skills. This approach seeks to minimize the requisite number of demonstrations by enhancing their distribution throughout the workspace. In a conducted experiment \((N = 24)\) , an augmented reality (AR)-based guidance system was employed to train novice users to produce additional demonstrations from areas with the highest entropy within the workspace. These novice users were trained for a few trials to teach the robot a generalizable task using a limited number of demonstrations. Subsequently, the users’ performance after training was assessed first on the same task (retention) and then on a new task (transfer) without guidance. The results indicate a substantial improvement in robot learning efficiency from the teacher’s demonstrations, with an improvement of up to 198% observed on the novel task. Furthermore, the proposed approach was compared to a state-of-the-art heuristic rule and found to improve robot learning efficiency by 210% compared to the heuristic rule. The scripts used in this article are available on GitHub .

We provide an experimental evaluation of a wearable augmented reality (AR) system we have developed for human-robot teams working on tasks requiring collaboration in shared physical workspace. Recent advances in AR technology have facilitated the development of more intuitive user interfaces for many human-robot interaction applications. While it has been anticipated that AR can provide a more intuitive interface to robot assistants helping human workers in various manufacturing scenarios, existing studies in robotics have been largely limited to teleoperation and programming. Industry 5.0 envisions cooperation between human and robot working in teams. Indeed, there exist many industrial tasks that can benefit from human-robot collaboration. A prime example is high-value composite manufacturing. Working with our industry partner towards this example application, we evaluated our AR interface design for shared physical workspace collaboration in human-robot teams. We conducted a multi-dimensional analysis of our interface using established metrics. Results from our user study (n = 26) show that, subjectively, the AR interface feels more novel and a standard joystick interface feels more dependable to users. However, the AR interface was found to reduce physical demand and task completion time, while increasing robot utilization. Furthermore, user’s freedom of choice to collaborate with the robot may also affect the perceived usability of the system.

In shared autonomy, the user input is blended with the assistive motion to accomplish a task where the user goal is typically unknown to the robot. Transparency between the human and robot is essential for effective collaboration. Prior works have provided methods for the robot to infer the user goal; however, they are usually dependent on the distance between the robot and object, which may not be directly associated with the real-time user control intention and thus cause low control feelings. Here, we propose a real-time goal prediction method driven by assistive motion generated by learning from demonstration (LfD) allowing more reactive assistive behaviors. This LfD-generated assistive motion is blended with the user input based on goal predictions to achieve targeted tasks. The LfD policy was learned offline and used with different users. To evaluate our proposed method, we compared it with a state-of-the-art Partially Observable Markov Decision Process (POMDP) based method using a distance cost, and a direct control method (i.e., joystick). A pilot study (N = 6) was conducted to control a 6-DoF Kinova Mico robotic arm to carry out three tasks: (1) reaching-and-grasping, (2) pouring, and (3) object-returning with the three control methods. We used both objective and subjective measures in the comparative study. Results show that our method has the shortest task completion time, the lowest amount of joystick control inputs among all three control methods, as well as a significantly lower angular difference between the user input and assistive motion compared to the POMDP-based method. Besides, it obtains the highest subjective score in the user preference and perceived speed ratings, and the second-highest in the control feeling and the robot did what I wanted ratings.

This paper investigate potential use of augmented reality (AR) for physical human-robot collaboration in large-scale, labour-intensive manufacturing tasks. While it has been shown that use of AR can help increase task efficiency in teleoperative and robot programming tasks involving smaller-scale robots, its use for physical human-robot collaboration in shared workspaces and large-scale manufacturing tasks have not been well-studied. With the eventual goal of applying our AR system to collaborative aircraft body manufacturing, we compare in a user study the use of an AR interface we developed with a standard joystick for human robot collaboration in an experiment task simulating industrial carbon-fibre-reinforced-polymer manufacturing procedure. Results show that use of AR yields reduced task time and physical demand, with increased robot utilization.

Programming and executing robot trajectories is a routine manufacturing procedure. However, current interfaces (i.e., teach pendants) are bulky, unintuitive, and interrupts task flow. Recently, augmented reality (AR) has been used to create alternative solutions. However, input modalities of such systems tend to be limited. By introducing the use of electromyography (EMG), we have created a novel multimodal wearable interface for online trajectory programming and execution. Through the use of EMG, our system aims to bridge the user's force activation to the robot arm force profile. Our proposed system provides two interaction methods for trajectory execution and force control using 1) arm EMG and 2) arm orientation. We compared these methods with a standard joystick in a user study to test their usability. Results show that proposed methods have increased physical demands but yield equivalent task performance, demonstrating the potential of our proposed interface to provide a wearable alternative solution.

Robot Learning from Demonstration (LfD) allows robots to implement autonomous manipulation by observing the movements executed by a demonstrator. As such, LfD has been established as a key element for useful user interactions in everyday environments. Kinesthetic teaching, a teaching technique within LfD, entails physically guiding the robot to achieve a task. When demonstrating complex actions on a multi-DoF manipulator, novice users typically encounter difficulties with trajectory continuity and joint orientation, necessitating training by an expert. A comparison between different training approaches is conducted in a study of nine novice users. These approaches are kinesthetic, observational and discovery-learning. The kinesthetic method utilizes record and playback functions implemented on a 7-DoF Barrett Technology WAM robot. A novice user passively holds the arm while an expert's trajectory is replayed. A visual demonstration by the expert is used for the observational training group. The discovery-learning group does not receive an expert demonstration; they use trial-and-error to produce the trajectory on their own. Task-space performance is evaluated pre- and post-training for each user to determine the relative and absolute performance improvements of the groups across the three training approaches. Absolute performance improvements are compared to the performance of an expert and a minimum-jerk trajectory to gauge how skillful the participant becomes with respect to the expert. The kinesthetic approach shows superior indicators of performance in trajectory similarity to the minimum-jerk trajectory with 39% and 13% improvement over the observational and discovery methods, respectively. Observational training shows greater improvement in terms of the smoothness of the velocity profile with 32.7% compared to 29.5% and 21.9% for both discovery and kinesthetic training, respectively.

Force Myography (FMG) is a method of tracking functional motor activity using volumetric changes associated with muscle function. With comparable accuracy and multiple advantages over traditional methods of functional motor activity tracking, FMG has shown a promising potential in terms of applications in human-machine interfaces, tele-operation and healthcare devices. This paper provides a study that explores the effect of the spatial coverage and placement of the Force Myography (FMG) measurements on the accuracy and predictability of the machine learning models of isometric hand force. Five participants were recruited in this study and were asked to exert isometric force along three perpendicular axes while wearing custom built FMG devices. During the tests, the isometric force was measured using a 6 degree-of-freedom (DOF) load cell whereas the FMG signals were recorded using a total number of 60 FSRs, which were embedded into four bands worn on the arm. General Regression Neural Network (GRNN) model was employed in this study for predicting the hand force in three axes from the recorded FMG signals. The regression model was trained using all possible band combinations to find the optimal placement for the FMG measurements. The results showed that the accuracy significantly improved when increasing the spatial coverage from 1 FMG band to 2 or 3 bands for all axes. While the accuracy slightly improved when the 4 bands used instead of 3. Specifically, the average R2across all subjects and axes are 0.68 ± 0.12, 0.84 ± 0.04, 0.91 ± 0.02 and 0.95 ± 0.01 using single, double, triple and four bands combination, respectively, in 5-fold cross-validation evaluation. The knowledge generated from this work aims serve as a guide towards the development of portable FMG based technology for widespread deployment in the general population.