Deep Reinforcement Learning (DRL) has made considerable advances in simulated and physical robot control tasks, especially when problems admit a fully observed Markov Decision Process (MDP) formulation. When observations only partially capture the underlying state, the problem becomes a Partially Observable MDP (POMDP), and performance rankings between algorithms can change. We empirically compare Proximal Policy Optimization (PPO), Twin Delayed Deep Deterministic Policy Gradient (TD3), and Soft Actor-Critic (SAC) on representative POMDP variants of continuous-control benchmarks. Contrary to widely reported MDP results where TD3 and SAC typically outperform PPO, we observe an inversion: PPO attains higher robustness under partial observability. We attribute this to the stabilizing effect of multi-step bootstrapping. Furthermore, incorporating multi-step targets into TD3 (MTD3) and SAC (MSAC) improves their robustness. These findings provide practical guidance for selecting and adapting DRL algorithms in partially observable settings without requiring new theoretical machinery.

Learning from Demonstration (LfD) empowers robots to acquire new skills through human demonstrations, making it feasible for everyday users to teach robots. However, the success of learning and generalization heavily depends on the quality of these demonstrations. Consistency is often used to indicate quality in LfD, yet the factors that define this consistency remain underexplored. In this article, we evaluate a comprehensive set of motion data characteristics to determine which consistency measures best predict learning performance. By ensuring demonstration consistency prior to training, we enhance models’ predictive accuracy and generalization to novel scenarios. We validate our approach with two user studies involving participants with diverse levels of robotics expertise. In the first study ( N = 24), users taught a PR2 robot to perform a button-pressing task in a constrained environment, while in the second study ( N = 30), participants trained an UR5 robot on a pick-and-place task. Results show that demonstration consistency significantly impacts success rates in both learning and generalization, with 70% and 89% of task success rates in the two studies predicted using our consistency metrics. Moreover, our metrics estimate generalized performance success rates with 76% and 91% accuracy. These findings suggest that our proposed measures provide an intuitive, practical way to assess demonstration data quality before training, without requiring expert data or algorithm-specific modifications. Our approach offers a systematic way to evaluate demonstration quality, addressing a critical gap in LfD by formalizing consistency metrics that enhance the reliability of robot learning from human demonstrations.

In human–robot teams, human situational awareness is the operator’s conscious knowledge of the team’s states, actions, plans and their environment. Appropriate human situational awareness is critical to successful human–robot collaboration. In human–robot teaming, it is often assumed that the best and required level of situational awareness is knowing everything at all times. This view is problematic, because what a human needs to know for optimal team performance varies given the dynamic environmental conditions, task context, and roles and capabilities of team members. We explore this topic by interviewing 16 participants with active and repeated experience in diverse human–robot teaming applications. Based on analysis of these interviews, we derive a framework explaining the dynamic nature of required situational awareness in human–robot teaming. In addition, we identify a range of factors affecting the dynamic nature of required and actual levels of situational awareness (i.e., dynamic situational awareness), types of situational awareness inefficiencies resulting from gaps between actual and required situational awareness, and their main consequences. We also reveal various strategies, initiated by humans and robots, that assist in maintaining the required situational awareness. Our findings inform the implementation of accurate estimates of dynamic situational awareness and the design of user-adaptive human–robot interfaces. Therefore, this work contributes to the future design of more collaborative and effective human–robot teams.

Positive human-perception of robots is critical to achieving sustained use of robots in shared environments. One key factor affecting human-perception of robots are their sounds, especially the consequential sounds which robots (as machines) must produce as they operate. This paper explores qualitative responses from 182 participants to gain insight into human-perception of robot consequential sounds. Participants viewed videos of different robots performing their typical movements, and responded to an online survey regarding their perceptions of robots and the sounds they produce. Topic analysis was used to identify common properties of robot consequential sounds that participants expressed liking, disliking, wanting or wanting to avoid being produced by robots. Alongside expected reports of disliking high pitched and loud sounds, many participants preferred informative and audible sounds (over no sound) to provide predictability of purpose and trajectory of the robot. Rhythmic sounds were preferred over acute or continuous sounds, and many participants wanted more natural sounds (such as wind or cat purrs) in-place of machine-like noise. The results presented in this paper support future research on methods to improve consequential sounds produced by robots by highlighting features of sounds that cause negative perceptions, and providing insights into sound profile changes for improvement of human-perception of robots, thus enhancing human robot interaction.

This tutorial aims to equip researchers with the knowledge and skills to leverage multimodal data in human-robot interaction (HRI) studies. It covers the HRI study cycle, from sensor selection to data analysis, introducing commonly used sensors, pre-processing data, feature extraction techniques, fusion techniques and analysis techniques: both frequentist and Bayesian. Hands-on exercises using public datasets are designed to provide practical experience. The concluding panel discussion on ethics and bias in HRI is focused on fostering broader ethical considerations of HRI studies. Website tutorial is found online at https://sites.google.com/monash.edu/multimodal-hri-study-tutorial.

The first workshop on human-robot interaction in extreme and challenging environments (exactingHRI: https://sites.google.com/monash.edu/exactinghril) focuses on the forefront of HRI research in applications where robots are working with diverse users in uncertain, unknown, or risky environments to deliver reliable outcomes in repeated sessions. In these scenarios, a robot's autonomous and interactive functions are put to the test, with errors likely to arise. Such HRI systems require design and evaluation in-situ with target users, i.e., “exacting” HRI. The exactingHRI 2025 workshop aims to bring together researchers that investigate the diverse human, robot, task, environment, and interaction factors that are challenging for state-of-the-art HRI systems, as well as innovative designs, theories, models, and methods that equip people and robots with the ability to address these challenges. Workshop presenters will share lessons they have learned from successful or failed attempts in testing their work in such difficult settings, in a bid to encourage and guide the necessary efforts that progress our field to solve real-world problems.

Online evaluation is increasingly adopted in robotics research, providing an efficient approach to collect data from large and diverse populations. However, there have been ongoing debates about online studies as a proxy for in-person studies, especially where a participant passively observes video of robot behaviours or interaction. We conduct an online video comparison study (N=178) evaluating three robot handover policies in a collaborative assembly task, namely an adaptive autonomous policy, a non-adaptive scripted policy, and teleoperation. Participants watched three sets of videos in third-person view, each consisting of 9 sequential handovers executing one of the policies. Compared to in-person participants in two previous studies who evaluated handovers as users, online participants were observant of different robot behaviours and human-robot collaboration contexts, with 76.4% and 71.9% recognising the adaptive handovers exhibited by the teleoperated and autonomous robot, respectively. However, as observers, online participants showed more critical subjective perceptions compared to the in-person participants with a user’s perspective. They valued efficiency over adaptation with twice more autonomous handovers rated as being too late compared to scripted handovers. Our work highlights the need to consider user contexts when evaluating human-robot collaboration.

Abstract Seamless integration of Planning Under Uncertainty and Reinforcement Learning (RL) promises to bring the best of both model-driven and data-driven worlds to multi-agent decision-making, resulting in an approach with assurances on performance that scales well to more complex problems. Despite this potential, progress in developing such methods has been hindered by the lack of adequate evaluation and simulation platforms. Researchers have had to rely on creating custom environments, which reduces efficiency and makes comparing new methods difficult. In this paper, we introduce POSGGym : a library for facilitating planning and RL research in partially observable, multi-agent domains. It provides a diverse collection of discrete and continuous environments, complete with their dynamics models and a reference set of policies that can be used to evaluate generalization to novel co-players. Leveraging POSGGym, we empirically investigate existing state-of-the-art planning methods and a method that combines planning and RL in the type-based reasoning setting. Our experiments corroborate that combining planning and RL can yield superior performance compared to planning or RL alone, given the model of the environment and other agents is correct. However, our particular setup also reveals that this integrated approach could result in worse performance when the model of other agents is incorrect. Our findings indicate the benefit of integrating planning and RL in partially observable, multi-agent domains, while serving to highlight several important directions for future research. Code available at: https://github.com/RDLLab/posggym .

Multi-task missions for unmanned aerial vehicles (UAVs) involving inspection and landing tasks are challenging for novice pilots due to the difficulties associated with depth perception and the control interface. We propose a shared autonomy system, alongside supplementary information displays, to assist pilots to successfully complete multi-task missions without any pilot training. Our approach comprises of three modules: (1) a perception module that encodes visual information onto a latent representation, (2) a policy module that augments pilot’s actions, and (3) an information augmentation module that provides additional information to the pilot. The policy module is trained in simulation with simulated users and transferred to the real world without modification in a user study ( \(n=29\) ), alongside alternative supplementary information schemes including learnt red/green light feedback cues and an augmented reality display. The pilot’s intent is unknown to the policy module and is inferred from the pilot’s input and UAV’s states. The assistant increased task success rate for the landing and inspection tasks from 16.67% and 54.29%, respectively, to 95.59% and 96.22%. With the assistant, inexperienced pilots achieved similar performance to experienced pilots. Red/green light feedback cues reduced the required time by 19.53% and trajectory length by 17.86% for the inspection task, where participants rated it as their preferred condition due to the intuitive interface and providing reassurance. This work demonstrates that simple user models can train shared autonomy systems in simulation, and transfer to physical tasks to estimate user intent and provide effective assistance and information to the pilot.

Teachable robots in education have the ability to increase student engagement and learning through personalised interactions and the use of social behaviours, such as speech, gaze and emotional expressions. Adaptation of these behaviours is motivated by an interest in tailoring the learning experience to an individual user and improving user outcomes. This work proposes an adaptive response-selection algorithm for a teachable robot which aims to increase user task engagement. A Q-learning algorithm learns an individualised policy and is rewarded based on the user’s paraphrasing behaviour and response time when teaching the robot. A user study is conducted across two user groups, recruited from Australia and Japan. This study evaluates the performance of the adaptive approach for response selection against a non-adaptive method and explores the differences in response and perception of the teaching task between the two participant groups. The results show that measures of task engagement increase more when using the adaptive approach compared to a non-adaptive method of response selection, but that this difference is not consistent across both participant groups. The adaptive approach is also shown to have a positive effect on user perceptions of the interaction.

Human operators in human-robot teams are commonly perceived to be critical for mission success. To explore the direct and perceived impact of operator input on task success and team performance, 16 real-world missions (10 h) were conducted based on the DARPA Subterranean Challenge. Missions involved deploying a heterogeneous team of robots to locate and identify artefacts such as climbing rope, drills and a mannequin representing a human survivor. Two conditions were evaluated: human operators that could control the robot team with state-of-the-art autonomy (Human-Robot Team) compared to autonomous missions without human operator input (Robot-Autonomy). Human interventions included creating waypoints to prioritise high-yield areas, and to navigate through error-prone spaces. Human-Robot Teams were often in directed autonomy mode (70% of mission time), found more items ( \(+\) 10.52%), traversed more distance ( \(+\) 12.71%), covered more unique ground ( \(+\) 10.56%), and longer time between safety-related events (34%). In routine conditions, both condition scores were comparable for artefacts, distance and coverage. Human-Robot Teams were faster at finding the first artefact but slower to respond to information from the robot team. Overall, operators contribute to mission-based outcomes, help to overcome environmental situations that can impede progress, and can assist robots to recover faster from difficult events.

Learning from Demonstration (LfD) is a framework that allows lay users to easily program robots. However, the efficiency of robot learning and the robot’s ability to generalize to task variations hinge upon the quality and quantity of the provided demonstrations. Our objective is to guide human teachers to provide more effective demonstrations, thus facilitating efficient robot learning. To achieve this, we propose to use a measure of uncertainty, namely task-related information entropy , as a criterion for suggesting informative demonstration examples to human teachers to improve their teaching skills. This approach seeks to minimize the requisite number of demonstrations by enhancing their distribution throughout the workspace. In a conducted experiment \((N = 24)\) , an augmented reality (AR)-based guidance system was employed to train novice users to produce additional demonstrations from areas with the highest entropy within the workspace. These novice users were trained for a few trials to teach the robot a generalizable task using a limited number of demonstrations. Subsequently, the users’ performance after training was assessed first on the same task (retention) and then on a new task (transfer) without guidance. The results indicate a substantial improvement in robot learning efficiency from the teacher’s demonstrations, with an improvement of up to 198% observed on the novel task. Furthermore, the proposed approach was compared to a state-of-the-art heuristic rule and found to improve robot learning efficiency by 210% compared to the heuristic rule. The scripts used in this article are available on GitHub .

Facial expression recognition (FER) has emerged as a promising approach to the development of emotion-aware intelligent systems. The performance of FER in multiple domains is continuously being improved, especially through advancements in data-driven learning approaches. However, a key challenge remains in utilizing FER in real-world contexts, namely ensuring user understanding of these systems and establishing a suitable level of user trust towards this technology. We conducted an empirical user study to investigate how explanations of FER can improve trust, understanding and performance in a human-computer interaction task that uses FER to trigger helpful hints during a navigation game. Our results showed that users provided with explanations of the FER system demonstrated improved control in using the system to their advantage, leading to a significant improvement in their understanding of the system, reduced collisions in the navigation game, as well as increased trust towards the system.

Multi-robot supervision becomes increasingly cognitively demanding as the ratio of robots to human supervisors rises, potentially leading to situational awareness (SA) losses and robot system failures. Nonverbal cues have been employed to direct supervisor attention and prevent awareness loss in diverse human-computer interaction (HCI) settings. This paper compares the effects of uni-modal and multi-modal audiovisual nonverbal cues on supervisor SA in a multi-robot supervision task. In a simulation-based navigation scenario, 50 participants monitored a multi-robot mission and responded to supervision requests from the robots. We evaluated supervisor SA using response speed and the situational awareness global assessment technique. Results demonstrate that supervisor awareness hinges on the communication method employed by the robots, with greater significance observed at higher awareness levels and when the robot-to-human ratio is higher. Findings also indicate the utility of sonification mapping in human-multirobot interactions and the benefits of multi-modal cues for sustaining awareness during multi-robot supervision.

In agricultural robotics, effective observation and localization of fruits present challenges due to occlusions caused by other parts of the tree, such as branches and leaves. These occlusions can result in false fruit localization or impede the robot from picking the fruit. The objective of this work is to push away branches that block the fruit’s view to increase their visibility. Our setup consists of an RGB-D camera and a robot arm. First, we detect the occluded fruit in the RGB image and estimate its occluded part via a deep learning generative model in the depth space. The direction to push to clear the occlusions is determined using classic image processing techniques. We then introduce a 3D extension of the 2D Hough transform to detect straight line segments in the point cloud. This extension helps detect tree branches and identify the one mainly responsible for the occlusion. Finally, we clear the occlusion by pushing the branch with the robot arm. Our method uses a combination of deep learning for fruit appearance estimation, classic image processing for push direction determination, and 3D Hough transform for branch detection. We validate our perception methods through real data under different lighting conditions and various types of fruits (i.e. apple, lemon, orange), achieving improved visibility and successful occlusion clearance. We demonstrate the practical application of our approach through a real robot branch pushing demonstration.

Service robots are increasingly employed in the hospitality industry for delivering food orders in restaurants. However, in current practice the robot often arrives at a fixed location for each table when delivering orders to different patrons in the same dining group, thus requiring a human staff member or the customers themselves to identify and retrieve each order. This study investigates how to improve the robot’s service behaviours to facilitate clear intention communication to a group of users, thus achieving accurate delivery and positive user experiences. Specifically, we conduct user studies (N=30) with a Temi service robot as a representative delivery robot currently adopted in restaurants. We investigated two factors in the robot’s intent communication, namely visualisation and movement trajectories, and their influence on the objective and subjective interaction outcomes. A robot personalising its movement trajectory and stopping location in addition to displaying a visualisation of the order yields more accurate intent communication and successful order delivery, as well as more positive user perception towards the robot and its service. Our results also showed that individuals in a group have different interaction experiences.

Robots are inherently imperfect, and collaborating with an error-prone robotic teammate can deteriorate perceptions of trust and the willingness of users to continue working with the robot. Evidence-based trust repair strategies can be implemented into a robot’s design to mitigate the decline of trust in human-robot relationships following errors. It is not yet clear what trust repair strategies are most effective. To address this shortcoming, this study investigates two novel trust repair strategies: offered and automatic physical repair. A between-subjects repeated measures study was performed to determine the extent to which each type of physical trust repair was successful in restoring participants’ perceptions of trust. The results indicated that, where the no-repair condition experienced a significant decrease in trust score, only the automatic repair was consistently successful in bypassing the trust decline. Detailed analysis showed that participants from the offered repair condition did not view the robot as providing the appropriate information, meaning that the offer itself may have confused them. Participants’ response rate to the MultiDimensional Measure of Trust also revealed that users were less willing to associate moral terms with robotic teammates, though this hesitancy may reduce over time. These results contribute to research on human-robot trust repair by uncovering that physical repair is effective when it is automatic, but not when it is offered. This finding will help to further elucidate what repair strategies work to mitigate trust decline and thus help inform robot design.

We study human-robot handovers in a naturalistic collaboration scenario, where a mobile manipulator robot assists a person during a crafting session by providing and retrieving objects used for wooden piece assembly (functional activities) and painting (creative activities). We collect quantitative and qualitative data from 20 participants in a Wizard-of-Oz study, generating the Functional And Creative Tasks Human-Robot Collaboration dataset (the FACT HRC dataset), available to the research community. This work illustrates how social cues and task context inform the temporal-spatial coordination in human-robot handovers, and how human-robot collaboration is shaped by and in turn influences people's functional and creative activities.

We propose a robotic manipulation method that can pivot objects on a surface using vision, wrist force and tactile sensing. We aim to control the rotation of an object around the grip point of a parallel gripper by allowing rotational slip, while maintaining a desired wrist force profile. Our approach runs an end-effector position controller and a gripper width controller concurrently in a closed loop. The position controller maintains a desired force using vision and wrist force. The gripper controller uses tactile sensing to keep the grip firm enough to prevent translational slip, but loose enough to allow rotational slip. Our sensor-based control approach relies on matching a desired force profile derived from object dimensions and weight, as well as vision-based monitoring of the object pose. The gripper controller uses tactile sensors to detect and prevent translational slip by tightening the grip when needed. Experimental results where the robot was tasked with rotating cuboid objects 90 degrees show that the multi-modal pivoting approach was able to rotate the objects without causing lift or translational slip, and was more energy-efficient compared to using a single sensor modality or pick-and-place.

Teachable robots can offer benefits to students through the use of social behaviours, such as speech, gaze, and gestures, to promote engagement and learning. Adapting these behaviours can deliver personalised interactions to better suit each individual. There is a growing body of research utilising reinforcement learning in social robotics, however there is limited research in the use of adaptive dialog behaviours for social robots. We propose an adaptive response-selection algorithm for a teachable robot which aims to improve user engagement in the teaching task. The proposed approach uses Q-learning to learn an individualised policy. The algorithm is rewarded according to the time taken per teaching input, and the amount paraphrasing in the user’s response. A user study has been conducted to evaluate the algorithm, compared to a method of random response-selection. The results indicate that an adaptive approach learns to select more rewarding actions over time, and personalise to the individual user.

A human supervisor’s Situational Awareness (SA) is a critical aspect for successful Human-Robot Teaming (HRT). SA has been estimated using different techniques; however, many of those are associated with various biases, including recall and overgeneralisation biases. A key SA metric is latency, the delay between the time the robotic system requires supervisor assistance and the time the supervisor identifies that need in HRT experiments. Eye movements are increasingly used to assess SA across a range of domains, enabling objective and continuous SA assessment. However, to date, only a small number of features have been evaluated for estimating different types of SA latencies. In this paper, we investigated how two types of SA latencies (perceptual and comprehending) correlate with eye movement data collected during a remote field experiment, where a human supervisor directed a team of robots in a smart farming context. We identified 39 instances of SA latencies (13 perceptual and 26 comprehending). These instances were used to identify how a human supervisor’s SA is affected by task context, and to evaluate correlations between five eye movement features and SA latencies. Two eye movement features related to fixation duration and saccade duration demonstrated very strong correlations ($r \approx - 0.8$ and $r \approx 0.85$). Our findings can be extended to estimate the real-time likelihood of the human experiencing SA latency.

Robotic vision, otherwise known as computer vision for robots, is a critical process for robots to collect and interpret detailed information related to human actions, goals, and preferences, enabling robots to provide more useful services to people. This survey and systematic review presents a comprehensive analysis on robotic vision in human-robot interaction and collaboration (HRI/C) over the past 10 years. From a detailed search of 3,850 articles, systematic extraction and evaluation was used to identify and explore 310 papers in depth. These papers described robots with some level of autonomy using robotic vision for locomotion, manipulation, and/or visual communication to collaborate or interact with people. This article provides an in-depth analysis of current trends, common domains, methods and procedures, technical processes, datasets and models, experimental testing, sample populations, performance metrics, and future challenges. Robotic vision was often used in action and gesture recognition, robot movement in human spaces, object handover and collaborative actions, social communication, and learning from demonstration. Few high-impact and novel techniques from the computer vision field had been translated into HRI/C. Overall, notable advancements have been made on how to develop and deploy robots to assist people.

When a robot learns from human examples, most approaches assume that the human partner provides examples of optimal behavior. However, there are applications in which the robot learns from nonexpert humans. We argue that the robot should learn not only about the human's objectives, but also about their expertise level. The robot could then leverage this joint information to reduce or increase the frequency at which it provides assistance to its human's partner or be more cautious when learning new skills from novice users. Similarly, by taking into account the human's expertise, the robot would also be able to infer a human's true objectives even when the human fails to properly demonstrate these objectives due to a lack of expertise. In this article, we propose to jointly infer the expertise level and the objective function of a human given observations of their (possibly) nonoptimal demonstrations. Two inference approaches are proposed. In the first approach, inference is done over a finite discrete set of possible objective functions and expertise levels. In the second approach, the robot optimizes over the space of all possible hypotheses and finds the objective function and the expertise level that best explain the observed human behavior. We demonstrate our proposed approaches both in simulation and with real user data.

In this article, we develop the method of continuous Pontryagin differentiable programming (Continuous PDP), which enables a robot to learn an objective function from a few sparsely demonstrated keyframes. The keyframes, labeled with some time stamps, are the desired task-space outputs, which a robot is expected to follow sequentially. The time stamps of the keyframes can be different from the time of the robot's actual execution. The method jointly finds an objective function and a time-warping function such that the robot's resulting trajectory sequentially follows the keyframes with minimal discrepancy loss. The Continuous PDP minimizes the discrepancy loss using projected gradient descent by efficiently solving the gradient of the robot trajectory with respect to the unknown parameters. The method is first evaluated on a simulated robot arm and then applied to a 6-DoF quadrotor to learn an objective function for motion planning in unmodeled environments. The results show the efficiency of the method, its ability to handle time misalignment between keyframes and robot execution, and the generalization of objective learning into unseen motion conditions.

Existing approaches to direct robot-to-human handovers are typically implemented on fixed-base robot arms, or on mobile manipulators that come to a full stop before performing the handover. We propose "on-the-go" handovers which permit a moving mobile manipulator to hand over an object to a human without stopping. The on-the-go handover motion is generated with a reactive controller that allows simultaneous control of the base and the arm. In a user study, human receivers subjectively assessed on-the-go handovers to be more efficient, predictable, natural, better timed and safer than handovers that implemented a "stop-and-deliver" behavior.

The use of hand-based gestures has been proposed as an intuitive way for people to communicate with robots. Typically the set of gestures is defined by the experimenter. However, existing works do not necessarily focus on gestures that are communicative, and it is unclear whether the selected gesture are actually intuitive to users. This paper investigates whether different people inherently use similar gestures to convey the same commands to robots, and how teaching of gestures when collecting demonstrations for training recognizers can improve resulting accuracy. We conducted this work in two stages. In Stage 1, we conducted an online user study (n=190) to investigate if people use similar gestures to communicate the same set of given commands to a robot when no guidance or training was given. Results revealed large variations in the gestures used among individuals With the absences of training. Training a gesture recognizer using this dataset resulted in an accuracy of around 20%. In response to this, Stage 2 involved proposing a common set of gestures for the commands. We taught these gestures through demonstrations and collected ~ 7500 videos of gestures from study participants to train another gesture recognition model. Initial results showed improved accuracy but a number of gestures had high confusion rates. Refining our gesture set and recognition model by removing those gestures, We achieved an final accuracy of 84.1 ± 2.4%. We integrated the gesture recognition model into the ROS framework and demonstrated a use case, where a person commands a robot to perform a pick and place task using the gesture set.

Research in biomechanics hypothesizes that human motion is optimal with respect to an unknown cost function that varies depending on the action and/or task. This unknown cost function is often approximated as the weighted sum of a set of features or basis cost functions. As a person performs a sequence of actions, the weights associated to each of these basis functions are likely to vary over time. Given a human demonstration and the corresponding cost weight trajectory recovered via inverse optimal control (IOC), this paper proposes an optimal control (OC) method that can generate robot motion based on human movement using time-varying cost function weights. By using time-varying weights, the proposed optimal control method can handle changing optimization criteria without segmentation. The method is evaluated both in simulation and with recorded human data. Using human demonstration data, we demonstrate the reproduction of pick-and-place motions with an average end-effector error at the pick place location within 0.82 cm, which is significantly lower than the average trajectory error, indicating that the approach correctly prioritizes reaching the pick and place locations without manual segmentation.

A promising characteristic of Deep Reinforcement Learning (DRL) is its capability to learn optimal policy in an end-to-end manner without relying on feature engineering. However, most approaches assume a fully observable state space, i.e. fully observable Markov Decision Processes (MDPs). In real-world robotics, this assumption is unpractical, because of issues such as sensor sensitivity limitations and sensor noise, and the lack of knowledge about whether the observation design is complete or not. These scenarios lead to Partially Observable MDPs (POMDPs). In this paper, we propose Long-Short-Term-Memory-based Twin Delayed Deep Deterministic Policy Gradient (LSTM-TD3) by introducing a memory component to TD3, and compare its performance with other DRL algorithms in both MDPs and POMDPs. Our results demonstrate the significant advantages of the memory component in addressing POMDPs, including the ability to handle missing and noisy observation data.

The aim of this study was to determine whether reinforcement learning could increase user engagement in interactive art installations. Building on a physical interactive art installation called The Plants [1] by Playable Streets, reinforcement learning was integrated into a web application adapted from the physical interactive art piece. The original installation consisted of real plants that visitors could touch to produce sounds. A digital model and interface was developed to simulate the physical installation. A user study was conducted with 178 participants. Three modes were examined: the original settings of the installation as designed by the artist; a predetermined fixed schedule of consistently changing sound banks; and a reinforcement learning mode, where an agent changes the interactive behaviours to maximise user engagement. User engagement was estimated by comparing the number of touches by an individual over successive time intervals.

A robot’s perceived assertiveness can influence how people assess its credibility and their willingness to comply with its suggestions during human-robot interaction. This study proposes a novel measurement of the perceived assertiveness of a robot using both objective assessment of a person’s compliance to a robot’s suggestions in a math quiz task and subjective assessments of a person’s perception and expectation of a robot in different hypothetical scenarios. The proposed measurement evaluates perceived assertiveness of a robot in three dimensions inspired by social behavioral studies on human-human interaction, namely social assertiveness, directiveness, and independence. We conducted an exploratory study using crowdsourcing to test the efficacy of this proposed measurement of perceived assertiveness. In particular, participants were exposed to robots that differed in terms of their anthropomorphism and their communicative styles. The communicative styles are designed following human studies of behaviors that are commonly associated with high or low assertiveness. Our results demonstrated the validity of the proposed measurement of perceived assertiveness. The observed discrepancy between the objective and subjective measurements highlights the necessity of evaluating human perception through multiple approaches. Moreover, gaps between intended and perceived robot assertiveness indicate a potential gap between human communication theories and their applicability to human-robot interaction.

This paper proposes a human-aware motion planner building on RRT-Connect, dubbed Human-Aware RRT-Connect. The planner considers a composite cost function that includes four criteria: human separation distance, human-robot center of mass distance, robot inertia and visibility. This choice of criteria ensures the robot maintains a safe distance and low inertia during motion while being as visible as possible to the human. A simulation study is conducted to demonstrate the planner performance. For the simulation study, the proposed offline Human-Aware RRT-Connect planner is compared to other offline planners through a set of scenarios that vary in environment and task complexity. Several human-robot configurations are tested in a shared workspace involving a simulated Franka Emika Panda arm and a human model. The paths generated by the Human-Aware RRT-Connect planner maintain larger separation distances from the human, are of lower inertia, and are more visible.

In this work, we designed a teachable robot that encourages a pair of students to discuss their thoughts and teaching decisions during the tutoring session. The robot adapts to the students’ talking activity and adjusts the frequency and type of encouragement. We hypothesize that the robot’s encouragement of group discussion can enhance the social engagement of group members, leading to improved learning and enjoyment. We ran a user study (n = 68), where a pair of participants (dyad) worked together to teach a humanoid robot about rocks and minerals. In the adaptive condition, the robot uses reinforcement learning to maximise interaction between the dyad members. Results show that the adaptive robot was successful in creating more dialogue between dyad members and in increasing task engagement, but did not affect learning or enjoyment. Over time, the adaptive robot was also able to encourage both members to contribute more equally to the conversation.

Physical agents that can autonomously generate engaging, life-like behavior will lead to more responsive and user-friendly robots and other autonomous systems. Although many advances have been made for one-to-one interactions in well-controlled settings, physical agents should be capable of interacting with humans in natural settings, including group interaction. To generate engaging behaviors, the autonomous system must first be able to estimate its human partners’ engagement level. In this article, we propose an approach for estimating engagement during group interaction by simultaneously taking into account active and passive interaction, and use the measure as the reward signal within a reinforcement learning framework to learn engaging interactive behaviors. The proposed approach is implemented in an interactive sculptural system in a museum setting. We compare the learning system to a baseline using pre-scripted interactive behaviors. Analysis based on sensory data and survey data shows that adaptable behaviors within an expert-designed action space can achieve higher engagement and likeability.

This article surveys the literature on human–robot object handovers. A handover is a collaborative joint action, where an agent, the giver, gives an object to another agent, the receiver. The physical exchange starts when the receiver first contacts the object held by the giver and ends when the giver fully releases the object to the receiver. However, important cognitive and physical processes begin before the physical exchange, including initiating implicit agreement with respect to the location and timing of the exchange. From this perspective, we structure our review into the two main phases delimited by the aforementioned events: a prehandover phase and the physical exchange. We focus our analysis on the two actors (giver and receiver) and report the state of the art of robotic givers (robot-to-human handovers) and the robotic receivers (human-to-robot handovers). We report a comprehensive list of qualitative and quantitative metrics commonly used to assess the interaction. While focusing our review on the cognitive level (e.g., prediction, perception, motion planning, and learning) and the physical level (e.g., motion, grasping, and grip release) of the handover, we also discuss safety. We compare the behaviors displayed during human-to-human handovers to the state of the art of robotic assistants and identify the major areas of improvement for robotic assistants to reach performance comparable to human interactions. Finally, we propose a minimal set of metrics that should be used in order to enable a fair comparison among the approaches.

Multi-step (also called n-step) methods in Reinforcement Learning (RL) have been shown to be more efficient than the 1-step method due to faster propagation of the reward signal, both theoretically and empirically, in tasks exploiting tabular representation of the value-function. Recently, research in Deep Reinforcement Learning (DRL) also shows that multi-step methods improve learning speed and final performance in applications where the value-function and policy are represented with deep neural networks. However, there is a lack of understanding about what is actually contributing to the boost of performance. In this work, we analyze the effect of multi-step methods on alleviating the overestimation problem in DRL, where multi-step experiences are sampled from a replay buffer. Specifically building on top of Deep Deterministic Policy Gradient (DDPG), we propose Multi- step DDPG (MDDPG), where different step sizes are manually set, and a variant called Mixed Multi-step DDPG (MMDDPG) where an average over different multi-step backups is used as an update target for the Q-value function. Empirically, we show that both MDDPG and MMDDPG are significantly less affected by the overestimation problem than DDPG with 1-step backup, which consequently results in better final performance and learning speed. We also discuss the advantages and disadvantages of different ways to do multi-step expansion in order to reduce approximation error, and expose the tradeoff between overestimation and underestimation that underlies offline multi-step methods. Finally, we compare the computational resource needs of MDDPG and MMDDPG with those of Twin Delayed Deep Deterministic Policy Gradient (TD3), a state-of-the-art algorithm proposed to address overestimation in actor-critic methods, since they show comparable final performance and learning speed.

Enabling a broader range of users to efficiently deploy autonomous mobile robots requires intuitive frameworks for specifying a robot's task and behaviour. We present a novel approach using learning from corrections (LfC), where a user is iteratively presented with a solution to a motion planning problem. Users might have preferences about parts of a robot's environment that are suitable for robot traffic or that should be avoided as well as preferences on the control actions a robot can take. The robot is initially unaware of these preferences; thus, we ask the user to provide a correction to the presented path. We assume that the user evaluates paths based on environment and motion features. From a sequence of corrections we learn weights for these features, which are then considered by the motion planner, resulting in future paths that better fit the user's preferences. We prove completeness of our algorithm and demonstrate its performance in simulations. Thereby, we show that the learned preferences yield good results not only for a set of training tasks but also for test tasks, as well as for different types of user behaviour.

The increasing presence of robots alongside humans, such as in human-robot teams in manufacturing, gives rise to research questions about the kind of behaviors people prefer in their robot counterparts. We term actions that support interaction by reducing future interference with others as supportive robot actions and investigate their utility in a co-located manipulation scenario. We compare two robot modes in a shared table pick-and-place task: (1) Task-oriented: the robot only takes actions to further its task objective and (2) Supportive: the robot sometimes prefers supportive actions to task-oriented ones when they reduce future goal-conflicts. Our experiments in simulation, using a simplified human model, reveal that supportive actions reduce the interference between agents, especially in more difficult tasks, but also cause the robot to take longer to complete the task. We implemented these modes on a physical robot in a user study where a human and a robot perform object placement on a shared table. Our results show that a supportive robot was perceived more favorably as a coworker and also reduced interference with the human in one of two scenarios. However, it also took longer to complete the task highlighting an interesting trade-off between task-efficiency and human-preference that needs to be considered before designing robot behavior for close-proximity manipulation scenarios.

Human-Robot Interaction (HRI) user studies are challenging to evaluate and compare due to a lack of standardization and the infrastructure required to implement each study. The lack of experimental infrastructure also makes it difficult to systematically evaluate the impact of individual components (e.g., the quality of perception software) on overall system performance. This work proposes a framework to ease the implementation and reproducibility of human-robot interaction user studies. The framework utilizes ROS middleware and is implemented with four modules: perception, decision, action, and metrics. The perception module aggregates sensor data to be used by the decision and action modules. The decision module is the task-level executive and can be designed by the HRI researcher for their specific task. The action module takes subtask requests from the decision module and breaks them down into motion primitives for execution on the robot. The metrics module tracks and generates quantitative metrics for the study. The framework is implemented with modular interfaces to allow for alternate implementations within each module and can be generalized for a variety of tasks and human/robot roles. The framework is illustrated through an example scenario involving a human and a Franka Emika Panda arm collaboratively assembling a toolbox together.

We study active preference learning as a frame-work for intuitively specifying the behaviour of autonomous robots. A user chooses the preferred behaviour from a set of alternatives, from which the robot learns the user's preferences, modeled as a parameterized cost function. Previous approaches present users with alternatives that minimize the uncertainty over the parameters of the cost function. However, different parameters might lead to the same optimal behaviour; as a consequence the solution space is more structured than the parameter space. We exploit this by proposing a query selection that greedily reduces the maximum error ratio over the solution space. In simulations we demonstrate that the proposed approach outperforms other state of the art techniques in both learning efficiency and ease of queries for the user. Finally, we show that evaluating the learning based on the similarities of solutions instead of the similarities of weights allows for better predictions for different scenarios.

This article proposes a framework for human-pose estimation from the wearable sensors that rely on a Lie group representation to model the geometry of the human movement. Human body joints are modeled by matrix Lie groups, using special orthogonal groups SO(2) and SO(3) for joint pose and special Euclidean group SE(3) for base-link pose representation. To estimate the human joint pose, velocity, and acceleration, we develop the equations for employing the extended Kalman filter on Lie groups (LG-EKF) to explicitly account for the non-Euclidean geometry of the state space. We present the observability analysis of an arbitrarily long kinematic chain of SO(3) elements based on a differential geometric approach, representing a generalization of kinematic chains of a human body. The observability is investigated for the system using marker position measurements. The proposed algorithm is compared with two competing approaches: 1) the extended Kalman filter (EKF) and 2) unscented KF (UKF) based on the Euler angle parametrization, in both simulations and extensive real-world experiments. The results show that the proposed approach achieves significant improvements over the Euler angle-based filters. It provides more accurate pose estimates, is not sensitive to gimbal lock, and more consistently estimates the covariances.

Curiosity-the intrinsic desire for new information-can enhance learning, memory, and exploration. Therefore, understanding how to elicit curiosity can inform the design of educational technologies. In this work, we investigate how a social peer robot's verbal expression of curiosity is perceived, whether it can affect the emotional feeling and behavioural expression of curiosity in students, and how it impacts learning. In a between-subjects experiment, 30 participants played the game LinkIt!, a game we designed for teaching rock classification, with a robot verbally expressing: curiosity, curiosity plus rationale, or no curiosity. Results indicate that participants could recognize the robot's curiosity and that curious robots produced both emotional and behavioural curiosity contagion effects in participants.

Estimating a user's expertise level based on observations of their actions will result in better human-robot collaboration, by enabling the robot to adjust its behaviour and the assistance it provides according to the skills of the particular user it's interacting with. This paper details an approach to incrementally and continually estimate the expertise of a user whose goal is to optimally complete a given task. The user's expertise level, here represented as a scalar parameter, is estimated by evaluating how far their actions are from optimal. The proposed approach was tested using data from an online study where participants were asked to complete various instances of a simulated kitting task. An optimal planner was used to estimate the “goodness” of all available actions at any given task state. We found that our expertise level estimates correlate strongly with observed after-task performance metrics and that it is possible to differentiate novices from experts after observing, on average, 33% of the errors made by the novices.

Learning from human interaction data is a promising approach for developing robot interaction logic, but behaviors learned only from offline data simply represent the most frequent interaction patterns in the training data, without any adaptation for individual differences. We developed a robot that incorporates both data-driven and interactive learning. Our robot first learns high-level dialog and spatial behavior patterns from offline examples of human--human interaction. Then, during live interactions, it chooses among appropriate actions according to its curiosity about the customer's expected behavior, continually updating its predictive model to learn and adapt to each individual. In a user study, we found that participants thought the curious robot was significantly more humanlike with respect to repetitiveness and diversity of behavior, more interesting, and better overall in comparison to a non-curious robot.

Robots that have different forms and capabilities are used in a wide variety of situations; however, one common point to all robots interacting with humans is their ability to communicate with them. In addition to verbal communication or purely communicative movements, robots can also use their embodiment to generate expressive movements while achieving a task, to convey additional information to its human partner. This article surveys state-of-the-art techniques that generate whole-body expressive movements in robots and robot avatars. We consider different embodiments such as wheeled, legged, or flying systems and the different metrics used to evaluate the generated movements. Finally, we discuss future areas of improvement and the difficulties to overcome to develop truly expressive motions in artificial agents.

In this paper, we consider a dynamical system whose trajectory is a result of minimizing a multiphase cost function. The multiphase cost function is assumed to be a weighted sum of specified features (or basis functions) with phase-dependent weights that switch at some unknown phase transition points. A new inverse optimal control approach for recovering the cost weights of each phase and estimating the phase transition points is proposed. The key idea is to use a length-adapted window moving along the observed trajectory, where the window length is determined by finding the minimal observation length that suffices for a successful cost weight recovery. The effectiveness of the proposed method is first evaluated on a simulated robot arm, and then, demonstrated on a dataset of human participants performing a series of squatting tasks. The results demonstrate that the proposed method reliably retrieves the cost function of each phase and segments each phase of motion from the trajectory with a segmentation accuracy above 90%.

One of the main challenges of bipedal gait is to avoid falling due to unknown disturbances. Compensating for these disturbances in bipeds is often achieved by leaning or stepping. In this work, the Spherical Foot Placement Estimator (SFPE) is introduced, which uses the biped's current kinematics and dynamics to predict if a step is needed, and if so where to step, to restore balance in 3D. An example of a controller using the SFPE is shown, which augments an existing optimal controller with both leaning and stepping: SFPE-based feedback is used to generate a desired momentum for momentum-based leaning while the SFPE point is used as a control reference for stepping. The new estimator outperforms existing balance criteria by providing both recovery step location prediction and momentum objectives with smooth dynamics.

In this paper we develop an approach for learning user preferences for complex task specifications through human-robot interaction. We consider the problem of planning robot motion in a known environment, but where a user has specified additional spatial and temporal constraints on allowable robot motions. To illustrate the impact of the user's constraints on performance, we iteratively present users with alternative solutions on an interface. The user provides a ranking of alternate paths, and from this we learn about the importance of different constraints. This allows for an accessible method for specifying complex robot tasks. We present an algorithm that iteratively builds a set of constraints on the relative importance of each user constraint, and prove that with sufficient interaction, the algorithm determines a user-optimal path. We demonstrate the practical performance by simulating realistic material transport scenarios in industrial facilities.

As robots' capability and autonomy improve, they are expected to increasingly operate in human environments, and interact with novice, untrained users. When robots operate in human or shared environments, their tasks and behaviours need to be specified; this task is typically performed by a human operator or supervisor. The human operator may specify constraints on robot behaviour to make the robot more predictable or align its behaviour with user expectations. However, these constraints may impact robot task performance. This paper investigates how novice users generate robot specifications and proposes metrics for quantifying specification quality. The proposed approach is evaluated with a user study, where novice users provide specifications for an autonomous robot operating in a shared warehouse environment. We find that untrained users create a wide variety of behaviour-limiting specifications, that users generally have difficulty creating efficient specifications, and that they were not able to correctly assess their own performance.

Robots and other autonomous systems interacting with humans should customize their behaviour to their human partner's preferences. We propose a method for learning and generating robot movement customized to individual preferences. Within a reinforcement learning framework, we generate rewards based on facial expressions observed during the robot's motion. Robot motions are parametrized; the rewards are used to modify these motion parameters using Q learning. The proposed approach is evaluated in a user study, using an interactive kinetic sculpture. The system interacts with participants and evolves its motion based on the rewards estimated from the participants' facial expressions. Our results show that, for a subset of participants, the system was able to successfully generate actions that resulted in higher than random rewards. The ability to successfully generate high-reward actions depends on: being able to recognize positive affect from the face, being able to generate actions that are pleasing to the participant, and being able to learn the mapping from rewards to actions.

Driver distraction is one of the leading causes of vehicle accidents and injury. Automated systems for identifying driver distraction are of great interest for improving road safety. This study develops a smartphone sensor based driver distraction system using an ensemble learning method. After data collection, linear velocity data is first linearly interpolated. Then, 3-axial acceleration and 3-axial gyro signals are filtered for reducing noise. Next, a sliding window is applied to IMU and GPS data collected by the smartphone for feature extraction, where temporal features are calculated. Ensemble learning of four standard classifiers is used to recognize distraction events: K-Nearest Neighbor, Logistic Regression, Gaussian Naive Bayes, Random Forest. To evaluate the proposed approach, 24 drivers were recruited to participate in a user study, driving on a route consisting of suburban and highway driving. Driver cognitive distraction was induced by asking the driver questions while driving. The experimental results show that the best weighted F1-score of our proposed system is 87% with all smartphone sensor signals.

While convolutional neural networks (CNNs) have been successfully applied to many challenging classification applications, they typically require large datasets for training. When the availability of labeled data is limited, data augmentation is a critical preprocessing step for CNNs. However, data augmentation for wearable sensor data has not been deeply investigated yet.

This paper proposes a new algorithm for human motion estimation using inertial measurement unit (IMU) measurements. We model the joints by matrix Lie groups, namely the special orthogonal groups SO(2) and SO(3), representing rotations in 2D and 3D space, respectively. The state space is defined by the Cartesian product of the rotation groups and their velocities and accelerations, given a kinematic model of the articulated body. In order to estimate the state, we propose the Lie Group Extended Kalman Filter (LG-EKF), thus explicitly accounting for the non-Euclidean geometry of the state space, and we derive the LG-EKF recursion for articulated motion estimation based on IMU measurements. The performance of the proposed algorithm is compared to the EKF based on Euler angle parametrization in both simulation and real-world experiments. The results show that for motion near gimbal lock regions, which is common for shoulder movement, the proposed filter is a significant improvement over the Euler angles EKF.

Inertial measurement units (IMUs) enable human motion measurement in any environment, which can be useful for human robot interaction, exoskeletons, and active prosthetics. This paper proposes an approach for estimating the orientation between a wearable IMU sensor and the body frame of the wearer using a simple and fast calibration procedure. The proposed approach uses the generalized Hebbian algorithm to incrementally estimate the axis aligned with gravity using acceleration measurements obtained during a static pose, and the axis perpendicular to the saggital plane using gyro measurements obtained during sagittal plane movements. An automated convergence criterion based on the sensor measurement variance is used. The proposed approach is tested in simulation and with human movement and demonstrates excellent and fast calibration performance.

The ability to accurately identify human activities is essential for developing automatic rehabilitation and sports training systems. In this paper, large-scale exercise motion data obtained from a forearm-worn wearable sensor are classified with a convolutional neural network (CNN). Time-series data consisting of accelerometer and orientation measurements are formatted as images, allowing the CNN to automatically extract discriminative features. A comparative study on the effects of image formatting and different CNN architectures is also presented. The best performing configuration classifies 50 gym exercises with 92.1% accuracy.

Service robots are becoming a widespread tool for assisting humans in scientific, industrial and even domestic settings. Yet, our understanding of how to motivate and sustain interactions between human users and robots remains limited. In this work, we conducted a study to investigate how surprising robot behaviour evokes curiosity and influences trust and engagement in the context of participants interacting with Recyclo, a service robot for providing recycling recommendations. In a Wizard-of-Oz experiment, 36 participants were asked to interact with Recyclo to recognize and sort a variety of objects, and were given object recognition responses that were either unsurprising or surprising. Results show that surprise gave rise to information seeking behavior indicative of curiosity, while having a positive influence on engagement and negative influence on trust.

In this paper, we propose a classifier-based approach for driving manoeuvre recognition from mobile phone data. We introduce a driving manoeuvre classifier using Support Vector Machines (SVM). We investigate the performance of a sliding window of velocity and angular velocity signals obtained using a smartphone as features for our classifier. Principal Component Analysis (PCA) is used for dimensionality reduction. The classifiers use a vehicle simulation for training data and experimental data for validation. A novel technique to extract the rotation matrix using PCA is presented to calibrate the smartphone's orientation. A classifier performance of 0.8158 average precision and 0.8279 average recall was achieved resulting in an average F1 score of 0.8194. Balanced accuracy was calculated to be 0.8874.

To enable long term, engaging social human-machine interaction, robots and other autonomous systems must be able to move beyond purely reactive interaction control strategies, and engage in shared-initiative interaction. In this paper, we describe an implementation of an interactive art sculpture which generates interactive behaviours using curiosity-based learning. Using its own internal motivation formulated as a curiosity drive, the system initiates interaction with and responds to human visitors, generating continuously evolving interactive behaviours. The proposed system was tested in a user study with a prototype interactive sculpture installation.

Driver behaviour has a significant influence on vehicle accidents. Measuring and providing feedback on driver behaviour can provide significant benefits for understanding and improving road safety. In order to detect driver actions and driving characteristics from the broadest population of drivers, mobile phones can be used to collect low cost information and provide easy accessibility, using sensors available on the mobile phone such as the GPS and IMU. Such information is collected as a time series dataset, which generally has high dimensional variables. Dealing with this high dimensional data is a crucial problem for statistical analysis. Feature abstraction techniques can reduce the dimensionality by extracting salient features from the dataset. This paper proposes a feature abstraction method using stacked sparse autoencoders in order to reduce driver dataset variables. The utility of the derived features is demonstrated on a driver action classification task.

In current physical rehabilitation protocols, patients typically perform exercises with intermittent feedback or guidance following the initial demonstrations from the physiotherapist. Although many patient-centered systems have been developed for home rehabilitation, few systems have been developed to aid the physiotherapist as well as patients in the rehabilitation clinic. This article proposes the Automated Rehabilitation System (ARS), a system designed specifically for rehabilitation clinics using an iterative design process, developed with physiotherapists and patients in a knee and hip replacement clinic. ARS consists of body-worn inertial measurement units that continuously measure the patient’s pose. The measured pose is graphically represented as an animation and overlaid with the instructed motion on a visual display shown to the patient during exercise performance. ARS allows physiotherapists to quantitatively measure patient movement, assess recovery progress, and manage and schedule exercise regimens for patients. The system requirements and design requirements were derived through a focus group with 13 physiotherapists. For patients, ARS provides visual feedback and a novel exercise guidance feature to aid them while exercising. The patient interface was evaluated in a user study with 26 outpatients. The results show that performing the exercises with the visual guidance tool improves the quality of exercise performance.

Movement primitive segmentation enables long sequences of human movement observation data to be segmented into smaller components, termed movement primitives, to facilitate movement identification, modeling, and learning. It has been applied to exercise monitoring, gesture recognition, human-machine interaction, and robot imitation learning. This paper proposes a segmentation framework to categorize and compare different segmentation algorithms considering segment definitions, data sources, application-specific requirements, algorithm mechanics, and validation techniques. The framework is applied to human motion segmentation methods by grouping them into online, semionline, and offline approaches. Among the online approaches, distance-based methods provide the best performance, while stochastic dynamic models work best in the semionline and offline settings. However, most algorithms to date are tested with small datasets, and algorithm generalization across participants and to movement changes remains largely untested.

The thirteen papers in this special section focus on the topic of movement science for humans and humanoids. The papers include the collection and organization of human movement data for enabling robotics research; the use of human movement as inspiration for humanoid planning, control, and motion generation; the development of algorithms for improved estimation of human and humanoid system parameters; and the use of human movement understanding in robotics applications including human-robot interaction and rehabilitation.

The anthropomorphic body form is a complex articulated system of links/limbs and joints, simultaneously redundant and underactuated, and capable of a wide range of sophisticated movement. The human body and its movement have long been a topic of study in physiology, anatomy, biomechanics, and neuroscience and have served as inspiration for humanoid robot design and control. This survey paper reviews the literature on robotics research using anthropomorphic design principles as an inspiration, at both the design and control levels. Next, anthropomorphic body modeling, motion analysis, and synthesis techniques are overviewed. Finally, key applications arising at the intersection of robotics and human movement science are introduced. The survey ends with a discussion of open research questions and directions for future work.

The ability to engage human observers is a key requirement for both social robots and the arts. In this paper, we propose an approach for adapting the Intelligent Adaptive Curiosity learning algorithm to distributed interactive sculptural systems. This Curiosity-Based Learning Algorithm (CBLA) allows the system to learn about its own mechanisms and its surroundings through self-experimentation and interaction. A novel formulation using multiple agents as learning subsets of the system that communicate through shared input variables enables us to scale to a much larger system with diverse types of sensors and actuators. Experiments on a prototype interactive sculpture demonstrate the exploratory patterns of the CBLA and collective learning behaviours through the integration of multiple learning agents.

In this paper we present a real-time method for identification of the dynamic parameters of a manipulator and its load using kinematic measurements and either joint torques or force and moment at the base. The parameters are estimated using the Extended Kalman Filter and constraints are imposed using Sigmoid functions to ensure the parameters remain within their physically feasible ranges, such as links having positive masses and moments of inertia. Identified parameters can be used in model based controllers. The presented approach is validated through simulation and on data collected with the Barret WAM manipulator. Using the estimated parameters instead of ones provided by the manufacturer greatly improves joint torque prediction.

This paper investigates the design and implementation of a finger-like robotic structure capable of reproducing human hand gestural movements performed by a multi-fingered, hand-like structure. In this work, we present a pneumatic circuit and a closed-loop controller for a finger-like soft pneumatic actuator. Experimental results demonstrate the performance of the pneumatic and control systems of the soft pneumatic actuator, and its ability to track human movement trajectories with affective content.

Soft pneumatic artificial muscles (SPAMs) are a type of pneumatic actuator that provide customizable motion trajectories in three dimensional space without the need for rigid links or a transmission mechanism. This paper presents a novel design for producing SPAMs, named wrapped SPAMs (WSPAMs). Unlike previous SPAM designs, the production process of WSPAM is highly repeatable, while the motion trajectory can be easily modified. A model for predicting the steady-state angular displacement of a WSPAM actuator based on its geometrical parameters and the elasticity of the materials used in its production is presented and experimentally validated.

Path following controllers make the output of a control system approach and traverse a prespecified path with noa prioritime-parametrization. In this paper, we present a method for path following control design applicable to framed curves generated by splines in the workspace of kinematically redundant mechanical systems. The class of admissible paths includes self-intersecting curves. Kinematic redundancies are resolved by designing controllers that solve a suitably defined constrained quadratic optimization problem. By employing partial feedback linearization, the proposed path following controllers have a clear physical meaning. The approach is experimentally verified on a four-degree-of-freedom (four-DOF) manipulator with a combination of revolute and linear actuated links and significant model uncertainty.

Humanoid robots have many potential applications in man-made environments, including performing hazardous tasks, assisting the elderly, and as a replacement for our aging workforce. However, generating a reliable gait for biped robots is challenging, particularly for dynamic gait and in the presence of unknown external disturbances, such as a bump from someone walking by. In this work, a 3D formulation of the Foot Placement Estimator is used with a high-level control strategy to achieve a dynamic gait capable of handling external disturbances. A key benefit of this approach is that the robot is able to respond in real time to external disturbances regardless of whether it is at rest or in motion. This strategy is implemented in simulation to control a 14-DOF lower-body humanoid robot being subjected to unknown external forces, both when at rest and while walking, and shown to generate stabilizing stepping actions.

This work proposes an approach for solving the linear regression problem by maximizing the dependence between prediction values and the response variable. The proposed algorithm uses the Hilbert-Schmidt independence criterion as a generic measure of dependence and can be used to maximize both nonlinear and linear dependencies. The algorithm is important in applications such as continuous analysis of affective speech, where linear dependence, or correlation, is commonly set as the measure of goodness of fit. The applicability of the proposed algorithm is verified using two synthetic, one affective speech, and one affective bodily posture datasets. Experimental results show that the proposed algorithm outperforms support vector regression (SVR) in 84% (264/314) of studied cases, and is noticeably faster than SVR, as an order of 25, on average.

For an engaging human-machine interaction, machines need to be equipped with affective communication abilities. Such abilities enable interactive machines to recognize the affective expressions of their users, and respond appropriately through different modalities including movement. This paper focuses on bodily expressions of affect, and presents a new computational model for affective movement recognition, robust to kinematic, interpersonal, and stochastic variations in affective movements. The proposed approach derives a stochastic model of the affective movement dynamics using hidden Markov models (HMMs). The resulting HMMs are then used to derive a Fisher score representation of the movements, which is subsequently used to optimize affective movement recognition using support vector machine classification. In addition, this paper presents an approach to obtain a minimal discriminative representation of the movements using supervised principal component analysis (SPCA) that is based on Hilbert-Schmidt independence criterion in the Fisher score space. The dimensions of the resulting SPCA subspace consist of intrinsic movement features salient to affective movement recognition. These salient features enable a low-dimensional encoding of observed movements during a human-machine interaction, which can be used to recognize and analyze human affect that is displayed through movement. The efficacy of the proposed approach in recognizing affective movements and identifying a minimal discriminative movement representation is demonstrated using two challenging affective movement datasets.

The Laban Effort and Shape components provide a systematic tool for a compact and informative description of the dynamic qualities of movements. To enable the application of Laban notation in computational movement analysis, measurable physical correlates of Effort and Shape components need to be identified. Such physical correlates enable quantification of Effort and Shape components, which in turn facilitates computational analysis of affective movements. In this work, two existing approaches to quantification of Laban Effort components (Weight, Time, Space, and Flow) based on measurable movement features (position, velocity, acceleration, and jerk) are adapted for hand and arm movements, and a new approach for quantifying Shape Directional based on the average trajectory curvature is proposed. The results show a high correlation between Laban annotations provided by a certified movement analyst (CMA) and the quantified Effort Weight (81%), Time (77%) and Shape Directional (93%) for an affective hand and arm movement dataset.

This paper presents a method for designing binary trees for SVM classification. The proposed algorithm, multi-modal binary tree (MBT) tolerates misclassification in the upper nodes of the tree, allowing points to be classified in either output regardless of the initial specified class groupings. MBT can separate classes that are inseparable with a single classifier by using a piecewise division. The algorithm also incorporates feature selection for the individual classifiers in the system. Classification results on several artificial and real data sets show that the proposed algorithm performs well compared to existing methods for multi-class SVM classification, and although the classifiers are larger, the time required to classify a point is smaller.

This paper proposes a trajectory generation and control strategy for generating stable gait subject to unknown disturbances, based on the concept of the Foot Placement Estimator (FPE). While most walking control strategies in the field of bipedal locomotion aim to constantly maintain balance, the Foot Placement Estimator (FPE) estimates where the foot must be placed in order to restore balance. One of the key novelties of the FPE approach is its natural extension to form complete gait cycles using a state machine and simple proportional-derivative controllers. In this paper, the FPE control strategy is extended from 2D to 3D robots, and demonstrated in simulation on a 14-DOF lower body bipedal robot.

Path following controllers make the output of a control system approach and traverse a pre-specified path with no a priori time-parametrization. This paper implements a path following controller, based on transverse feedback linearization (TFL), which guarantees invariance of the path to be followed. The coordinate and feedback transformation employed allows one to easily design control laws to generate arbitrary desired motions on the path for the closed-loop system. The approach is applied to an uncertain and simplified model of a robot manipulator for which none of the dynamic parameters are measured. The controller is made robust to modelling uncertainties using Lyapunov redesign. The robustified controller is tested on a 4-degree-of-freedom (4-DOF) manipulator with a combination of revolute and linear actuated links. The experimental results show a substantial improvement when using the robust controller for path following versus standard state feedback.

The ability to learn is essential for robots if they are to function within human environments. Learning requires an understanding of the underlying structure of what has been observed. This paper proposes a learning method that automatically creates Petri nets from observation of human demonstrations to model the underlying structure of tasks. The Petri net can be learned via a single or multiple demonstrations. The learned Petri nets are capable of generating action sequences to allow a robot to imitate the task. The proposed model also allows for generalization and variations in performing the task. The proposed method is tested on demonstrations of block stacking tasks and verified through robot imitation of the tasks in simulation and in physical experiments.

During robot imitation learning, a key problem when observing the motions of a demonstrator is the modeling and recognition of movement prototypes. This paper proposes using Affinity Propagation (AP) to cluster motions modeled using either Dynamic Movement Primitives (DMPs) or Hidden Markov Models (HMMs). The proposed AP clustering algorithm is simple and efficient, provides robust results and automatically identifies representative exemplars for each motion group, leading to a minimal representation of the observations that can also be used to generate motions. In experiments using videos and motion capture data of human demonstrations, it is shown that the weight parameters of the DMP model can be used as features for motion recognition and the proposed method can distinguish between different (coarse distinction) or similar (fine distinction) motion groups.

To facilitate human-robot interactions with the user, it is necessary for the robot to identify the interaction partner. We propose the use of a single wearable sensor worn at the center of the user's belt to record the gait when the interaction partner approaches the robot. Based on the data of a single gait cycle recorded with a single inertial measurement unit (IMU), we identify a person by his/her walking style. For identification, we first detect individual strides. We introduce a simple feature that characterizes the individual's asymmetry of gait and classify the individual using a Bayes classifier. To evaluate our approach, we collect motion data from 20 persons; the classification accuracy based on the proposed asymmetry-based feature reaches 99.3%. We further investigate the robustness of our approach against slight variations in the sensor placement, variations in speed, and walking straight versus walking on a curved route.

This paper proposes an approach for the temporal segmentation of human body movements using IMU (Inertial Measurement Unit). The approach is based on online HMM-based segmentation of continuous time series data. In previous studies, the real-time segmentation of human body movement using joint angles acquired by optical motion capture has been realized, using stochastic motion modeling. The approach is now adapted for angular velocity data. The segmented motions are recognized via HMM models. The segmentation and recognition results of the proposed algorithm are demonstrated with experiments. Auto segmentation of each motion and recognition of motion patterns are verified using angular velocity data obtained by IMU sensors and the Wii remote. The success rate of auto segmentation using the data obtained by Wii remote was more than 80% on average.

Optical motion tracking has enhanced human movement analysis in medicine, biomechanics, and rehabilitation science by providing highly accurate joint angle measurements over time. However, analyzing the large amount of recorded data is challenging. The process is usually simplified by calculating descriptive measures, such as the minimum, mean, or maximum, from the time series data. We propose a novel technique for the analysis of human motion data, which considers the complete time series data and is based on the F-statistic traditionally used in medical and biomechanical studies. The time series data is modeled by a Hidden Markov Model (HMM) and the F-statistic is reformulated using the Kullback-Leibler divergence for comparing HMMs. This provides a novel technique to enhance the analysis of human movement data and includes an automatic generation of group-specific trajectories to simplify visual data analysis. It is further suitable as time-series based, univariate feature selection technique in machine learning.

Body movements communicate affective expressions and, in recent years, computational models have been developed to recognize affective expressions from body movements or to generate movements for virtual agents or robots which convey affective expressions. This survey summarizes the state of the art on automatic recognition and generation of such movements. For both automatic recognition and generation, important aspects such as the movements analyzed, the affective state representation used, and the use of notation systems is discussed. The survey concludes with an outline of open problems and directions for future work.

Feature selection can decrease classifier size and improve accuracy by removing noisy and/or redundant features. However, it is possible for feature selection to yield features that are only partially informative about the classes in the set. These features are beneficial for distinguishing between some classes but not others. In these cases, it is beneficial to divide the large classification problem into a set of smaller problems, where a more specific set of features can be used to classify different classes. Dividing a problem this way is also common when the base classifier is binary, and the problem needs to be reformulated as a set of two-class problems so it can be handled by the classifier. This paper presents a method for multiclass classification that simultaneously formulates a binary tree of simpler classification subproblems and performs feature selection for the individual classifiers. The feature selected hierarchical classifier (FSHC) is tested against several well-known techniques for multiclass division. Tests are run on nine different real data sets and one artificial data set using a support vector machine (SVM) classifier. The results show that the accuracy obtained by the FSHC is comparable with other common multiclass SVM methods. Furthermore, the results demonstrate that the algorithm creates solutions with fewer classifiers, fewer features, and a shorter testing time than the other SVM multiclass extensions.

The human body has many biomechanical degrees of freedom, and thus, multiple movement strategies can be employed to execute a given task. Joint loading patterns and risk of injury are highly sensitive to the movement strategy employed. This paper develops a computational framework to automatically identify and recognize different movement strategies to perform a task from human motion data. A divisive clustering approach is developed to identify movement strategies. Hidden Markov models (HMMs) are trained with the clustered observation sequences to generate strategy-specific models that are improved iteratively by using the maximum likelihood to relocate sequences to the most suitable cluster. Differences in individual joint trajectories are compared across strategies using a stochastic distance measure. The proposed algorithm is compared against three existing algorithms - joint contribution vector, decision tree, and HMM-based agglomerative clustering. Experimental results indicate that the proposed approach performs better than existing algorithms to detect motion strategies and automatically determine the differences between the strategies.

Most of the systems for recognition of activities aim to identify a set of normal human activities. Data is either recorded by computer vision or sensor based networks. These systems may not work properly if an unusual event or abnormal activity occurs, especially ones that have not been encountered in the past. By definition, unusual events are mostly rare and unexpected, and therefore very little or no data may be available for training. In this paper, we focus on the challenging problem of detecting unusual temporal events in a sensor network and present three Hidden Markov Models (HMM) based approaches to tackle this problem. The first approach models each normal activity separately as an HMM and the second approach models all the normal activities together as one common HMM. If the likelihood is lower than a threshold, an unusual event is identified. The third approach models all normal activities together in one HMM and approximates an HMM for the the unusual events. All the methods train HMM models on data of the usual events and do not require training data from the unusual events. We perform our experiments on a Locomotion Analysis dataset that contains gyroscope, force sensor, and accelerometer readings. To test the performance of our approaches, we generate five types of unusual events that represent random activity, extremely unusual events, unusual events similar to specific normal activities, no or little motion and normal activity followed by no or little motion. Our experiments suggest that for a moderately sized time frame window, these approaches can identify all the five types of unusual events with high confidence.

Model-based control strategies for robot manipulators can present numerous performance advantages when an accurate model of the system dynamics is available. In practice, obtaining such a model is a challenging task which involves modeling such physical processes as friction, which may not be well understood and difficult to model. This paper proposes an approach for online learning of the inverse dynamics model using Gaussian Process Regression. The Sparse Online Gaussian Process (SOGP) algorithm is modified to allow for incremental updates of the model and hyperparameters. The influence of initialization on the performance of the learning algorithms, based on any a-priori knowledge available, is also investigated. The proposed approach is compared to existing learning and fixed control algorithms and shown to be capable of fast initialization and learning rate.

This paper presents a study on human performance in recognizing affective expressions conveyed through movements of hand-like structures. One movement sequence, closing and opening the hand, was performed by a demonstrator in 3 sets of 5 repeated trials, each set intending to convey a different affective expression. Three different expressions, sadness, happiness and anger, were considered. Expressive movement animations were replicated with a human-like hand model, a stick hand model and with a model resembling a palm frond structure. The structures tested have identical kinematics but different physical appearance. The ability of a human to correctly identify the intended expressive movements performed on these different structures was tested with 66 users viewing videos of the animated structures and reporting via an online questionnaire. Results show that anger is reliably perceived by observers from animated movements on different structures, while the other emotions are easily misperceived. The physical appearance of the structure has some impact on perception performance, but was not found to be statistically significant in this study. Furthermore, analyzing the participants' responses in the context of the valence-arousal model of emotion shows that the subjects were able to recognize the arousal component of the affective hand movements across all structures.

This work presents a method for improving classifier accuracy through joint feature selection and hierarchical classifier design with genetic algorithms. The hierarchical classifier divides the classification problem into a set of smaller problems using multiple feature-selected classifiers in a tree configuration to separate the data into progressively smaller groups of classes. This allows the use of more specific feature sets for each set of classes. Several existing performance measures for evaluating the feature sets are investigated, and a new measure, count-based RELIEF is proposed. The joint feature selection and hierarchical classifier design method is tested on two artificial data sets. Results indicate that the feature selected hierarchical classifiers are able to achieve better accuracy than a non-hierarchical classifier using feature selection alone. The newly proposed performance measure is also tested and shown to provide a better indication of classifier performance than existing methods.

This paper proposes a particle filter based marker-less upper body motion capture system, capable of running in realtime. This system is designed for a humanoid robot application, and thus a monocular image sequence is used as input. We first set up a model of the human body, a sub-model which includes 11 Degrees of Freedom is used for the upper body tracking. Considering the realtime processing requirements, two time efficient cues are implemented in the likelihood calculation, namely the edge cue and the distance cue. The system is tested using a publicly available database, which consists of both the videos and the ground truth data, enabling quantitative error analysis. The system successfully tracks the human through arbitrary upper body motion at 20Hz.

This paper proposes a novel approach for extracting a model of movement primitives and their sequential relationships during online observation of human motion. In the proposed approach, movement primitives, modeled as hidden Markov models, are autonomously segmented and learned incrementally during observation. At the same time, a higher abstraction level hidden Markov model is also learned, encapsulating the relationship between the movement primitives. For the higher level model, each hidden state represents a motion primitive, and the observation function is based on the likelihood that the observed data is generated by the motion primitive model. An approach for incremental training of the higher order model during online observation is developed. The approach is validated on a dataset of continuous movement data.

In this paper, we propose a marker-less full body human motion capture system designed for humanoid robot applications. The system is based on a stereo camera, and therefore has strong portability. Tracking is implemented within the particle filter framework, and the high dimensionality problem is solved through partitioned sampling. Taking advantage of the stereo setup, we propose a depth cue which resolves the problem of missing depth information in monocular tracking. Three other cues, the edge cue, the color cue and the distance cue, are also integrated into the system to enhance the tracking performance. The system is tested using the publicly available CMU MOCAP database which also includes ground truth data, and this enables us to analyze the results quantitatively and compare the relative usefulness of different cues. The system is shown to be capable of tracking challenging videos accurately and robustly in near real-time.

This paper proposes an approach to hierarchy formation of human behaviors, extraction of the behavioral transitions, and their application to prediction and automatic generation of behaviors. Human demonstrator motion patterns are stored as motion symbols, which abstract the motion data by using Hidden Markov Models. The stored motion patterns are organized into a hierarchical tree structure, which represents the similarity among the motion patterns and provides abstracted motion patterns. Concatenated sequences of motion patterns are stochastically represented as transitions between the abstracted motion patterns by using an Ngram Model, and the transitional relationships of the human behaviors are extracted. The behavioral hierarchy and transition model make it possible to predict human behaviors during observation and to generate sequences of motion patterns automatically while maintaining a natural motion stream, as if the system is a “crystal ball” to reflect future behaviors. The experiments validates the proposed framework by using a developed visualization system, which shows the demonstrator or the operator the established hierarchical tree and the transition network of the motion patterns, predicted behaviors and generated sequences of the motion patterns.

status: Published

This paper proposes a novel approach for motion primitive segmentation from continuous full body human motion captured on monocular video. The proposed approach does not require a kinematic model of the person, nor any markers on the body. Instead, optical flow computed directly in the image plane is used to estimate the location of segment points. The approach is based on detecting tracking features in the image based on the Shi and Thomasi algorithm [1]. The optical flow at each feature point is then estimated using the Lucas Kanade Pyramidal Optical Flow estimation algorithm [2]. The feature points are clustered and tracked on-line to find regions of the image with coherent movement. The appearance and disappearance of these coherent clusters indicates the start and end points of motion primitive segments. The algorithm performance is validated on full body motion video sequences, and compared to a joint-angle, motion capture based approach. The results show that the segmentation performance is comparable to the motion capture based approach, while using much simpler hardware and at a lower computational effort.

In previous work, the authors have been developing a stochastic model based approach for on-line segmentation of whole body human motion patterns during human motion observation and learning, using a simplified kinematic model of the human body. In this paper, we extend the proposed approach to larger, more realistic kinematic models, which can better represent a larger variety of human motions. These larger models may include spherical in addition to revolute joints. We examine the effects on segmentation performance due to motion representation choice, and compare the segmentation efficacy when Cartesian or joint angle data is used. The approach is tested on whole body human motion data modeled with a 42DoF kinematic model. The results indicate that Cartesian data seems to correspond most closely to the human evaluation of segment points. The experiments also demonstrate the efficacy of the segmentation approach for large kinematic models and a variety of human motions.

In order to develop a friendly and safe interaction between humans and robots, it is essential for the robot to evaluate user's affective states and respond accordingly. However, affective states are typically assessed using offline questionnaires and user reports. In this paper we investigate the use of an online-device for collecting real-time user reports of affective state during interaction with a robot. These reports are compared to both previous survey reports taken after the interaction, and the affective states estimated by an inference system. The aim is to evaluate and characterize the physiological signal-based inference system and determine which factors significantly influence its performance. This analysis will be used in future work, to fine tune the affective estimations by identifying what kind of variations in physiological signals precede or accompany the variations in reported affective states.

This paper proposes an on-line, interactive approach for incremental learning and visualization of full body motion primitives from observation of human motion. The human demonstrator motion is captured in a motion capture studio. The continuous observation sequence is first partitioned into motion segments, using stochastic segmentation. Motion segments are next incrementally clustered and organized into a hierarchical tree structure representing the known motion primitives. At the same time, the sequential relationship between motion primitives is learned, to enable the generation of coherent sequences of motion primitives. An on-line visualization system is also developed to allow the demonstrator to visualize the motion database and the motion primitives learned by the system, thus giving the demonstrator insight into the learning process and the ability to interactively modify the demonstration based on the current state of the knowledge base. The developed system has many potential applications for motion analysis, prediction and imitation learning for humanoid robots.

This paper describes a novel approach for incremental learning of human motion pattern primitives through online observation of human motion. The observed time series data stream is first stochastically segmented into potential motion primitive segments, based on the assumption that data belonging to the same motion primitive will have the same underlying distribution. The motion segments are then abstracted into a stochastic model representation and automatically clustered and organized. As new motion patterns are observed, they are incrementally grouped together into a tree structure, based on their relative distance in the model space. The tree leaves, which represent the most specialized learned motion primitives, are then passed back to the segmentation algorithm so that as the number of known motion primitives increases, the accuracy of the segmentation can also be improved. The combined algorithm is tested on a sequence of continuous human motion data that are obtained through motion capture, and demonstrates the performance of the proposed approach.

This paper describes a novel approach for incremental learning of human motion pattern primitives through on-line observation of human motion. The observed motion time series data stream is first stochastically segmented into potential motion primitive segments, based on the assumption that data belonging to the same motion primitive will have the same underlying distribution. The motion segments are then abstracted into a stochastic model representation, and automatically clustered and organized. As new motion patterns are observed, they are incrementally grouped together based on their relative distance in the model space. The resulting representation of the knowledge domain is a tree structure, with specialized motions at the tree leaves, and generalized motions closer to the root. The tree leaves, which represent the most specialized learned motion primitives, are then passed back to the segmentation algorithm, so that as the number of known motion primitives increases, the accuracy of the segmentation can also be improved. The combined algorithm is tested on a sequence of continuous human motion data obtained through motion capture, and demonstrates the performance of the proposed approach.

This paper proposes a method to recover missing data during observation by factorial hidden Markov models (FHMMs). The fundamental idea of the proposed method originates from the mimesis model, inspired by the mirror neuron system. By combining the motion recognition from partial observation algorithm and the proto-symbol based duplication of observed motion algorithm, whole body motion imitation from partial observation can be achieved. The algorithm for missing data recovery uses the same basic strategy as the whole body motion imitation from partial observation, but requires more accurate spatial representability. FHMMs allow for more efficient representation of a continuous data sequence by distributed state representation compared to hidden Markov models (HMMs). The proposed algorithm is tested with human motion data and the experimental results show improved representability compared to the conventional HMMs.

This paper develops an approach for on-line segmentation of whole body human motion patterns during human motion observation and learning. A Hidden Markov Model is used to represent the incoming data sequence, where each model state represents the probability density estimate over a window of the data. Based on the assumption that data belonging to the same motion primitive will have the same underlying distribution, the segmentation is implemented by finding the optimum state sequence over the developed model. The basic algorithm is modified to add the capability for modifying the model based on known motion primitives. The inclusion of such scaffolding motion primitives can improve the performance of the basic segmentation algorithm. The modified algorithm is tested on a corpus of continuous human motion data to show the efficacy of the proposed approach.

This paper describes an improved methodology for human motion recognition and imitation based on Factorial Hidden Markov Models (FHMM). Unlike conventional Hidden Markov Models (HMMs), FHMMs use a distributed state representation, which allows for more efficient representation of each time sequence. Once the FHMMs are trained with exemplar motion data, they can be used to generate sample trajectories for motion production, and produce significantly more accurate trajectories compared to single Hidden Markov chain models. Due to the additional information encoded in FHMMs models, FHMM models have a higher Kullback- Leibler distance compared to single Markov chain models, making it easier to distinguish between similar models. The efficacy of using FHMMs is tested on a database of human motions obtained through motion capture. The results show that FHMMs provide better generalization to new data when compared to conventional HMMs during motion recognition, as well as providing a better fit for generated data.

Dynamic Parameter Identification is a useful tool for developing and evaluating robot control strategies. However, a multi degree of freedom robot arm has many parameters, and the process of determining them is challenging. Much research has been done in this area and experimental methods have been applied on several robot arms. To our knowledge, there is currently no set of inertial parameters, either by modelling or by estimation, available for the CRS A460/A465 arm, a popular laboratory table top robot. In this paper we review and compare a number of methods for dynamic parameter identification and for generating trajectories suitable for estimating the identifiable dynamic parameters of a given robot. We then present a step by step process for dynamic parameter identification of a serial manipulator, and demonstrate this process by experimentally identifying the dynamic parameters of the CRS A460 robot.

hierarchical model incorporating motion patterns, proto symbols and words is proposed. The proto symbols abstract motion patterns, while the words are associated with the proto symbols stochastically. This paper describes the construction of a word space, where words are located in a multidimensional space based on dissimilarities among the words. The dissimilarity between two words can be calculated by using association probabilities that the words generate motion proto symbols. The word space encapsulates relations among the words such as similar or dissimilar pairs of words. The word space also allows motion recognition based on words. The validity of the constructed word space is demonstrated on a motion capture database. Moreover, the addition of the word associations is found to change the conventional proto symbol space so that the discrimination among the proto symbols is improved.

This paper describes a novel algorithm for autonomous and incremental learning of motion pattern primitives by observation of human motion. Human motion patterns are abstracted into a Hidden Markov Model representation, which can be used for both subsequent motion recognition and generation, analogous to the mirror neuron hypothesis in primates. As new motion patterns are observed, they are incrementally grouped together using hierarchical agglomerative clustering based on their relative distance in the HMM space. The clustering algorithm forms a tree structure, with specialized motions at the tree leaves, and generalized motions closer to the root. The generated tree structure will depend on the type of training data provided, so that the most specialized motions will be those for which the most training has been received. Tests with motion capture data for a variety of motion primitives demonstrate the efficacy of the algorithm.

In order for humans and robots to interact in an effective and intuitive manner, robots must obtain information about the human affective state in response to the robot's actions. This secondary mode of interactive communication is hypothesized to permit a more natural collaboration, similar to the “body language” interaction between two cooperating humans. This paper describes the implementation and validation of a hidden Markov model (HMM) for estimating human affective state in real time, using robot motions as the stimulus. Inputs to the system are physiological signals such as heart rate, perspiration rate, and facial muscle contraction. Affective state was estimated using a two-dimensional valence-arousal representation. A robot manipulator was used to generate motions expected during human–robot interaction, and human subjects were asked to report their response to these motions. The human physiological response was also measured. Robot motions were generated using both a nominal potential field planner and a recently reported safe motion planner that minimizes the potential collision forces along the path. The robot motions were tested with 36 subjects. This data was used to train and validate the HMM model. The results of the HMM affective estimation are also compared to a previously implemented fuzzy inference engine.

In order for humans and robots to interact in an effective and intuitive manner, robots must obtain information about the human affective state in response to the robot's actions. This secondary mode of interactive communication is hypothesized to permit a more natural collaboration, similar to the "body language" interaction between two cooperating humans. This paper describes the implementation and validation of a hidden Markov model for estimating human affective state in real-time, using robot motions as the stimulus. Inputs to the system are physiological signals such as heart rate, perspiration rate, and facial muscle contraction. Affective state was estimated using a two dimensional valence-arousal representation. A robot manipulator was used to generate motions simulating human-robot interaction, and human subjects were asked to report their response to the motions. The human physiological response was also measured. Robot motions were generated using both a nominal potential field planner and a recently reported safe motion planner that minimizes the potential collision forces along the path. The robot motions were tested with 36 subjects. This data was used to train and validate the HMM model. The results of the HMM affective estimation are also compared to a previously implemented fuzzy inference engine

This paper describes an experiment to determine the feasibility of using physiological signals to determine the human response to robot motions during direct human-robot interaction. A robot manipulator is used to generate common interaction motions, and human subjects are asked to report their response to the motions. The human physiological response is also measured. Motion paths are generated using a classic potential field planner and a safe motion planner, which minimizes the potential collision force along the path. A fuzzy inference engine is developed to estimate the human response based on the physiological measures. Results show that emotional arousal can be detected using physiological signals and the inference engine. Comparison of initial results between the two planners shows that subjects report less anxiety and surprise with the safe planner for high planner speeds.