Trust is a critical factor in human–robot interaction. This study investigates how using human data to train reinforcement learning (RL) agents affects perceived performance and trust. We conducted two experiments: an online study using the video game Seaquest, and a physical robot study involving a wire loop task. In both, RL agents were trained using demonstrations from human users with varying skill levels, as well as without human data. Participants then rated the agents’ performance and how much they trusted them. Our results for the online study indicate that RL agents trained with expert human data received significantly higher trust ratings than other agents and that trust was higher in replay and long training conditions compared to short and no training. Videos showing beginner players were also trusted, despite lower performance, although these differences did not always reach statistical significance. This highlights that trust is influenced by factors beyond mere performance, suggesting a role for cues such as perceived human involvement. In the physical robot study, our findings indicate that trust was significantly influenced by the type of training data, with data from an experienced human having a positive effect under certain training conditions. Performance ratings were influenced primarily by the training approach, with replay receiving higher ratings than short training. Although trust was notably higher under certain conditions, performance ratings did not always align with trust, indicating a partial dissociation between the two measures. Together, these results suggest that training with high-quality human data can enhance the trustworthiness of RL agents under appropriate training conditions. However, trust depends not only on performance but also on how human-like an agent’s behavior appears. Designing agents for trust-sensitive applications may therefore benefit from aligning behavior with user expectations rather than optimizing performance alone.

In medical and industrial domains, providing guidance for assembly processes can be critical to ensure efficiency and safety. Errors in assembly can lead to significant consequences such as extended surgery times and prolonged manufacturing or maintenance times in industry. Assembly scenarios can benefit from in-situ augmented reality visualization, i.e., augmentations in close proximity to the target object, to provide guidance, reduce assembly times, and minimize errors. In order to enable in-situ visualization, 6D pose estimation can be leveraged to identify the correct location for an augmentation. Existing 6D pose estimation techniques primarily focus on individual objects and static captures. However, assembly scenarios have various dynamics, including occlusion during assembly and dynamics in the appearance of assembly objects. Existing work focus either on object detection combined with state detection, or focus purely on the pose estimation. To address the challenges of 6D pose estimation in combination with assembly state detection, our approach ASDF builds upon the strengths of YOLOv8, a real-time capable object detection framework. We extend this framework, refine the object pose, and fuse pose knowledge with network-detected pose information. Utilizing our late fusion in our Pose2State module results in refined 6D pose estimation and assembly state detection. By combining both pose and state information, our Pose2State module predicts the final assembly state with precision. The evaluation of our ASDF dataset shows that our Pose2State module leads to an improved assembly state detection and that the improvement of the assembly state further leads to a more robust 6D pose estimation. Moreover, on the GBOT dataset, we outperform the pure deep learning-based network and even outperform the hybrid and pure tracking-based approaches.

Networked devices, especially those comprising multiple identical devices, are extensively utilized in industrial scenarios. However, their complexity poses unique challenges in diagnostic processes, demanding efficient methodologies to identify and assess risks. The application of Failure Mode and Effects Analysis (FMEA) for analyzing complex systems, especially those consisting of networked devices, appears to be limited, particularly in identifying critical risk factors. In this paper, we propose a novel pipeline to diagnose networked systems by fusing FMEA with a Delphi (expert) Study. Our approach leverages the collective knowledge of a group of experts through a structured Delphi Study enabling them to contribute individually but also to interact. We demonstrate the applicability of our approach through a case study involving a system of networked mobile robots. Based on a Fault Tree Analysis (FTA), Risk Priority Numbers (RPN) of minimal fault tree cut sets are calculated to identify the most critical mechatronic failures. Our findings show that our methodology provides an RPN ranking that closely aligns with expert insights, highlighting its efficacy in accurately assessing risk in complex networked systems.

This article presents a systematic review on wearable robotic devices that use human-in-the-loop optimization (HILO) strategies to improve human-robot interaction. A total of 46 HILO studies were identified and divided into upper and lower limb robotic devices. The main aspects from HILO were identified, reviewed, and classified in four areas: 1) human-machine systems; 2) optimization methods; 3) control strategies; and 4) experimental protocols. A variety of objective functions (physiological, biomechanical, and subjective), optimization strategies, and optimized control parameters configurations used in different control strategies are presented and analyzed. An overview of experimental protocols is provided, including metrics, tasks, and conditions tested. Moreover, the relevance given to training or adaptation periods was explored. We outline an HILO framework that includes current wearable robots, optimization strategies, objective functions, control strategies, and experimental protocols. We conclude by highlighting current research gaps and defining future directions to improve the development of advanced HILO strategies in upper and lower limb wearable robots.

People with motor disabilities of neuromuscular origin often struggle with operating the computer through a commercially available mouse and QWERTY keyboard. This work presents an alternative mouse and keyboard interface, which is personalizable to meet the individual needs of the target population to allow a more efficient use of the computer. The regular/commercially available mouse is therefor replaced by a spectacle frame equipped with pressure sensors for mouse click activation and an inertial measurement unit for mouse cursor control through head movements. The alternative keyboard is based around a ten button keyboard, with multiple letters assigned to each key. The users can then chose the desired word from a list of word suggestions that is compiled based on the keyboard input. Prior experiments, the users went through a software guided calibration procedure to experimentally determine individual thresholds and parameters. The system was tested by two nondisabled participants through typing and click tests, followed by a questionnaire to give feedback on the system.

This interdisciplinary workshop aims to break boundaries between the researchers who develop human models (e.g., from the fields of human factors, cognitive psychology, and computational neuroscience) and roboticists who use human models in different human-robot interaction (HRI) contexts. The keynote talks, contributed submissions, and interactive discussions will focus on the questions such as: How can modeling humans help us understand and design human-robot interactions? What kinds of models are useful for which HRI contexts (physical/cognitive interactions) and purposes (behavior prediction/personalization/theory -of- mind/etc.)? What common lessons can be learned from human behavior modeling in HRI across different application domains? How can modeling humans in HRI tasks help us to better understand human cognition/behavior? By stimulating an interdisciplinary conver-sation around these questions, we aim to raise awareness of the benefits of modeling and expose the wider HRI community to a variety of different modeling approaches, and facilitate the HRI researchers who already engage in modeling to exchange views on methodology of modeling and best nractices from diverse fields.

Subjective experience of human control over remote, artificial, or virtual limbs has traditionally been investigated from two separate angles: presence research originates from teleoperation, aiming to capture to what extent the user feels like actually being in the remote or virtual environment. Embodiment captures to what extent a virtual or artificial limb is perceived as one’s own limb. Unfortunately, the two research fields have not interacted much. This survey intends to provide a coherent overview of the literature at the intersection of these two fields to further that interaction. Two rounds of systematic research in topic-related data bases resulted in 414 related articles, 14 of which satisfy the deliberately strict inclusion criteria: 2 theoretical frameworks that highlighted intersections and 12 experimental studies that evaluated subjective measures for both concepts. Considering the surrounding literature as well, theoretical and experimental potential of embodiment and presence are discussed and suggestions to apply them in teleoperation research are derived. While increased publication activity is observed between 2016 and 2018, potentially caused by affordable virtual reality technologies, various open questions remain. To tackle them, human-in-the-loop experiments and three guiding principles for teleoperation system design (mechanical fidelity, spatial bodily awareness, and self-identification) are suggested.

The question how humans integrate artificial lower limb devices into their body schema has distinct potential for engineering motion assistance systems, e.g., the design of robotic prostheses. Adding robotic technology to existing psychological experiments enables a deeper investigation of multisensory interaction between proprioceptive, visual, and tactile stimuli during motion. This paper reports the design and control of a robot to investigate embodiment with regard to the lower limbs. In an evaluation study, the rubber hand illusion is transferred to the whole leg for the first time. Participants performed knee bends according to three different conditions being imitated by a robotic leg. The occurrence of a robotic leg illusion was subjectively assessed by a questionnaire and objectively measured by the proprioceptive drift. Considering both metrics, the results show a successful integration of the robotic leg into the body schema. Motion synchronization appears to be a paramount factor, whereby the study indicates that acoustical stimulation might also be relevant. The interrelation between mechatronic design and control of the human-in-the-loop experiment and the factors influencing the illusion are discussed and alternative experimental setups are suggested.

For individuals having a motor disorder of neuromuscular origin, computer usage can be challenging. Due to different medical conditions, alternative input methodologies such as speech or eye tracking are no option. Here, piezo sensors, inertial measurement units and force resistance sensors are used to develop input devices that can compensate for mouse and keyboard. The devices are tested in a case study with one potential user with ataxia. Future user studies will deliver additional insights in the users' specific needs and further improve the developments.

This work introduces a novel hybrid control architecture for a hexapod platform (Weaver), making it capable of autonomously navigating in uneven terrain. The main contribution stems from the use of vision-based exteroceptive terrain perception to adapt the robot's locomotion parameters. Avoiding computationally expensive path planning for the individual foot tips, the adaptation controller enables the robot to reactively adapt to the surface structure it is moving on. The virtual stiffness, which mainly characterizes the behavior of the legs' impedance controller is adapted according to visually perceived terrain properties. To further improve locomotion, the frequency and height of the robot's stride are similarly adapted. Furthermore, novel methods for terrain characterization and a keyframe based visual-inertial odometry algorithm are combined to generate a spatial map of terrain characteristics. Localization via odometry also allows for autonomous missions on variable terrain by incorporating global navigation and terrain adaptation into one control architecture. Autonomous runs on a testbed with variable terrain types illustrate that adaptive stride and impedance behavior decreases the cost of transport by 30 % compared to a non-adaptive approach and simultaneously increases body stability (up to 88 % on even terrain and by 54 % on uneven terrain). Weaver is able to freely explore outdoor environments as it is completely free of external tethers, as shown in the experiments.

This paper describes the experience of the first author with the Point-and-Click Interface of the OnScreenDualScribe, created by the last author. The new interface is an innovative extension to the previous interface which required the use of the DualPad. The main differences between the two interfaces are highlighted. The user took several writing tests with the Point-and Click Interface and compares her results with two of interfaces she uses the most for writing, Dragon NaturallySpeaking and SofType. Finally, the first author recommends several improvements to the interface which would make the software a better alternative for her.

Legged robots such as hexapods have the potential to traverse unstructured terrain. This paper introduces a novel hexapod robot (Weaver) using a hierarchical controller, with the ability to efficiently traverse uneven and inclined terrain. The robot has five joints per leg and 30 degrees of freedom overall. The two redundant joints improve the locomotion of the robot by controlling the body pose and the leg orientation with respect to the ground. The impedance controller in Cartesian space reacts to unstructured terrain and thus achieves self-stabilizing behavior without prior profiling of the terrain through exteroceptive sensing. Instead of adding force sensors, the force at the foot tip is calculated by processing the current signals of the actuators. This work experimentally evaluates Weaver with the proposed controller and demonstrates that it can effectively traverse challenging terrains and high gradient slopes, reduce angular movements of the body by more than 55% and reduce the cost of transport (up to 50% on uneven terrain and by 85% on a slope with 20 °). The controller also enables Weaver to walk up inclines of up to 30 °, and remain statically stable on inclines up to 50 °. Furthermore, we present a new metric for legged robot stability performance along with a method for proprioceptive terrain characterization.

Understanding the integration of user-proximal robots in the body schema of their human users has a distinct potential to improve human-robot interaction. Robotic devices can help to investigate the psychological fundamentals of body schema integration. While the Rubber Hand Illusion experiment indicates how artifacts can be perceived as a part of the own body, it relies on a passive limb that does not perform motions during the examinations. Novel setups aim at Robotic Hand/Leg Illusions induced by robotic devices which imitate human motions. Although such devices distinctly extend experimental possibilities, their design is rather proprietary and unstructured up to now. This paper analyzes the requirements of robotic hand and leg illusion setups based on systematic discussion of a multidisciplinary team of researchers from engineering and psychology. In a comparative study, requirements are collected and structured, their similarities and differences are determined, and the most important ones are extracted yielding design implications. The requirements with the highest priority are setup characteristics that concern the occurrence and quality of the illusion, i.e., hiding the real limb, anatomical plausibility, visual appearance, temporal delay, and software-controlled experimental conditions. Based on the results, the design of future robotic devices for the exploration of human body schema integration might be guided and supported.

Sensor gloves are widely adopted input devices for several kinds of human-robot interaction applications. Existing glove concepts differ in features and design, but include limitations concerning the captured finger kinematics, position/orientation sensing, wireless operation, and especially economical issues. This paper presents the DAGLOVE which addresses the mentioned limitations with a low-cost design (ca. 300 €). This new sensor glove allows separate measurements of proximal and distal finger joint motions as well as position/orientation detection with an inertial measurement unit (IMU). Those sensors and tactile feedback induced by coin vibration motors at the fingertips are integrated within a wireless, easy-to-use, and open-source system. The design and implementation of hardware and software as well as proof-of-concept experiments are presented. An experimental evaluation of the sensing capabilities shows that proximal and distal finger motions can be acquired separately and that hand position/orientation can be tracked. Further, teleoperation of the iCub humanoid robot is investigated as an exemplary application to highlight the potential of the extended low-cost glove in human-robot interaction.

This paper introduces a new approach to measuring relative movement between the residual limb and fitted prosthetic socket in an amputee during gait. Considering requirements motivated by the specific measurement task, a system for the recognition of residual limb movement within the socket system in dynamic gait situations is designed. To evaluate the feasibility of the measurement task with this sensor concept based on an optical 2D-motion sensor, a standalone functional model is established. The functional model is introduced regarding hardware and software components. This paper further presents preliminary experimental evaluation of the functional model. Results considering precision and accuracy of the functional model are displayed. Dependencies to movement velocity, overcome distance, and traveling direction are assessed. Depending on the parameters mentioned above, the realized prototype of the sensor concept shows relative errors regarding accuracy within the range of 2 to 6 %. Precision depends on movement direction as well as path lengths. Overall, the proposed sensor concept seems suitable for acquiring the relative movement between residual limb and prosthetic socket during amputee gait.

This paper presents a systematic approach for experimental identification and assessment of mechanical effects on the dynamics of elastic actuators. The variable torsion stiffness (VTS) actuator is used as an example. As a basis for parameter identification, a flexible joint robot model considering friction and damping is used. To identify and assess occurring effects, a component-wise experimental investigation of the VTS drive train is performed. In this, influences of friction and inertia are examined using numerical least-squares regression to identify link inertia, friction in bearings, stiffness and damping in the elastic element, and the friction of gear box and actuator based on measured data. Comparing simulations to measured data, allows an evaluation of the obtained particular parameters and an assessment of their impact on overall system dynamics. Hence, the component-wise procedure enables to decide if those should be considered. This approach can be generalized to other elastic actuators by adapting model structure and/or performed experiments.

The integration of prostheses or wearable robotics into the body schema of their users is a fundamental requirement for the acceptance and control of such artificial devices. Duration and progress of integration are primarily influenced by visual, tactile, and proprioceptive perception. This paper describes the Int2Bot, a robot for the assessment of lower limb body schema integration during postural motion. The robot is designed to imitate human squatting movements to investigate the integration of artificial limbs into the body schema. The psychological and technical concepts as well as the mechatronic implementation and control are presented along with interface extensions comprising human knee position sensing and tactile user-feedback. The performance of the robot is examined by experiments excluding and including the human-robot interface and a human user. Those without interface show that the robot itself can perform considerably fast squats with 0.8 Hz, which comes up to maximum human capabilities. The computed torque control achieves good tracking results and fuzzy-based friction compensation further reduces position errors by up to 50%. Yet, results considering the vision-based part of the human-robot interface show that the setup is mainly limited due to delays in motion acquisition with the RGB-D sensor.

The acceptance of artificial devices like prostheses or other wearable robots requires their integration into the body schemas of the users. Different factors induce, influence and support the integration and acceptance of the device that substitutes or augments a part of the body. Previous studies have shown that the inducing and maintaining factors are visual, tactile and proprioceptive informations as well as their multi-sensory integration. This paper describes the vision-based part of the human-robot interface in the Int2Bot, which is a robot for the investigation of lower limb body schema integration during postural movements. The psychological approach and the technical setup of the robot, which is designed to imitate postural movements in the sagittal plane to imitate the human subject while performing squats, are outlined. To realize the imitation, an RGB-D sensor, in form of a Microsoft Kinect, is used to capture the subjects motions without contact and thereby avoid disturbances of body schema integration. For generation of the desired joint trajectories to be tracked by the control algorithm, different methods like an extended Kalman filter, inverse kinematics, an inverse kinematics algorithm using Jacobian transpose and approaches based on kinematic assumptions are presented, evaluated and compared based on human data. Benchmarking the results with data acquired using a professional motion capturing system shows that best overall joint angle estimations are achieved with the extended Kalman filter. Finally, the practical implementation within the robot is presented and the tracking behavior using the trajectories generated with the extended Kalman filter are analyzed.

The Prosthesis-User-in-the-Loop simulator concept represents an approach to integrate users to prosthetic development by a holistic simulation of gait with a prosthesis. It aims at a more user-centered design of lower limb prosthetic devices by utilizing user experience and assessment. As this requires a complex mechanical robot design and sophisticated control strategies that allow for restoring lost biomechanical function, this paper presents the conception and design of a hardware simulator for proof-of-concept studies of those issues. For those investigations, the ankle joints of healthy praticpants are locked mechanically to induce a temporary disability. The task of the simulator is to provide a simulation of physiological gait by artificially restoring ankle functionality. Therefore, the biody-namic behaviour of the locked ankle joint and the enviroment have to be mimicked mechnically. After introducing Prosthesis-User-in-the-Loop simulator idea, the conception of a proof-of-concept simulator is presented. From this, an analytical model is derived and inverse dynamics simulation are used for design. The resulting mechanism is limited to sagittal plane movements and thus has three degrees of freedom. The actuators are dimensioned to meet the requirements of walking motions in the human subject with maximum body height among the test population.

This paper introduces a robotic concept for the assessment of psychological factors in prosthetic design. Its aim is to imitate the postural movements of the participants while those are conducting squatting movements in order to investigate the integration of artificial limbs to the subject's body scheme. Therefore, the robot mimics the functionality and appearance of the human foot, shank and thigh as well as the ankle and knee joint. To induce a more realistic outer appearance, the hull of a shop-window mannequin is used as cladding. The robot is controlled by a computed torque control combined with a RGB-D sensor for the acquisition of the desired trajectories from the participant. In the test setup one leg of the participant is hidden from his view while the robot stands next to him and imitates the movements of this leg. This paper gives an insight in the theory of body schema integration. The concept of the robot is described and detailed information about the mechanical design and actuator dimensioning in accordance with psychological and biomechanical requirements are given. Furthermore, the concept of the human-machine interface, the control algorithm and simulations based on experimental data from a human subject are presented.