Efficient Human-Robot Interaction requires flexible role switching and robust role distribution. Leader and follower agents are mostly recognized based on interaction forces. This often comes with the drawback of reduced robustness and the necessity of models of the robot's environment or human be-haviour. This video presents the recently proposed Elasto-Plastic Robot Compliance (EPRC) concept that, being power-based, additionally considers the robot velocity in role recognition. Thus, active environments can be detected inducing enhanced virtual plastic compliance of the robot. Such plastic compliance refers to evasive robot motions induced specifically by active environments, after which the robot is not pushed back to the initial point of contact in contrast to elastic compliance. The video further presents advantages as role distribution specific to different degrees-of-freedom, independence from model generation and unrestricted force application.

Virtual Fixtures facilitate teleoperation, for in-stance by guiding the human operator. Developing these Virtual Fixtures in tasks with tight tolerances remains challenging. Fixtures with a high stiffness allow for more precise guidance, whereas a lower stiffness is required to allow for corrections. We observed that many assembly operations can be split into different phases - approaching, positioning, in-contact manipulation - each with different accuracy requirements. Therefore, we propose to use multi-modal fixtures, satisfying the different requirements of these phases: i.e. a position-based Trajectory Fixture for approaching and a more accurate Visual Servoing Fixture for the positioning phase. A state estimation and arbitration component ensures smooth transitions between the fixtures to provide optimal support for the operator and to achieve global availability paired with local precision at the same time. It also allows a high stiffness to be used throughout, thus achieving good guidance for all phases. The approach is validated in an application from a space scenario, consisting of the assembly of a CubeSat subsystem. The empirical results from a pilot study on this task show that our approach is faster and requires less interaction force from the operator than the baseline method.

SwarmRail represents a novel solution to overhead manipulation from a mobile unit that drives in an aboveground rail-structure. The concept is based on the combination of omnidirectional mobile platform and L-shaped rail profiles that form a through-going central gap. This gap makes possible mounting a robotic manipulator arm overhead at the underside of the mobile platform. Compared to existing solutions, SwarmRail enables continuous overhead manipulation while traversing rail crossings. It also can be operated in a robot swarm, as it allows for concurrent operation of a group of mobile SwarmRail units inside a single rail network. Experiments on a first functional demonstrator confirm the functional capability of the concept. Potential fields of applications reach from industry over logistics to vertical farming.

Telerobotic systems must adapt to new environmental conditions and deal with high uncertainty caused by long-time delays. As one of the best alternatives to human-level intelligence, Reinforcement Learning (RL) may offer a solution to cope with these issues. This paper proposes to integrate RL with the Model Mediated Teleoperation (MMT) concept. The teleoperator interacts with a simulated virtual environment, which provides instant feedback. Whereas feedback from the real environment is delayed, feedback from the model is instantaneous, leading to high transparency. The MMT is realized in combination with an intelligent system with two layers. The first layer utilizes Dynamic Movement Primitives (DMP) which accounts for certain changes in the avatar environment. And, the second layer addresses the problems caused by uncertainty in the model using RL methods. Augmented reality was also provided to fuse the avatar device and virtual environment models for the teleoperator. Implemented on DLR's Exodex Adam hand-arm haptic exoskeleton, the results show RL methods are able to find different solutions when changes are applied to the object position after the demonstration. The results also show DMPs to be effective at adapting to new conditions where there is no uncertainty involved.

Advances in exoskeleton technology now enable interacting with rigid objects in a virtual or remote environment using one's hand and fingertips. However, interaction with non-solid materials - such as liquids, sediments and regolith - alongside solids, can greatly extend the versatility of this technology. Rendering rigid objects adequately requires a control loop with high update rates, whereas fluid dynamics equations are computationally expensive. To accommodate this, the fluid dynamics can be simplified - particularly for fluids with high viscosity - resulting in a fast-to-calculate model to enabling haptic rendering of viscous fluids and rigid bodies simultaneously using DLR's Exodex Adam hand exoskeleton. Viscosity as a proprioceptive cue of fluids can be presented to the human through force feedback at multiple points on the human hand - fingers and palm - letting the user interact with a virtual environment in a more natural way and making the experience more immersive. We carry out two user studies to investigate the human perception abilities of virtual fluids rendered with simplified dynamics, and the discernability of different viscosity in virtual fluids compared real fluids. Results show that virtual media can give the user the perception of interacting with a fluid, even with simplified models, at a high update frequency. Furthermore, the material discernibility corresponds well to actual interaction with real viscous fluids. This shows great promise forward for haptic in-hand interaction in fluid and mixed media environments.

Accurate real-time estimation of the pose and configuration of the human hand attached to a dexterous haptic input device is crucial to improve the interaction possibilities for teleoperation and in virtual and augmented reality. In this paper, we present an approach to reconstruct the pose of the human hand and the joint angles of the fingers when wearing a novel fixed-base (grounded) hand exoskeleton. Using a kinematic model of the human hand built from MRI data, we can reconstruct the hand pose and joint angles without sensors on the human hand, from attachment points on the first three fingers and the palm. We test the accuracy of our approach using motion capture as a ground truth. This reconstruction can be used to determine contact geometry and force-feedback from virtual or remote objects in virtual reality or teleoperation.

This video presents a collision avoidance framework for mechanisms with complex geometries. The performance of the framework is showcased with the haptic interface HUG [3]. We are able to avoid contacts with the robot links and with moving objects in the environment in 1 kHz. The main contribution of our approach is its generic and extensible nature; it can be applied to any mechanism consisting of arbitrarily complex rigid bodies, in contrast to common solutions that use simplified models [2], [7]. In the preprocessing phase, first, the kinematic chain of the mechanism is described [1]. Second, we generate voxelized distance fields and point-sphere hierarchies for the geometry of each mechanism link and each object in the environment [6]. After that, our system requires only the joint angles and information of the environment state (e.g., object poses tracked by optical sensors) to compute collision avoidance forces. At runtime, each link is artificially dilated by a safety isosurface. If a point of an object goes through this surface, a normal force scaled by its penetration depth is computed and applied to the corresponding link. If humans are generically modeled as mechanisms and properly tracked, our system can also prevent collisions with them, ensuring save human-machine collaboration. Figure 1 illustrates the framework and its basic components. The multi-body collision computation architecture was first developed for virtual maintenance simulations with haptic feedback [5], [4], and thereafter extended to collision avoidance of mechanisms. A first prototype was previously published in [8].

This work presents an evaluation study in which the effects of a penalty-based and a constraint-based haptic rendering algorithm on the user performance and perception are analyzed. A total of N = 24 participants performed in a within-design study three variations of peg-in-hole tasks in a virtual environment after trials in an identically replicated real scenario as a reference. In addition to the two mentioned haptic rendering paradigms, two haptic devices were used, the HUG and a Sigma.7, and the force stiffness was also varied with maximum and half values possible for each device. Both objective (time and trajectory, collision performance, and muscular effort) and subjective ratings (contact perception, ergonomy, and workload) were recorded and statistically analyzed. The results show that the constraint-based haptic rendering algorithm with a lower stiffness than the maximum possible yields the most realistic contact perception, while keeping the visual inter-penetration between the objects roughly at around 15% of that caused by penalty-based algorithm (i.e., non perceptible in many cases). This result is even more evident with the HUG, the haptic device with the highest force display capabilities, although user ratings point to the Sigma.7 as the device with highest usability and lowest workload indicators. Altogether, the paper provides qualitative and quantitative guidelines for mapping properties of haptic algorithms and devices to user performance and perception.

Collision detection and force computation between complex geometries are essential technologies for virtual reality and robotic applications. Penalty-based haptic rendering algorithms provide a fast collision computation solution, but they cannot avoid the undesired interpenetration between virtual objects, and have difficulties with thin non-watertight geometries. God object methods or constraint-based haptic rendering approaches have shown to solve this problem, but are typically complex to implement and computationally expensive. This paper presents an easy-to-implement god object approach applied to six-DoF penalty-based haptic rendering algorithms. Contact regions are synthesized to penalty force and torque values and these are used to compute the position of the god object on the surface. Then, the pose of this surface proxy is used to render stiff and stable six-DoF contacts with friction. Independently of the complexity of the used geometries, our implementation runs in only around 5 μs and the results show a maximal penetration error of the resolution used in the penalty-based haptic rendering algorithm.

We present a virtual reality platform which addresses and integrates some of the currently challenging research topics in the field of virtual assembly: realistic and practical scenarios with several complex geometries, bimanual six-DoF haptic interaction for hands and arms, and intuitive navigation in large workspaces. We put an especial focus on our collision computation framework, which is able to display stiff and stable forces in 1 kHz using a combination of penalty- and constraint-based haptic rendering methods. Interaction with multiple arbitrary geometries is supported in realtime simulations, as well as several interfaces, allowing for collaborative training experiences. Performance results for an exemplary car assembly sequence which show the readiness of the system are provided.

Controlling haptic devices in an optimal way is crucial to achieve both, best performance and most realistic haptic feedback. The present article investigates control design of a single degree of freedom haptic device that is interacting with a human operator and rendering a virtual wall affected by time delay. To this end, it suggests different optimization criteria based on the step response of the haptic system. These criteria cover fundamental requirements for efficiently using haptic devices, particularly fast settling and minimum overshoot. For each criterion an optimal path and point inside the stable region of the virtual wall parameters is derived. These optima depend mainly on the system mass, sampling time and time delay. This approach is supported by experiments on two devices, a Falcon haptic device and a DLR/KUKA Light-Weight Robot arm.

The growth of space debris is becoming a serious problem. There is an urgent need for mitigation measures based on maintenance, repair and de-orbiting technologies. Our video presents a virtual reality framework in which robotic maintenance tasks of satellites can be simulated interactively. The two key components of this framework are a realistic virtual reality simulation and an immersive interaction device. The peculiarity of the virtual reality simulation is the combination of a physics engine based on Bullet with an extremely efficient haptic rendering algorithm inspired by an enhanced version of the Voxmap-Pointshell Algorithm. A central logic module controls all states and objects in the virtual world. To enable the human operator an optimal immersion into the virtual environment, the DLR bimanual haptic device is used as interaction device. Equipped with two light-weight robot arms, this device is able to provide realistic haptic feedback at both human hands, while covering the major part of human operator's workspace. The applicability of this system is enhanced by additional force sensors, active hand interfaces with an additional degree of freedom, smart safety technologies and intuitive robot data augmentation. Our platform can be used for verification or training purposes of robotic systems interacting in space environments.

This work presents an evaluation study of two different collision feedback modalities for virtual assembly verification: visual and force feedback. Forty-three subjects performed several assembly tasks (peg-in-hole, narrow passage) designed with two levels of difficulty. The used haptic rendering algorithm is based on voxel and point data-structures. Both objective - time and collision performance - and subjective measures have been recorded and analyzed. The comparison of the feedback modalities revealed a clear and highly significant superiority of force feedback in virtual assembly scenarios. The objective data shows that whereas the assembly time is similar in most cases for both conditions, force collision feedback yields significantly smaller collision forces, which indicate higher assembly precision. The subjective ratings of the participants define the force feedback condition as the most appropriate for determining clearances and correcting collision configurations, being the best suited modality to predict mountability.

In the present study, a vibrotactile feedback device for spatial guidance was evaluated in a tracking task paradigm. Participants (N = 18) had to translate and rotate virtual objects according to the vibrotactile vs. verbal cues without visual information. Both types of spatial guidance were evaluated using objective performance data (i.e. speed, accuracy) as well as subjective judgments. Results indicate that distinguishing spatial cues during the translational task was more difficult when being guided by vibrotactile feedback compared to verbal feedback. Nevertheless, individuals with vibrotactile guidance showed better performance at rotational tasks. Implications for the further design process and other areas of application are discussed.

Abstract-This article accompanies a video that presents a bimanual haptic device composed of two DLR/KUKA Light weight Robot (LWR) arms. The LWRs have similar dimensions to human arms, and can be operated in torque and position control mode at an update rate of 1kHz. The two robots are mounted behind the user, such that the intersecting workspace of the robots and the human arms becomes maximal. In order to enhance user interaction, various hand interfaces and additional tactile feedback devices can be used together with the robots. The presented system is equipped with a thorough safety architecture that assures safe operation for human and robot. Additionally, sophisticated control strategies improve performance and guarantee stability. The introduced haptic system is well suited for versatile applications in remote and virtual environments, especially for large unsealed movements.

This paper generalizes the Time Domain Passivity Control concept originally introduced by J.-H. Ryu et al. (2004) in order to work for multi-degree of freedom (DoF) haptic systems with time delay. Its energy computation (named passivity observer) factors in the phase shift caused by time delay, and is improved by an energy estimation. Moreover, the variable damping of the passivity controller is generalized such that weighting by the mass matrix of the haptic device is possible. This transformation takes into account the direction-dependent inertia of multi-DoF haptic devices. Furthermore, a stability boundary for this damping is introduced for one as well as for several DoF allowing for high energy dissipation. Additionally, it is briefly shown that one single multi-DoF Cartesian passivity controller is advantageous compared to independent single-DoF passivity controllers in each joint of the haptic device. Finally, the generalized Time Domain Passivity Controller is experimentally verified using the DLR light weight robot arm as haptic device.

In virtual assembly verification or remote maintenance tasks, bimanual haptic interfaces play a crucial role in successful task completion. This paper proposes a method for objectively comparing how well a haptic interface covers the reachable workspace of human arms. Two system configurations are analyzed for a recently introduced haptic device that is based on two DLR-KUKA light weight robots: the standard configuration, where the device is opposite the human operator, and the ergonomic configuration, where the haptic device is mounted behind the human operator. The human operator directly controls the robotic arms using handles. The analysis is performed using a representation of the robot arm workspace. The merits of restricting the comparisons to the most significant regions of the human workspace are discussed. Using this method, a greater workspace correspondence for the ergonomic configuration was shown.

Robot Programming by Demonstration requires comprehending the usage of a robotic system. This article is about accelerating the training of these skills, using the example of a DLR/KUKA light-weight robot. An augmented reality and a virtual reality setup are presented that aim to demonstrate and evaluate skills transfer of two different sub-tasks of this system: Avoiding robot singularities and setting correct compliance parameters. For this purpose training accelerators are introduced for visualising robot singularities, exploring robot singularities and feeling compliance parameters. An evaluation procedure for all three accelerators is suggested and has been performed on the first two. As interesting evaluation result a contrast to the Cognitive Theory of Multimedia Learning hypothesis could be observed: Additional visual information on the robot singularities impairs the participants' performance.

We present a benchmarking suite for rigid object collision detection and collision response schemes. The proposed benchmarking suite can evaluate both the performance as well as the quality of the collision response. The former is achieved by densely sampling the configuration space of a large number of highly detailed objects; the latter is achieved by a novel methodology that comprises a number of models for certain collision scenarios. With these models, we compare the force and torque signals both in direction and magnitude.

Recent analysis on the stability boundary for haptic rendering assumed a stabilizing effect through a human operator holding a haptic device, without considering his/her dynamics directly. This paper derives stability boundaries of a linear model of a haptic system including those dynamics. It shows that all three elements of the human arm modeled as mass-spring-damper system contribute to stability. The haptic system itself is composed of a haptic device colliding with a virtual wall modeled as time-delayed discrete-time spring-damper system. Furthermore, the article proves that the recently found linear stability condition for haptic devices of Gil et al. still holds if a human is holding the haptic device. Finally, a relation to Colgatepsilas passivity condition defining a robustly stable region is given.

The influence of viscous damping and delay on the stability of haptic systems is studied in this paper. The stability boundaries have been found by means of different approaches. Although the shape of these stability boundaries is quite complex, a new linear condition which summarizes the relation between virtual stiffness, viscous damping and delay is proposed. This condition is independent of the mass of the haptic device. The theoretical results are supported by simulations and experimental data using the DLR light-weight robot.

Physical damping is increasing the z-width of haptic simulations. This paper derives the normalized stability boundaries for physically damped one degree of freedom haptic devices colliding with a virtual wall represented as spring-damper system. These boundaries are independent of the haptic device's mass and the sampling time. Furthermore, the dependency of the maximum stable virtual stiffness is discussed. Moreover, this paper illustrates that the passive region which is defined by Colgate's passivity condition is a subset inside the stable region for undelayed systems, but not for delayed systems