We propose a novel pipeline for unknown object grasping in shared robotic autonomy scenarios. State-of-the-art methods for fully autonomous scenarios are typically learning-based approaches optimised for a specific end-effector, that generate grasp poses directly from sensor input. In the domain of assistive robotics, we seek instead to utilise the user’s cognitive abilities for enhanced satisfaction, grasping performance, and alignment with their high level task-specific goals. Given a pair of stereo images, we perform unknown object instance segmentation and generate a 3D reconstruction of the object of interest. In shared control, the user then guides the robot end-effector across a virtual hemisphere centered around the object to their desired approach direction. A physics-based grasp planner finds the most stable local grasp on the reconstruction, and finally the user is guided by shared control to this grasp. In experiments on the DLR EDAN platform, we report a grasp success rate of 87% for 10 unknown objects, and demonstrate the method’s capability to grasp objects in structured clutter and from shelves.

As labor shortage increases in the health sector, the demand for assistive robotics grows. However, the needed test data to develop those robots is scarce, especially for the application of active 3D object detection, where no real data exists at all. This short paper counters this by introducing such an annotated dataset of real environments. The captured environments represent areas which are already in use in the field of robotic health care research. We further provide ground truth data within one room, for assessing SLAM algorithms running directly on a health care robot.

From robotic space assistance to healthcare robotics, there is increasing interest in robots that offer adaptable levels of autonomy. In this paper, we propose an action representation and planning framework that is able to generate plans that can be executed with both shared control and supervised autonomy, even switching between them during task execution. The action representation - Constraint Action Templates (CATs) - combine the advantages of Action Templates [1] and Shared Control Templates [2]. We demonstrate that CATs enable our planning framework to generate goal-directed plans for variations of a typical task of daily living, and that users can execute them on the wheelchair-robot EDAN in shared control or in autonomous mode.

Controlling a robotic arm to achieve manipulation tasks is challenging for humans. Especially if only low-dimensional input signals can be provided, as is often the case for users with motor impairments. Using shared control to provide task-specific guidance and constraints facilitates control – for instance with the Shared Control Templates (SCT) framework – and enables even complex activities of daily living to be performed successfully. However, designing SCTs is a laborious task requiring robotic expertise. To make such design easier and faster, we propose a method for semi-automatically designing SCTs on the basis of demonstrations. Furthermore, we propose two similarity metrics, and demonstrate how these can be used to transfer knowledge from one SCT to another. We demonstrate that the SCTs so acquired can be successfully used in shared control for everyday tasks such as opening a drawer or a cupboard on our assistive robot EDAN.

The cooperation between robots and astronauts will become a core element of future space missions. This is accompanied by the demand for suitable input devices. An interface based on electromyography (EMG) represents a small, light, and wearable device to generate a continuous three-dimensional (3D) control signal from voluntarily muscle activation of the operator's arm. We analyzed the influence of microgravity on task performance during a two-dimensional (2D) task on a screen. Six subjects performed aiming and tracking tasks in parabolic flights. Three different levels of fixation-fixed feet using foot straps, semi-free by using a foot rail, and free-floating feet-are tested to investigate how much user fixation is required to operate via the interface. The user study showed that weightlessness affects the usage of the interface only to a small extent. Success rates between 89${\%}$ and 96${\%}$ are reached within all conditions during microgravity. A significant effect between 0 and 1G could not be identified for the test series of fixed and semi-free feet, while free-floating feet showed significantly worse results in fine and gross motion times in 0G compared to ground tests (with success rates of 92${\%}$ for 0G and 99${\%}$ for 1G). Further adaptation to the altered proprioception may be needed here. Hence, foot rails as already mounted in the International Space Station (ISS) would be sufficient to use the interface in weightlessness. Low impact of microgravity, high success rates, and an easy handling of the system, indicates a high potential of an EMG-based interface for teleoperation in space.

Light-weight robotic manipulators can be used to restore the manipulation capability of people with a motor disability. However, manipulating the environment poses a complex task, especially when the control interface is of low bandwidth, as may be the case for users with impairments. Therefore, we propose a constraint-based shared control scheme to define skills which provide support during task execution. This is achieved by representing a skill as a sequence of states, with specific user command mappings and different sets of constraints being applied in each state. New skills are defined by combining different types of constraints and conditions for state transitions, in a human-readable format. We demonstrate its versatility in a pilot experiment with three activities of daily living. Results show that even complex, high-dimensional tasks can be performed with a low-dimensional interface using our shared control approach.

Injuries, accidents, strokes, and other diseases can significantly degrade the capabilities to perform even the most simple activities in daily life. A large share of these cases involves neuromuscular diseases, which lead to severely reduced muscle function. However, even though affected people are no longer able to move their limbs, residual muscle function can still be existent. Previous work has shown that this residual muscular activity can suffice to apply an EMG-based user interface. In this paper, we introduce DLR's robotic wheelchair EDAN (EMG-controlled Daily Assistant), which is equipped with a torque-controlled, eight degree-of-freedom light-weight arm and a dexterous, five-fingered robotic hand. Using electromyography, muscular activity of the user is measured, processed and utilized to control both the wheelchair and the robotic manipulator. This EMG-based interface is enhanced with shared control functionality to allow for efficient and safe physical interaction with the environment.

Light-weight robotic manipulators in combination with power wheelchairs can help to restore the mobility of people with disabilities. While such systems are available on the market, they typically are limited to fully manual control modes. In research, shared control methods are employed, to increase the usability of these systems. Here, we present an additional extension, by introducing a whole-body control concept to the assistive robotic system EDAN. Combined with shared control, the whole-body controller allows the realization of complex tasks which necessitate the coordination of arm and platform, while ensuring compliant behavior resulting from the impedance control law. The implemented approach is analyzed and validated in an exemplary task of opening a door, passing through it and closing it afterwards. While this task would exceed the reachability of the arm in a classical approach, the combination of whole-body control with a shared control scheme allows for quick and efficient execution.

This paper examines the capability to incorporate spatial force feedback to the human toe when teleoperating a robotic arm in a force task. Due to the growing complexity of teleoperated systems new means of feedback get increasingly important. To investigate the viability of spatial toe-feedback, experiments with 12 subjects were conducted. The participants had to teleoperate a DLR Light-Weight Robot (LWR) via optical tracking of one finger in order to push a toy train. The orientation of the rail was unknown to the subject and had to be explored using the haptic feedback - a three-dimensional spatial force to the toe, reflecting the contact forces at the robotic end-effector - in absence of visual feedback. The rail was mounted in one of four possible orientations (differences of 45°). The main task of the experiment was to identify the present orientation. In our study subjects could successfully identify the orientation of the rail in more than two thirds of all trials (68%). In almost half of the trials (44%) the subjects were able to move the train along the rails long enough to reach the bumpers at the end and identify them as such. Assuming no feedback would be provided at all, the first metric has a chance level of 25%, and reaching the bumper can be considered impossible. Thus, we can conclude that humans can incorporate spatial force feedback to the toe into their sensorimotor loop.

Tool-held hitting tasks, like hammering a nail or striking a ball with a bat, require humans, and robots, to purposely collide and transfer momentum from their limbs to the environment. Due to the vibrational dynamics, every tool has a location where a hit is most efficient results in minimal tool vibrations, and consequently maximum energy transfer to the environment. In sports, this location is often referred to as the “sweet spot” of a bat, or racquet. Our recent neuroscience study suggests that humans optimize hits by using the jerk and torque felt at their hand. Motivated by this result, in this work we first analyze the vibrational dynamics of an end-effector-held bat to understand the signature projected by a sweet spot on the jerk and torque sensed at the end-effector. We then use this analysis to develop a controller for a robotic “baseball hitter”. The controller enables the robot-hitter to iteratively adjust its swing trajectory to ensure that the contact with the ball occurs at the sweet spot of the bat. We tested the controller on the DLR LWR III manipulator with three different bats. Like a human, our robot hitter is able to optimize the energy transfer, specifically maximize the ball velocity, during hits, by using its end effector position and torque sensors, and without any prior knowledge of the shape, size or material of the held bat.

A fundamental problem in human-robot collaboration is to ensure safety for humans being located in the workspace of the robot. Several new robots, referred to as collaborative robots, are pushing into the market. Most of these so-called co-bots have similar properties. They are small, lightweight and designed with big roundings to ensure safety in the case of a collision with a human. Equipped with torque sensors, external torque observers, tactile skins, etc., they are able to stop the robot when an emergency occurs. While developing more and more co-bots, the main focus lies on the robot itself. But to make a robot deployable, a special tool for a defined task is needed. These tools are often sharp-edged and dangerous in case of a collision with a human. In this paper we present a new safety module for robots to ensure safety for different tools in collaborative tasks. This module, filled with air pressure during the robot motion, covers mounted tools and carried workpieces. In case of a non or very slow moving robot, the safety module is able to pull back and the tool is uncovered. In our experiments we found out that we can increase the velocity up to 1 m/s while satisfying the requirements of the ISO/TS 15066 and retain the full functionality of the tool.

This paper proposes a scheme to provide flexible semi-autonomous grasping capabilities to an assistive robotic manipulator. The testbed consists of a five-finger robotic hand mounted on a robotic arm. During teleoperation, the position of the hand is continuously controlled in the three translational degrees of freedom, and the user has no direct influence over the rotational behavior. The proposed semi-autonomy scheme assists the user for moving and orienting the hand towards the object, and automates the grasping process when it is triggered. The velocity commands issued by the user are enhanced using virtual fixtures, which are not preprogrammed to support one approach direction to the (known) object, but are adapted online according to the intended movement. The approach is validated with a psycho-physical user study where the participants grasp objects in a simulation environment using a SpaceMouse interface. This setting serves as a testbed for the target application in which disabled subjects will control the real robotic system with an interface based on bio-signals. The user study compares the semi-autonomous and the pure teleoperation modes in terms of objective and subjective measures, showing an increase in performance and a decrease in workload for the proposed semi-autonomous mode.

Motion planning in robotics is a very large field of research. Many different approaches have been developed to create smooth trajectories for robot movement. For example there are optimization algorithms, which optimize kinematic or dynamic properties of a trajectory. Furthermore, nonlinear programming methods like e.g. optimal control, or polynomial based methods are widely used for trajectory generation. Most of these techniques are used to calculate a trajectory in advance, or they are limited to create point-to-point motions, where the robot needs to stop when switching to the next target point, especially, when interpolating in rotational space. In this paper, we combine a low-pass filter and spherical linear interpolation to realize a velocity-limited online trajectory generator for robot orientations in quaternion space. We use the developed motion generator for mirroring a human arm motion with a robot, recorded by a low frequency visual tracking. Using the proposed method, we can replicate the motion of the operator's arm with very little delay and thereby achieve an easy-to-use interface. Furthermore, as we can strictly limit the velocity of the generated motion, the approach can safely be used in human robot collaboration applications.

The development of new, light robotic systems has opened up a wealth of human-robot interaction applications. In particular, the use of robot manipulators as personal assistant for the disabled is realistic and affordable, but still requires research as to the brain-computer interface. Based on our previous work with tetraplegic individuals, we investigate the use of low-cost yet stable surface Electromyography (sEMG) interfaces for individuals with Spinal Muscular Atrophy (SMA), a disease leading to the death of neuronal cells in the anterior horn of the spinal cord; with sEMG, we can record remaining active muscle fibers. We show the ability of two individuals with SMA to actively control a robot in 3.5D continuously decoded through sEMG after a few minutes of training, allowing them to regain some independence in daily life. Although movement is not nearly as fast as natural, unimpaired movement, reach and grasp success rates are near 100% after 50s of movement.

In this paper we describe and practically demonstrate a robotic arm/hand system that is controlled in real-time in 6D Cartesian space through measured human muscular activity. The soft-robotics control architecture of the robotic system ensures safe physical human robot interaction as well as stable behaviour while operating in an unstructured environment. Muscular control is realised via surface electromyography, a non-invasive and simple way to gather human muscular activity from the skin. A standard supervised machine learning system is used to create a map from muscle activity to hand position, orientation and grasping force which then can be evaluated in real time - the existence of such a map is guaranteed by gravity compensation and low-speed movement. No kinematic or dynamic model of the human arm is necessary, which makes the system quickly adaptable to anyone. Numerical validation shows that the system achieves good movement precision. Live evaluation and demonstration of the system during a robotic trade fair is reported and confirms the validity of the approach, which has potential applications in muscle-disorder rehabilitation or in teleoperation where a close-range, safe master/slave interaction is required, and/or when optical/magnetic position tracking cannot be enforced.

In this overview paper we present current work on safety analysis for physical Human-Robot Interaction (pHRI) and motion control methods for robotic co-workers. In particular, we introduce the analysis tools for investigating the potential injury a human would suffer during robot-human impacts. Furthermore, we outline our concept for establishing a procedure towards standardized crash testing in robotics with automobile crash-test dummies. Since it is only possible to investigate blunt impacts with these devices, we developed a drop testing setup for analyzing soft-tissue injury in robotics from a biomechanics perspective. In the second part of the paper, some of our methods for task preserving and task relaxing motion schemes are described, which enable collision avoidance in real-time. The algorithms are well suited to work in an integrated fashion with the soft robotics control developed for the DLR Lightweight Robot III (LWR-III). In addition, it is shown how the torque sensing capabilities of the robot can be used to support reactive motion schemes. Finally, an overview of our human-friendly control architecture for the LWR-III is given, which unifies the rich bundle of developed methods for this manipulator.