We propose a novel pipeline for unknown object grasping in shared robotic autonomy scenarios. State-of-the-art methods for fully autonomous scenarios are typically learning-based approaches optimised for a specific end-effector, that generate grasp poses directly from sensor input. In the domain of assistive robotics, we seek instead to utilise the user’s cognitive abilities for enhanced satisfaction, grasping performance, and alignment with their high level task-specific goals. Given a pair of stereo images, we perform unknown object instance segmentation and generate a 3D reconstruction of the object of interest. In shared control, the user then guides the robot end-effector across a virtual hemisphere centered around the object to their desired approach direction. A physics-based grasp planner finds the most stable local grasp on the reconstruction, and finally the user is guided by shared control to this grasp. In experiments on the DLR EDAN platform, we report a grasp success rate of 87% for 10 unknown objects, and demonstrate the method’s capability to grasp objects in structured clutter and from shelves.

Assistive robots promise to be of great help to wheelchair users with motor impairments, for example for activities of daily living. Using shared control to provide task-specific assistance – for instance with the Shared Control Templates (SCT) framework – facilitates user control, even with low-dimensional input signals. However, designing SCTs is a laborious task requiring robotic expertise. To facilitate their design, we propose a method to learn one of their core components – active constraints – from demonstrated end-effector trajectories. We use a probabilistic model, Kernelized Movement Primitives, which additionally allows adaptation from user commands to improve the shared control skills, during both design and execution. We demonstrate that the SCTs so acquired can be successfully used to pick up an object, as well as adjusted for new environmental constraints, with our assistive robot EDAN.

Purposeful interaction with objects usually requires certain constraints to be respected, e.g. keeping a bottle upright to avoid spilling. In reinforcement learning, such constraints are typically encoded in the reward function. As a consequence, constraints can only be learned by violating them. This often precludes learning on the physical robot, as it may take many trials to learn the constraints, and the necessity to violate them during the trial-and-error learning may be unsafe. We have serendipitously discovered that constraint representations for shared control - in particular Shared Control Templates (SCTs) - are ideally suited for safely guiding RL. Representing constraints explicitly, rather than implicitly in the reward function, also simplifies the design of the reward function. The main advantage of the approach is safer, faster learning without constraint violations (even with sparse reward functions). We demonstrate this in a pouring task in simulation and on a real robot, where learning the task requires only 65 episodes in 16 minutes.

From robotic space assistance to healthcare robotics, there is increasing interest in robots that offer adaptable levels of autonomy. In this paper, we propose an action representation and planning framework that is able to generate plans that can be executed with both shared control and supervised autonomy, even switching between them during task execution. The action representation - Constraint Action Templates (CATs) - combine the advantages of Action Templates [1] and Shared Control Templates [2]. We demonstrate that CATs enable our planning framework to generate goal-directed plans for variations of a typical task of daily living, and that users can execute them on the wheelchair-robot EDAN in shared control or in autonomous mode.

Controlling a robotic arm to achieve manipulation tasks is challenging for humans. Especially if only low-dimensional input signals can be provided, as is often the case for users with motor impairments. Using shared control to provide task-specific guidance and constraints facilitates control – for instance with the Shared Control Templates (SCT) framework – and enables even complex activities of daily living to be performed successfully. However, designing SCTs is a laborious task requiring robotic expertise. To make such design easier and faster, we propose a method for semi-automatically designing SCTs on the basis of demonstrations. Furthermore, we propose two similarity metrics, and demonstrate how these can be used to transfer knowledge from one SCT to another. We demonstrate that the SCTs so acquired can be successfully used in shared control for everyday tasks such as opening a drawer or a cupboard on our assistive robot EDAN.

Light-weight robotic manipulators can be used to restore the manipulation capability of people with a motor disability. However, manipulating the environment poses a complex task, especially when the control interface is of low bandwidth, as may be the case for users with impairments. Therefore, we propose a constraint-based shared control scheme to define skills which provide support during task execution. This is achieved by representing a skill as a sequence of states, with specific user command mappings and different sets of constraints being applied in each state. New skills are defined by combining different types of constraints and conditions for state transitions, in a human-readable format. We demonstrate its versatility in a pilot experiment with three activities of daily living. Results show that even complex, high-dimensional tasks can be performed with a low-dimensional interface using our shared control approach.

Injuries, accidents, strokes, and other diseases can significantly degrade the capabilities to perform even the most simple activities in daily life. A large share of these cases involves neuromuscular diseases, which lead to severely reduced muscle function. However, even though affected people are no longer able to move their limbs, residual muscle function can still be existent. Previous work has shown that this residual muscular activity can suffice to apply an EMG-based user interface. In this paper, we introduce DLR's robotic wheelchair EDAN (EMG-controlled Daily Assistant), which is equipped with a torque-controlled, eight degree-of-freedom light-weight arm and a dexterous, five-fingered robotic hand. Using electromyography, muscular activity of the user is measured, processed and utilized to control both the wheelchair and the robotic manipulator. This EMG-based interface is enhanced with shared control functionality to allow for efficient and safe physical interaction with the environment.

In contrast to underactuated robotic hands the DLR AWIWI II hand of the David robot is fully controllable because each finger with 4 joints is actuated by 6 or 8 tendons respectively. For such fingers all joint angles (generalized positions) or joint torques (generalized forces) can be controlled independently. Usually, the specifications in joint space are converted to desired tendon forces or motor torques, which are regulated by an inner loop impedance controller. However, this conversion typically exhibits couplings between the components of the joint angle vector or the joint torque vector respectively, which arise when using the well known equations. Therefore the usual force control and position control schemes are reviewed and a generic computation of the desired tendon forces is presented. This is also done for the control of the Cartesian position and force at the finger endpoint. Thus the main contribution of the paper is the inhibition of couplings in joint space or at the Cartesian endpoint. This is demonstrated in simulations of the index finger of the DLR David hand.

Light-weight robotic manipulators in combination with power wheelchairs can help to restore the mobility of people with disabilities. While such systems are available on the market, they typically are limited to fully manual control modes. In research, shared control methods are employed, to increase the usability of these systems. Here, we present an additional extension, by introducing a whole-body control concept to the assistive robotic system EDAN. Combined with shared control, the whole-body controller allows the realization of complex tasks which necessitate the coordination of arm and platform, while ensuring compliant behavior resulting from the impedance control law. The implemented approach is analyzed and validated in an exemplary task of opening a door, passing through it and closing it afterwards. While this task would exceed the reachability of the arm in a classical approach, the combination of whole-body control with a shared control scheme allows for quick and efficient execution.