In fast changing assembly scenarios, it is required to adapt the task execution to the current state of the setup without extensive calibration routines. Therefore, it is important to estimate the geometric uncertainties and contact states during the assembly execution. We use a sequential Monte Carlo (SMC) method to track the relative poses between workpieces during a robotic assembly based on joint torque and position measurements only. In contrast to existing approaches, we focus on assembly tasks where the workpiece is not fixed in the workcell, but can, for example, slide on a table surface. We propose a new constraint-based propagation model for the SMC approach: a compensation motion for the samples dependent on the violation of contact constraints is derived. This allows us to track the motion of the workpieces in cases where a common random diffusion model fails. The method is evaluated with experiments using an assembly scenario with two KUKA LBR iiwa robot arms and shows accurate tracking performance.

This paper proposes a scheme to provide flexible semi-autonomous grasping capabilities to an assistive robotic manipulator. The testbed consists of a five-finger robotic hand mounted on a robotic arm. During teleoperation, the position of the hand is continuously controlled in the three translational degrees of freedom, and the user has no direct influence over the rotational behavior. The proposed semi-autonomy scheme assists the user for moving and orienting the hand towards the object, and automates the grasping process when it is triggered. The velocity commands issued by the user are enhanced using virtual fixtures, which are not preprogrammed to support one approach direction to the (known) object, but are adapted online according to the intended movement. The approach is validated with a psycho-physical user study where the participants grasp objects in a simulation environment using a SpaceMouse interface. This setting serves as a testbed for the target application in which disabled subjects will control the real robotic system with an interface based on bio-signals. The user study compares the semi-autonomous and the pure teleoperation modes in terms of objective and subjective measures, showing an increase in performance and a decrease in workload for the proposed semi-autonomous mode.

We present a virtual reality platform which addresses and integrates some of the currently challenging research topics in the field of virtual assembly: realistic and practical scenarios with several complex geometries, bimanual six-DoF haptic interaction for hands and arms, and intuitive navigation in large workspaces. We put an especial focus on our collision computation framework, which is able to display stiff and stable forces in 1 kHz using a combination of penalty- and constraint-based haptic rendering methods. Interaction with multiple arbitrary geometries is supported in realtime simulations, as well as several interfaces, allowing for collaborative training experiences. Performance results for an exemplary car assembly sequence which show the readiness of the system are provided.

In this work, we deal with the problem of planning a manipulation task for a robotic system composed of at least one dexterous arm and a dexterous multi-fingered hand. The goal of the local planner is to include both, the arm and the hand, in the execution of the task in a coordinated way. This is achieved by using the workspace of the hand which is computed offline. During the online planning, the current in-hand manipulation capability is evaluated taking advantage of the dimensions of the hand workspace and considering the task itself. Dynamic weights enable the computation of the instantaneous contributions of the two subsystems on the motion of the manipulated object. The method is evaluated in simulation as well as in several experiments on the real robot.

This work integrates visual and physical constraints to perform real-time depth-only tracking of articulated objects, with a focus on tracking a robot's manipulators and manipulation targets in realistic scenarios. As such, we extend DART, an existing visual articulated object tracker, to additionally avoid interpenetration of multiple interacting objects, and to make use of contact information collected via torque sensors or touch sensors. To achieve greater stability, the tracker uses a switching model to detect when an object is stationary relative to the table or relative to the palm and then uses information from multiple frames to converge to an accurate and stable estimate. Deviation from stable states is detected in order to remain robust to failed grasps and dropped objects. The tracker is integrated into a shared autonomy system in which it provides state estimates used by a grasp planner and the controller of two anthropomorphic hands. We demonstrate the advantages and performance of the tracking system in simulation and on a real robot. Qualitative results are also provided for a number of challenging manipulations that are made possible by the speed, accuracy, and stability of the tracking system.

Traditional approaches in grasping consider separately the planning of contact points and forces. This often leads to optimally simulated grasps that fail in a real execution when the forces applied at the chosen contact points cannot resist expected perturbations like the own object weight. This paper presents a method that combines a grasp planner with the computation of grasp forces, which provides contact points and contact forces that take into account joint and torque limits of the robotic hand for fingertip grasps. The planner starts with a given hand pose relative to the object, and uses the concept of reachable independent contact regions to obtain a set of force closure grasps. Within this set, the grasp that best counteracts a given external wrench is chosen. The evaluation examples illustrate how the capabilities of the robotic hand can be used more effectively by this joint planning approach.

In the quest for better human-machine interfaces (HMIs) for teleoperation and virtual reality, we hereby present the first integrated application of medical ultrasound imaging to remotely control a virtual piano playing environment in real-time.

In-hand manipulation with a multifinger hand is defined as changing the object pose from an initial to a final grasp configuration, while maintaining the fingertip contacts on the object surface. Given only the task constraints, represented as a desired motion of the object and an external force to be applied or resisted, the problem can be expressed as finding a good set of contact points on the object and a corresponding hand configuration compatible with the task to be executed. This paper presents a method for solving such problem, taking into account the kinematic structure and torque limits of the hand, the force closure condition (which must be guaranteed during the whole trajectory), and task compatibility. The feasibility of such method is tested in simulation of 2D and 3D examples.

Classic telepresence approaches allow a human to interact with a remote or a virtual reality environment (VR) with force feedback. Coupling with a remote robot can be used to work in dangerous environments without the human being on-site. The coupling with a VR system can be used for training and verification of task sequences or robotic actions.

The growth of space debris is becoming a serious problem. There is an urgent need for mitigation measures based on maintenance, repair and de-orbiting technologies. Our video presents a virtual reality framework in which robotic maintenance tasks of satellites can be simulated interactively. The two key components of this framework are a realistic virtual reality simulation and an immersive interaction device. The peculiarity of the virtual reality simulation is the combination of a physics engine based on Bullet with an extremely efficient haptic rendering algorithm inspired by an enhanced version of the Voxmap-Pointshell Algorithm. A central logic module controls all states and objects in the virtual world. To enable the human operator an optimal immersion into the virtual environment, the DLR bimanual haptic device is used as interaction device. Equipped with two light-weight robot arms, this device is able to provide realistic haptic feedback at both human hands, while covering the major part of human operator's workspace. The applicability of this system is enhanced by additional force sensors, active hand interfaces with an additional degree of freedom, smart safety technologies and intuitive robot data augmentation. Our platform can be used for verification or training purposes of robotic systems interacting in space environments.

This paper presents the identification of contact formations using force torque information. As force torque measurements do not map uniquely to their corresponding contact formations, three steps are performed: Initially, the wrench space for each contact formation is computed automatically. Then, a contact formation graph is augmented with a similarity index that reflects the similarity of contact formations with respect to their spanned wrench spaces. A particle filter is used to represent the likeliness of a contact formation given a force torque measurement. Finally, this probability distribution is resolved taking the similarity index, the transitions of the contact formation graph and the history of identified contact formations into account. This allows the recognition of the order of demonstrated contact formations by a measured set of forces and torques. The approach is verified by experiments.

Abstract-This article accompanies a video that presents a bimanual haptic device composed of two DLR/KUKA Light weight Robot (LWR) arms. The LWRs have similar dimensions to human arms, and can be operated in torque and position control mode at an update rate of 1kHz. The two robots are mounted behind the user, such that the intersecting workspace of the robots and the human arms becomes maximal. In order to enhance user interaction, various hand interfaces and additional tactile feedback devices can be used together with the robots. The presented system is equipped with a thorough safety architecture that assures safe operation for human and robot. Additionally, sophisticated control strategies improve performance and guarantee stability. The introduced haptic system is well suited for versatile applications in remote and virtual environments, especially for large unsealed movements.

Independent Contact Regions allow a robust finger placement on the object, despite of potential errors in finger position. They are computed without considering the kinematics of the end-effector, and are usually applied to off-line grasp planners. This paper presents an approach to obtain Reachable Independent Contact Regions by including the hand kinematics in the computational loop. The regions are computed in a short time, which allows real-time applications in virtual grasping. Potential applications of the proposed approach include regrasp planning, and dual-hand manipulation of objects.

This paper presents the graspability map, a novel approach to represent for a particular object the positions and orientations that a given mechanical hand can adopt to achieve a force closure precision grasp. The algorithm is based on the intersection between the fingertip workspaces and the object, plus the verification of a necessary condition for force closure grasps. The maps are computed offline and can be used for comparing the grasp capabilities of different mechanical hands with respect to some benchmark objects. The maps have also potential applications in online grasp and manipulation planning.

This paper generalizes the Time Domain Passivity Control concept originally introduced by J.-H. Ryu et al. (2004) in order to work for multi-degree of freedom (DoF) haptic systems with time delay. Its energy computation (named passivity observer) factors in the phase shift caused by time delay, and is improved by an energy estimation. Moreover, the variable damping of the passivity controller is generalized such that weighting by the mass matrix of the haptic device is possible. This transformation takes into account the direction-dependent inertia of multi-DoF haptic devices. Furthermore, a stability boundary for this damping is introduced for one as well as for several DoF allowing for high energy dissipation. Additionally, it is briefly shown that one single multi-DoF Cartesian passivity controller is advantageous compared to independent single-DoF passivity controllers in each joint of the haptic device. Finally, the generalized Time Domain Passivity Controller is experimentally verified using the DLR light weight robot arm as haptic device.