Research has focused on developing control strategies that allow to work with imprecisely calibrated or even uncalibrated robot systems. A popular approach is to close the control loop with visual feedback (visual servoing), even though this requires continuous, high rate image processing. In contrast, we developed a strategy that can compensate calibration errors by integrating sparse visual feedback asynchronously. This strategy was motivated by neuroscientific studies which show that human grasping definitely requires only sparse asynchronous visual feedback, even with disturbed visual input. We analyze the performance of our control strategy on a robotic hand-eye system consisting of a 6 d.o.f. manipulator and a pan-tilt head with a stereo camera system. To analyze thoroughly which calibration errors can be compensated by sparse feedback, we furthermore built a complete simulation model of the real hand-eye system, including the vision modules. Taken together, experimental and simulation results show that calibration errors can be compensated with a very moderate amount of (asynchronous) visual feedback. Thus, the proposed control strategy can indeed be seen as a serious alternative to visual servoing for real robotics applications. This shows that robotics can profit from taking a closer look at the results of neuroscience.

In the field of robot motion control, visual servoing has been proposed as the suitable strategy to cope with imprecise models and calibration errors. Remaining problems such as the necessity of a high rate of visual feedback are deemed to be solvable by the development of real-time vision modules. However, human grasping, which still outshines its robotic counterparts especially with respect to robustness and flexibility, definitely requires only sparse, asynchronous visual feedback. We therefore examined current neuroscientific models for the control of human reach-to-grasp movements with the emphasis lying on the visual control strategy used. From this, we developed a control model that unifies the two robotic strategies look-then-move and visual servoing, thereby compensating the problems that each strategy shows when used alone.

The pervasiveness of computers in everyday life coupled with recent rapid advances in computer technology have created both the need and the means for sophisticated human-computer interaction (HCI) technology. Despite all the progress in computer technology and robotic manipulation, the interfaces for controlling manipulators have changed very little in the last decade. Therefore human-computer interfaces for controlling robotic manipulators are of great interest. A flexible and useful robotic manipulator is one capable of movement in three translational degrees of freedom, and three rotational degrees of freedom. In addition to research labs, six degree of freedom robots can be found in construction areas or other environments unfavorable for human beings. This paper proposes an intuitive and convenient visually guided interface for controlling a robot with six degrees of freedom. Two orthogonal cameras are used to track the position and the orientation of the hand of the user. This allows the user to control the robotic arm in a natural way.

In the field of hand-eye coordination, most state-of-the-art systems still require the user to select the grasping points manually. We present a system which autonomously determines 3D grasping points on unknown objects from a pair of greyscale images. The object to be grasped is segmented automatically when put into the scene. Grasping points are searched on the object silhouette; their stability is evaluated by a heuristic algorithm, primarily based on the skeleton of the region. The 3D grasping pose is estimated by triangulation using a simplified geometrical model of the camera system; the corresponding points in the second image are determined via dynamic programming. The whole system has been implemented and validated on the experimental hand-eye system MINERVA.

This paper presents an approach to the recognition of articulated 3D objects in monocular video images. A hierarchical object representation models objects as a composition of rigid components which are explicitly connected by specific kinematic constraints, e.g., rotational and/or translational joints. The recognition task follows this tree-like structure by first estimating the 3D pose of the static component (root) and afterwards determining the relative 3D pose of the remaining components recursively. This method limits the search space for the actual correspondences between image and model features and copes with the problem of self-occlusion. Experiments in the context of autonomous, mobile robots show the practicability of this approach.

The design of mobile robots which can cope with unexpected disturbances like obstacles or misplaced objects is an active field of research. Such an autonomous robot assesses the situation by comparing data from one or more sensors with an internal representation of its environment. In this paper we present a hierarchically structured world model that combines a general geometric object representation with sensor- and task-specific features and therefore can be used for various sensors and perception tasks. By predicting only those features that, first, can be detected by the sensor and, secondly, are relevant for the current perception task, sensor data interpretation gets faster and more robust. This is illustrated at an exemplary perception task concerning the video-based determination of the joint states of articulated objects.