In April 1993 for the first time in the history of space flight, a small multisensory robot performed a number of prototype tasks on-board a spacecraft (spacelab D2 on shuttle COLUMBIA) in different operational modes that are feasible today, namely preprogrammed remotely controlled operations by the astronauts using a control ball and a stereo TV-monitor, as well as remotely controlled from ground via the human operator and machine intelligence. In these operational modes the robot successfully closed and opened connector plugs (bayonet closure), assembled structures from single parts and captured a free-floating object. Several key technologies have made this space robot technology experiment ROTEX a big success: multisensory gripper technology, local (shared autonomy) sensory feedback control concepts, and the powerful delay-compensating 3D-graphics simulation (predictive simulation) in the telerobotic ground station. This paper focusses on the tele-sensor-programming approach and the predictive simulation used for remote ground control.>

In early 1993 the space robot technology experiment ROTEX was flown with space-shuttle Columbia. A multisensory robot onboard the spacecraft successfully worked in autonomous modes, teleoperated by astronauts, as well as in different telerobotic ground control modes. These included online teleoperational and telesensor-programming: a task-level oriented programming technique involving learning-by-showing concepts in a virtual environment. The robot's key features were its multisensory gripper and the local sensory feedback schemes that are the basis for shared autonomy. The corresponding man-machine interface concepts, which use a six-degree-of-freedom non-force-reflecting control ball and visual feedback to the human operator, are explained. Stereographic simulation on the ground was used to predict not only the robot's free motion but even the sensor-based path refinement onboard. Prototype tasks performed by this space robot were the assembly of a truss structure, connecting/disconnecting an electrical plug (orbit replaceable unit exchange), and grasping free-floating objects.>

The paper first outlines the recent sensory developments in the DFVLR robotics lab, these are different force-torque sensors, laser range finders, inductive sensors and sensor balls for robot and 3D-computer-grafic control. With the example of our proposed multisensory arrangement implying vision, range sensing, force-torque and speech, the fine-motion planning and path generation techniques as developed in our lab are discussed. The special case of a two-arm cooperative robot using two force-torque-sensors is treated in more detail. Practical results are demonstrated in a film including cooperative two arm robot control.