This paper reports that the superposition of a small set of behaviors, learned via teleoperation, can lead to robust completion of an articulated reach-and-grasp task. The results support the hypothesis that a robot can learn to interact purposefully with its environment through a developmental acquisition of sensory-motor coordination. Teleoperation can bootstrap the process by enabling the robot to observe its own sensory responses to actions that lead to specific outcomes within an environment. It is shown that a reach-and-grasp task, learned by an articulated robot through a small number of teleoperated trials, can be performed autonomously with success in the face of significant variations in the environment and perturbations of the goal. In particular, teleoperation of the robot to reach and grasp an object at nine different locations in its workspace enabled robust autonomous performance of the task anywhere within the workspace. Superpositioning was performed using the Verbs and Adverbs algorithm that was developed originally for the graphical animation of articulated characters. The work was performed on Robonaut, the NASA space-capable humanoid at Johnson Space Center, Houston, TX.

A mobile version of the NASA/DARPA Robonaut humanoid recently completed initial autonomy trials working directly with humans in cluttered environments. This compact robot combines the upper body of the Robonaut system with a Segway ™ Robotic Mobility Platform yielding a dexterous, maneuverable humanoid ideal for interacting with human co-workers in a range of environments. This system uses stereovision to locate human teammates and tools and a navigation system that uses laser range and vision data to follow humans while avoiding obstacles. Tactile sensors provide information to grasping algorithms for efficient tool exchanges. The autonomous architecture utilizes these pre-programmed skills to form complex behaviors. The initial behavior demonstrates a robust capability to assist a human by acquiring a tool from a remotely located individual and then following the human in a cluttered environment with the tool for future use.

Pose estimation is a fundamental problem in machine vision. Silhouette and 3D point matching are two of the many popular methods for attacking this problem. The appearance of a silhouette is highly sensitive to changes in the object's planar (WRT the object plane) location and orientation. Therefore, silhouette matching can be used to accurately measure 3 of the 6 pose parameters. The appearance of a silhouette is less sensitive to DOFs that produce out-of-plane motion and so it enables only rough measurement of these 3 pose parameters. 3D point matching, which employs range data, can be used to accurately determine the 3 out-of-plane pose parameters. However, to recognize specific 3d points, one must typically make strong assumptions about the types of features present on an object's surface. Our goal is to solve the more general pose problem, where specific types of features cannot be relied upon because they might not be present. This paper first introduces a novel approach to silhouette matching which employs binary range maps and statistically generated templates. It then describes a hybrid method for template-based pose estimation that leverages silhouette matching for 3 of 6 pose parameters. The remaining 3 parameters are determined using range measurements that do not require the presence of specific features or artifacts. This approach provides a high degree of precision in all 6 DOFs, yet its computational efficiency enables real-time performance. Results from dexterous robot (Robonaut) experiments, using this pose algorithm, are discussed.

This paper addresses the problem of automatic skill acquisition by a robot. It reports that six trials of a reach-grasp-release-retract skill are sufficient for learning a canonical description of the task under the following circumstances: The robot is Robonaut, NASA's space-capable, dexterous humanoid. Robonaut was teleoperated by a person using full immersion Virtual Reality technology that transforms the operator's arm and hand motions into those of the robot. The operator's sole source of real-time feedback was visual. During the six trials all of the Robot's sensory inputs and motor control parameters were recorded as time-series. Later the time-series from each trial was partitioned into the same number of episodes as a function of changes in the motor parameter sequence. The episodes were time normalized and averaged across trials The resultant motor parameter sequence and sensor signals were used to control the robot without the teleoperator. The robot was able to perform the task autonomously with robot starting positions and object locations both similar to, and different from the original trials.

Ego-motion (self-motion) of the camera is a considerable problem in outdoor rover applications. Stereo tracking of a single moving target is a difficult problem that becomes even more challenging when rough terrain causes significant and high-acceleration motion of the camera in the world. The paper discusses the use of inertial measurements to estimate camera ego-motion and the use of these estimates to augment stereo tracking. Some initial results from an outdoor rover are presented, illustrating the efficacy of the method. The method causes fast but predictable image location transients, but reduces the amplitude of image location transients due to the rough terrain.

To interact effectively with humans, mobile robots will need certain skills. One particularly important skill is the ability to pursue moving agents. To do this, the robot needs a robust visual tracking algorithm and an effective obstacle avoidance algorithm, plus a means of integrating these two behaviors in a seamless manner. In this paper, we introduce the proximity space method as a means for performing real-time, behavior-based control of visual gaze. We then show how this method is integrated with robot motion using an intelligent control architecture that can automatically reconfigure the robot's behaviors in response to environmental changes. The resulting implementation pursues people and other robots around our laboratory for extended periods of time.