The sensory ego-sphere (SES) is a short-term memory for a robot in the form of an egocentric, tessellated, spherical, sensory-motor map of the robot's locale. This paper reports on: (1) the mapping to the SES of images with much higher resolution than the SES itself, and (2) the processing of visual attention on the SES. Described is a procedure to store spatially-overlapping imagery in the SES database and to generate and update a spherical composite of its visual contents. The composite image serves both as a visual map of the locale and as a representation of the local contents of the underlying full-resolution imagery. Visual attention enables fast alignment of overlapping images without warping or position optimization, since an attentional point (AP) on the composite typically corresponds to one on each of the collocated regions in the images. Such alignment speeds analysis of the multiple images of the area. Compositing and attention were performed two ways and compared: (1) APs were computed directly on the composite and not on the full resolution images until the time of retrieval. (2) The attentional operator was applied to all incoming imagery. Then the locations and values of the APs on the composite were computed as a function of those points within the corresponding regions of the collocated images. It was found that although the second method was slower, it produced consistent and, thereby, more useful APs

This paper reports that the superposition of a small set of behaviors, learned via teleoperation, can lead to robust completion of an articulated reach-and-grasp task. The results support the hypothesis that a robot can learn to interact purposefully with its environment through a developmental acquisition of sensory-motor coordination. Teleoperation can bootstrap the process by enabling the robot to observe its own sensory responses to actions that lead to specific outcomes within an environment. It is shown that a reach-and-grasp task, learned by an articulated robot through a small number of teleoperated trials, can be performed autonomously with success in the face of significant variations in the environment and perturbations of the goal. In particular, teleoperation of the robot to reach and grasp an object at nine different locations in its workspace enabled robust autonomous performance of the task anywhere within the workspace. Superpositioning was performed using the Verbs and Adverbs algorithm that was developed originally for the graphical animation of articulated characters. The work was performed on Robonaut, the NASA space-capable humanoid at Johnson Space Center, Houston, TX.

It has been demonstrated that a robot can learn to interact purposefully with its environment through a developmental acquisition of sensory-motor coordination. Teleoperation can bootstrap the process by enabling the robot to observe its own sensory responses to actions that lead to specific outcomes within an environment. This paper reports the results of learning to navigate an indoor environment by a simple mobile platform using only LIDAR and odometry. Descriptors of sensory-motor coordination in the form of LIDAR scan images coupled to concurrent motor activity are extracted from five teleoperated trials of driving the robot through the environment. From these a topological action map of the environment is constructed. A hierarchical discrete event dynamical system is defined wherein higher levels encompass increasing time intervals, longer sensory and motor histories, more complex behaviors, and higher degrees of abstraction. An event that cannot be handled at the current level precipitates a jump to the next higher level for processing. The sensory-motor associations learned and encoded in the action-map enable autonomous navigation while avoiding localization errors. Test results are given that demonstrate the robustness of autonomous navigation through the learned environment in the face of both stationary and moving obstacles.

This paper addresses the problem of automatic skill acquisition by a robot. It reports that six trials of a reach-grasp-release-retract skill are sufficient for learning a canonical description of the task under the following circumstances: The robot is Robonaut, NASA's space-capable, dexterous humanoid. Robonaut was teleoperated by a person using full immersion Virtual Reality technology that transforms the operator's arm and hand motions into those of the robot. The operator's sole source of real-time feedback was visual. During the six trials all of the Robot's sensory inputs and motor control parameters were recorded as time-series. Later the time-series from each trial was partitioned into the same number of episodes as a function of changes in the motor parameter sequence. The episodes were time normalized and averaged across trials The resultant motor parameter sequence and sensor signals were used to control the robot without the teleoperator. The robot was able to perform the task autonomously with robot starting positions and object locations both similar to, and different from the original trials.

This paper presents a method to extract salient visual features from imagery-motor sequences for mobile robot navigation based on the sensory-motor coordination principle. Salient feature extraction consists of three steps: tele-operation, offline association, and evaluation. First, the mobile robot is tele-operated in an environment along a path several times. All visual sensor data, along with non-visual sensor data and motor drive commands, are recorded. In the offline association step, these recorded sensory-motor sequences are partitioned into episodes according to the motor commands. Attention operator, texture energy, and average linkage clustering are used to discover the consistent visual features in the imagery sequences. Finally, these features, along with consistent non-visual features, are then used to drive the robot in the learned environment.

Shows how a Sensory EgoSphere (SES), a robot-centric geodesic dome that represents the short term memory of a mobile robot, could enhance a human-robot interface. It is proposed that the addition of this visual representation of the sensor data on a mobile robot enhances the effectiveness of a human-robot interface. The SES migrates information presentation to the user from the sensing level to the perception level. The composition of the vision with other sensors on the SES surrounding the robot gives clarity and ease of interpretation. It enables the user to better visualize the present circumstances of the robot. The human-robot interface (HRI) will be implemented through a graphical user interface (GUI) which contains the SES, command prompt, compass, environment map, sonar and laser display. This paper proposes that the SES increases situational awareness and allows the human supervisor to accurately ascertain the present perception (sensory input) of the robot and use this information to assist the robot in getting out of difficult situations.

This paper describes the definition and testing of a new type of median filter for images. The topological median filter implements some existing ideas and some new ideas on fuzzy connectedness to improve, over a conventional median filter, the extraction of edges in noise. The concept of alpha-connectivity is defined and used to create an algorithm for computing the degree of connectedness of a pixel to all the other pixels in an arbitrary neighborhood. The resulting connectivity map of the neighborhood effectively disconnects peaks in the neighborhood that are separated from the center pixel by a valley in the brightness topology. The median of the connectivity map is an estimate of the median of the peak or plateau to which the center pixel belongs. Unlike the conventional median filter, the topological median is relatively unaffected by disconnected features in the neighborhood of the center pixel. Four topological median filters are defined. Qualitative and statistical analyses of the four filters are presented. It is demonstrated that edge detection can be more accurate on topologically median filtered images than on conventionally median filtered images.

This paper proposes a method for the determination of sensory-motor coordination (SMC) parameters through the teleoperation of a humanoid robot designed for human-robot interaction. It is argued that SMC in a complex environment must be acquired rather than programmed. It is demonstrated that the acquisition of SMC parameters through teleoperation during a task can enable a robot to determine the category of the outcome of the task during autonomous operation. The outcome can be determined without the a priori programming by a person of sensory cues. That is, the robot learns its own sensory cues.

The paper proposes a method for the learning of Sensory-Motor Coordination (SMC) through the teleoperation of a humanoid robot designed for human-robot interaction. It is argued that SMC in a complex environment must be learned rather than programmed. Schema theory is reviewed as a tool for the description of animal behavior at the level of functional modules and higher. SMC is shown to be necessary for the formation of schema assemblages for the control of behavior. Examples are given of four behavior based robot control architectures that implicitly use SMC-driven schema control. It is shown that while the robots are capable of learning, they all rely, to a certain extent, on ad hoc choices of SMC by the designers. A description of the humanoid robot and the sensors it uses for reaching and grasping is given. The proposed method of learning via teleoperation is described. Sensory data acquired through grasping is presented.

Evolutionary robotics is an exciting new area of research with the potential to provide ways to build robots that are beyond our current design abilities. Instead of building robots from the ground up, as with other approaches, robot controllers are evolved using algorithms inspired by biological evolution. The purpose of this work is to develop a tool for the evolution of mobile robot controllers. The controllers are evolved in a realistic simulation. The evolutionary algorithms are provided by the GALib library. Testing demonstrates that the system is capable of producing useful robot controllers. Users can configure the system so that a variety of controller types can be evolved. With a system such as this, users can conduct a range of experiments without having to create custom software.

The paper reports on an experimental comparison of two visual object recognition methods: a radial basis function network (RBFN) which is an artificial neural network, and a synthetic discriminant function network (SDFN) which classifies objects statistically via analysis with optimal spatial filters. Both methods require training with a set of images representative of the objects to be recognized. A comparative performance analysis was performed after training both networks with the same image sets. The algorithms were implemented on a Pentium-class PC under MS Windows NT 4.0. Training images were captured from a color CCD camera with standard NTSC resolution. Experiments were performed on both methods to determine the number of images per object necessary to train the networks, to estimate the two networks' accuracy of recognition, and to characterize their tolerance to image noise. It was found that when presented with a new image of one of the objects, RBFNs are more accurate at recognition than SDFNs. However, SDFNs are slightly more accurate in the presence of additive noise. Under the conditions of the experiments, RBFNs were found to provide an overall minimum classification accuracy of close to ninety percent.

An active vision system enables the selective capture of important information for various applications. With an active vision system, the control of gaze direction toward a visual target can simplify many of the robot's interactions with the world. This work presents the implementation of a camera head control system for a mobile robot, called HelpMate. Helpmate's binocular head is composed of two Sony color cameras mounted on two independent pan-tilt heads. The camera head control behavior is designed to be similar to that of human eyes, which include saccade, smooth pursuit, vergence and vestibulo-ocular reflex. Saccade allows the robot to quickly direct its gaze to the target, smooth pursuit keeps both cameras fixated on the moving target, vergence makes sure that the two cameras are looking at the same object, and vestibulo-ocular reflex compensates the camera motion for the robot body's movement. Control methods have been designed and tested through experimentation. By using color segmentation, HelpMate can detect and locate a human holding a specific colored object, who is approaching within its range. Once the target is located, the two cameras are fixated, and HelpMate's body turns toward the target. The camera head then smoothly tracks the moving target and guides the robot to follow the target as it moves.

Due to a continuous increase in the need to provide more services to health care through robotic aid systems developed for the disabled, the need for addressing human-robot interaction (HRI) has gained the utmost importance. HRI is based on a multi-agent architecture, the Intelligent Machine Architecture (IMA) (R.A. Peters II et al., 1999; W.A. Alford et al., 1999), a new approach for software design for intelligent machines that are principally limited by difficulty in integrating existing algorithms, models, and subsystems. A system of interactive software agents, designed within IMA, encapsulates the various hardware elements, environmental elements, behaviors and tasks. IMA suggests "a society of agents" (M. Minsky, 1985) paradigm to create complex high-level functionality from any sublevels of components. The high-level agents currently employ finite state machines to sequence the activation and suppression of behavior agents. A state machine is realized by two components: state machine engine and state machine representation. Model Integrated Computing (MIC) (G. Nordstrom et al., 1998) is used to model the behavior of the IMA agents. The paper presents a toolkit that designs the agent's behavior through a state-transition metamodeling environment and builds the agent with the help of an existing Agent Builder application.

For a humanoid robot to interact easily with a person, the robot should have human-like sensory capabilities and attentional mechanisms. Particularly useful is an active vision head controlled by a visual attention system that selects viewpoints in the environment as a function of the robot's task. This paper describes a model of human visual attention called FeatureGate, which is a locally connected, pyramidal, artificial neural network that operates on 2D feature maps of the environment. Given a set of feature maps, and the description of a specific target, FeatureGate finds the location whose features most closely match those of the target. The paper describes the network, its implementation, a series of tests that characterize its performance with respect to a person's performance on a similar task, and its use in the control of an active vision system.

Morphological openings and closings are useful for the smoothing of gray-scale images. However, their use for image noise reduction is limited by their tendency to remove important, thin features from an image along with the noise. The paper presents a description and analysis of a new morphological image cleaning algorithm (MIC) that preserves thin features while removing noise. MIC is useful for gray-scale images corrupted by dense, low-amplitude, random, or patterned noise. Such noise is typical of scanned or still-video images. MIC differs from previous morphological noise filters in that it manipulates residual images-the differences between the original image and morphologically smoothed versions. It calculates residuals on a number of different scales via a morphological size distribution. It discards regions in the various residuals that it judges to contain noise. MIC creates a cleaned image by recombining the processed residual images with a smoothed version. The paper describes the MIC algorithm in detail, discusses the effects of parametric variations, presents the results of a noise analysis and shows a number of examples of its use, including the removal of scanner noise. It also demonstrates that MIC significantly improves the JPEG compression of a gray-scale image.

In this paper a system that successfully detects stationary objects in indoor scenes is presented. The doors and ceiling lamps have been chosen to be detected in the initial phase of the authors' system. The scene was divided into four different regions, The edges resulting from the walls/ceiling and the walls/floor intersections are the boundaries that separate the different regions. The use of the log-polar mapping transforms the problem of finding four edges of unknown orientations, that separated the different regions, into finding four horizontal edges. The log-polar map was performed such that it possesses the varying resolution property. The objects were detected using model-matching in the log-polar domain. The varying resolution property of the mapping simplified the model-matching problem. The detection (of edges and then objects) was very fast due to the small size of the log-polar image (64/spl times/64) pixels.>