Reservoir computing (RC) is a powerful framework for learning and generating time-series data, including musical rhythmic patterns. However, its ability to capture and reproduce long-period rhythmic patterns remains limited. In this study, we address this limitation by leveraging slow feature analysis (SFA) to extract a slowly varying oscillatory feature that captures the characteristics of long-period patterns, specifically the periods of the target rhythms. We propose a novel approach that integrates the slow feature oscillation as an input to the RC model, thereby enhancing the learning and generation of long-period rhythmic patterns. The effectiveness of the proposed method is demonstrated through tasks involving synthetic time-series data containing simple long-period rhythmic patterns. The proposed method outperforms conventional RC in terms of rhythm reproduction and its fidelity. Furthermore, an experiment involving a human hi-hat drumming performance shows that the proposed method significantly improves the long-period rhythmic characteristics in the generated time-series data. These findings highlight the potential of combining SFA with RC to advance the modeling and generation of complex temporal patterns.

Reservoir computing, a machine learning framework used for modeling the brain, can predict temporal data with little observations and minimal computational resources. However, it is difficult to accurately reproduce the long-term target time series because the reservoir system becomes unstable. This predictive capability is required for a wide variety of time-series processing, including predictions of motor timing and chaotic dynamical systems. This study proposes oscillation-driven reservoir computing (ODRC) with feedback, where oscillatory signals are fed into a reservoir network to stabilize the network activity and induce complex reservoir dynamics. The ODRC can reproduce long-term target time series more accurately than conventional reservoir computing methods in a motor timing and chaotic time-series prediction tasks. Furthermore, it generates a time series similar to the target in the unexperienced period, that is, it can learn the abstract generative rules from limited observations. Given these significant improvements made by the simple and computationally inexpensive implementation, the ODRC would serve as a practical model of various time series data. Moreover, we will discuss biological implications of the ODRC, considering it as a model of neural oscillations and their cerebellar processors.

A critical aspect in robot technology lies in ensuring that the robot arm executes the intended task or movement. The conventional teaching–playback technique for programming robot arm movements generates excessive torque when the initial posture differs from that during motion teaching or when external disturbances disrupt robotic arm execution. To address this problem, this study proposed a motion generation system with echo state networks (ESNs) for robot arms. These networks can reproduce trajectories including features such as attractors and bifurcations by training time-series data originating from a dynamical system. By leveraging this capability, the proposed approach enables an ESN to infer a dynamical system underlying demonstrated motions to perform a specific task on the basis of state feedback. Subsequently, the trained ESN generates reference signals to replicate the learned motion based on the current robot state. These reference signals can be used for linear feedback at the joint level. Redundant robots can easily transition to unobserved states because of their degrees of freedom. However, the trained ESN exhibits remarkable generalization capabilities, effectively guiding the robot back to its intended motion in those unobserved states. Numerical simulations confirmed that the proposed system dynamically adapts to robot behavior, effectively mitigating excessive acceleration, even in scenarios with initial posture deviations or external disturbances, thereby outperforming conventional teaching–playback methods.

Reservoir computing provides an attractive alternative for time series data representation due to their gradient-free learning and energy efficient operation. In a ‘reservoir computer’ (RC), the reservoir is a random recurrent neural network that forms a high dimensional non-linear dynamical system which can be mapped to a desired sequence or dynamical system through linear regression. Although, the non-linearity of the reservoir gives its power, there is no guarantee that it is the most suitable for the task at hand. To this end, in this study, we propose to amplify the effectiveness of reservoirs by augmenting them with higher order terms computed based on reservoir unit activations. We name this model as Higher-order-augmented Reservoir Computer (ha-RC). We test the efficacy of ha-RCs by using two types of tasks: time series representation and long-term prediction, i.e. learning a dynamical system. Our experiments show that with the proposed model, a given learning accuracy can be achieved with significantly less memory resource compared to the baseline of standard Reservoir Computer (s-RC). This becomes possible as ha-RC can handle complex learning problems with much smaller reservoirs compared to s-RC models. This memory efficiency directly translates to energy efficiency as less number of arithmetic operations are needed with ha-RCs to reach the same level of accuracy compared to s-RCs. These points make our proposed model ideal for hardware implementation and edge computing.

Adaptive kinematic/dynamic control of a robot arm (manipulator) is important for the robust execution of various tasks, including target tracking, when changes in the environment and robot’s physical parameters, such as mass and friction, occur. For highly accurate model-free adaptive tracking control, we propose an error-correction model using the reservoir of basal dynamics (reBASICS) computing method. reBASICS is trained to minimize multimodal errors, such as those in the end-effector position (visually detected) and torques (proprioceptively detected), and to correct robot trajectories. If errors are corrected simultaneously, the corrections can interfere with each other. Therefore, the model initially corrects torques to stabilize robot movements, followed by position correction. In the simulation of a two-link robot arm, approximate inverse kinematics (IK) and proportional-derivative (PD) controllers were assumed to produce tracking errors. The results showed that reducing errors based on reBASICS produced smaller tracking errors from the reference trajectory compared to that using a conventional echo state network (ESN). Furthermore, reducing both position and torque errors resulted in better performance than reducing position-only and torque-only errors.

Learning from demonstration (LfD) with behavior cloning is attractive for its simplicity; however, compounding errors in long and complex skills can be a hindrance. Considering a target skill as a sequence of motor primitives is helpful in this respect. Then the requirement that a motor primitive ends in a state that allows the successful execution of the subsequent primitive must be met. In this study, we focus on this problem by proposing to learn an explicit correction policy when the expected transition state between primitives is not achieved. The correction policy is learned via behavior cloning by the use of Conditional Neural Motor Primitives (CNMPs) that can generate correction trajectories in a context-dependent way. The advantage of the proposed system over learning the complete task as a single action is shown with a table-top setup in simulation, where an object has to be pushed through a corridor in two steps. Then, the applicability of the proposed method to bi-manual knotting in the real world is shown by equipping an upper-body humanoid robot with the skill of making knots over a bar in 3D space.

Reservoir Computing, in particular Echo State Networks (ESNs) offer a lightweight solution for time series representation and prediction. An ESN is based on a discrete time random dynamical system that is used to output a desired time series with the application of a learned linear readout weight vector. The simplicity of the learning suggests that an ESN can be used as a lightweight alternative for movement primitive representation in robotics. In this study, we explore this possibility and develop Context-based Echo State Networks (CESNs), and demonstrate their applicability to robot movement generation. The CESNs are designed for generating joint or Cartesian trajectories based on a user definable context input. The context modulates the dynamics represented by the ESN involved. The linear read-out weights then can pick up the context-dependent dynamics for generating different movement patterns for different contexts. To achieve robust movement execution and generalization over unseen contexts, we introduce a novel data augmentation mechanism for ESN training. We show the effectiveness of our approach in a learning from demonstration setting. To be concrete, we teach the robot reaching and obstacle avoidance tasks in simulation and in real-world, which shows that the developed system, CESN provides a lightweight movement primitive representation system that facilitate robust task execution with generalization ability for unseen seen contexts, including extrapolated ones.

Generation of complex patterns at a specific timing is crucial to most forms of learning and behavior, which are acquired through dopamine-modulated plasticity in the striatum. However, the neural mechanisms of such reward-based spatiotemporal processing remain unknown. Inspired by the cortico-striatal circuits, this study developed a new reservoir computing method, a class of recurrent neural networks, based on reward-modulated Hebbian learning (RMHL) for the spatiotemporal motor learning. We utilized a reservoir of basal dynamics (reBASICS), which generated self-sustained limit cycle oscillations with various frequencies, as a reservoir structure. Then, the oscillations were linearly integrated as readout output, in which readout weights were modulated with RMHL. The simulations showed that reBASICS-based RMHL was able to accomplish both motor timing and pattern drawing tasks, for which existing reservoir-based RMHL failed. Further, introducing an eligibility trace mechanism into RMHL allowed the model to learn motor timing even when reward-based modulation was delayed. In conclusion, this model is proposed as a new computational model of temporal processing of the striatum, where the cortical areas generate stable oscillations. From the oscillatory dynamics, spatiotemporal patterns are learned using RMHL in the striatum.

The ability of the brain to generate complex spatiotemporal patterns with specific timings is essential for motor learning and temporal processing. An approach that can model this function, using the spontaneous activity of a random neural network (RNN), is associated with orbital instability. We propose a simple system that learns an arbitrary time series as the linear sum of stable trajectories produced by several small network modules. New finding in computer experiments is that the trajectories of the module outputs are orthogonal to each other. They created a dynamic orthogonal basis acquiring a high representational capacity, which enabled the system to learn the timing of extremely long intervals, such as tens of seconds for a millisecond computation unit, and also the complex time series of Lorenz attractors. This self-sustained system satisfies the stability and orthogonality requirements and thus provides a new neurocomputing framework and perspective for the neural mechanisms of motor learning.

In this study, we extend our robot trust model into a multimodal setting in which the Nao robot leverages audio-visual data to perform a sequential multimodal pattern recalling task while interacting with a human partner who has different guiding strategies: reliable, unreliable, and random. Here, the humanoid robot is equipped with a multimodal auto-associative memory module to process audio-visual patterns to extract cognitive load (i.e., computational cost) and an internal reward module to perform cost-guided reinforcement learning. After interactive experiments, the robot associates a low cognitive load (i.e., high cumulative reward) yielded during the interaction with high trustworthiness of the guiding strategy of the partner. At the end of the experiment, we provide a free choice to the robot to select a trustworthy instructor. We show that the robot forms trust in a reliable partner. In the second setting of the same experiment, we endow the robot with an additional simple theory of mind module to assess the efficacy of the instructor in helping the robot perform the task. Our results show that the performance of the robot is improved when the robot bases its action decisions on factoring in the instructor assessment.

Learning to interact with the environment not only empowers the agent with manipulation capability but also generates information to facilitate building of action understanding and imitation capabilities. This seems to be a strategy adopted by biological systems, in particular primates, as evidenced by the existence of mirror neurons that seem to be involved in multi-modal action understanding. How to benefit from the interaction experience of the robots to enable understanding actions and goals of other agents is still a challenging question. In this study, we propose a novel method, deep modality blending networks (DMBN), that creates a common latent space from multi-modal experience of a robot by blending multi-modal signals with a stochastic weighting mechanism. We show for the first time that deep learning, when combined with a novel modality blending scheme, can facilitate action recognition and produce structures to sustain anatomical and effect-based imitation capabilities. Our proposed system, which is based on conditional neural processes, can be conditioned on any desired sensory/motor value at any time step, and can generate a complete multi-modal trajectory consistent with the desired conditioning in one-shot by querying the network for all the sampled time points in parallel avoiding the accumulation of prediction errors. Based on simulation experiments with an arm-gripper robot and an RGB camera, we showed that DMBN could make accurate predictions about any missing modality (camera or joint angles) given the available ones outperforming recent multimodal variational autoencoder models in terms of long-horizon high-dimensional trajectory predictions. We further showed that given desired images from different perspectives, i.e. images generated by the observation of other robots placed on different sides of the table, our system could generate image and joint angle sequences that correspond to either anatomical or effect-based imitation behavior. To achieve this mirror-like behavior, our system does not perform a pixel-based template matching but rather benefits from and relies on the common latent space constructed by using both joint and image modalities, as shown by additional experiments. Moreover, we showed that mirror learning (in our system) does not only depend on visual experience and cannot be achieved without proprioceptive experience. Our experiments showed that out of ten training scenarios with different initial configurations, the proposed DMBN model could achieve mirror learning in all of the cases where the model that only uses visual information failed in half of them. Overall, the proposed DMBN architecture not only serves as a computational model for sustaining mirror neuron-like capabilities, but also stands as a powerful machine learning architecture for high-dimensional multi-modal temporal data with robust retrieval capabilities operating with partial information in one or multiple modalities.

Forming trust in a biological or artificial interaction partner that provides reliable strategies and employing the learned strategies to scaffold another agent are critical problems that are often addressed separately in human-robot and robot-robot interaction studies. In this paper, we provide a unified approach to address these issues in robot-robot interaction settings. To be concrete, we present a trust-based affective computational account of scaffolding while performing a sequential visual recalling task. In that, we endow the Pepper humanoid robot with cognitive modules of auto-associative memory and internal reward generation to implement the trust model. The former module is an instance of a cognitive function with an associated neural cost determining the cognitive load of performing visual memory recall. The latter module uses this cost to generate an internal reward signal to facilitate neural cost-based reinforcement learning (RL) in an interactive scenario involving online instructors with different guiding strategies: reliable, less-reliable, and random. These cognitive modules allow the Pepper robot to assess the instructors based on the average cumulative reward it can collect and choose the instructor that helps reduce its cognitive load most as the trustworthy one. After determining the trustworthy instructor, the Pepper robot is recruited to be a caregiver robot to guide a perceptually limited infant robot (i.e., the Nao robot) that performs the same task. In this setting, we equip the Pepper robot with a simple theory of mind module that learns the state-action-reward associations by observing the infant robot’s behavior and guides the learning of the infant robot, similar to when it went through the online agent-robot interactions. The experiment results on this robot-robot interaction scenario indicate that the Pepper robot as a caregiver leverages the decision-making policies – obtained by interacting with the trustworthy instructor– to guide the infant robot to perform the same task efficiently. Overall, this study suggests how robotic-trust can be grounded in human-robot or robot-robot interactions based on cognitive load, and be used as a mechanism to choose the right scaffolding agent for effective knowledge transfer.

It is unclear how a human attributes the cause of failure to the robot in a human-robot interaction. We aim to identify the relationship between causal attribution and mind perception in a repeated game with an agent. We investigated causal attribution of the participant to the agent: which decision of the participant or the partner agent caused the unexpectedly small amount of the reward. We conducted experiments with three agent conditions: a human, robot, and computer. The results showed that the agency score negatively correlated with the degree of causal attribution to the partner agent. In particular, correlations of scores of “thought,” “memory,” “planning,” and “self-control” that are sub-items of agency were significant. This implied the impression that “the agent acted to succeed” might reduce causal attribution. In addition, we found that decrease in the scores of mind perception correlated with the degree of causal attribution to the partner agent. This suggests that a sense of betrayal of the prior expectation by the partner agent through the game might lead to causal attribution to the partner agent.

Natural policy gradient (NPG) methods are promising approaches to finding locally optimal policy parameters. The NPG approach works well in optimizing complex policies with high-dimensional parameters, and the effectiveness of NPG methods has been demonstrated in many fields. However, the incremental estimation of the NPG is computationally unstable owing to its high sensitivity to the step-sizes values, especially to the one used to update the estimate of NPG. In this study, we propose a new incremental and stable algorithm for the NPG estimation. We call the proposed algorithm the implicit incremental natural actor critic (I2NAC), and it is based on the idea of the implicit update. The convergence analysis for I2NAC is provided. Theoretical analysis results indicate the stability of I2NAC and the instability of conventional incremental NPG methods. Numerical experiments were performed, and the results show that I2NAC is less sensitive to the values of the meta-parameters, including the step-size for the NPG update, compared to the existing incremental NPG method.

Humans use various types of modalities to express own internal states. If a robot interacting with humans can pay attention to limited signals, it should select more informative ones to estimate the partners' states. We propose an active perception method that controls the robot's attention based on an energy minimization criterion. An energy-based model, which has learned to estimate the latent state from sensory signals, calculates energy values corresponding to occurrence probabilities of the signals; The lower the energy is, the higher the likelihood of them. Our method therefore selects the modality that provides the lowest expectation energy among available ones to exploit more frequent experiences. We employed a multimodal deep belief network to represent relationships between humans' states and expressions. Our method demonstrated better performance for the modality selection than other methods in a task of emotion estimation. We discuss the potential of our method to advance human-robot interaction.

Personality impressions of robots have been regarded as one of the crucial factors in human-robot interaction. To design the personality impressions, we should know how the visual, auditory, and tactile impressions determine the personality impressions. In this study, we investigated the relationships between touch sensations and personality impressions with a child-type android robot in two conditions where 40 Japanese participants touched a part of the robot with different appearance of the face. Factor and path analyses were conducted on the evaluation scores of the sensations and impressions provided by the participants. As a result, two significant positive causal relationships (p < 0.001) were found between the Preference and Resilience touch sensations, and the Likability and Capability personality impressions, respectively, in both robot conditions. On the other hand, several other causal relationships were found only in one condition. This suggests that there are appearance-dependent and appearance-independent relationships between touch sensations and personality impressions.

Collaborative robots are quickly gaining momentum in real-world settings. This has motivated many new research questions in human-robot collaboration. In this paper, we address the questions of whether and when a robot should take initiative during joint human-robot task execution. We develop a system capable of autonomously tracking and performing table-top object manipulation tasks with humans and we implement three different initiative models to trigger robot actions. Human-initiated help gives control of robot action timing to the user; robot-initiated reactive help triggers robot assistance when it detects that the user needs help; and robot-initiated proactive help makes the robot help whenever it can. We performed a user study (N=18) to compare these trigger mechanisms in terms of task performance, usage characteristics, and subjective preference. We found that people collaborate best with a proactive robot, yielding better team fluency and high subjective ratings. However, they prefer having control of when the robot should help, rather than working with a reactive robot that only helps when it is needed.

The characteristics of the spoken language used to address infants have been eagerly studied as a part of the language acquisition research. Because of the uncontrollability factor with regard to the infants, the features and roles of infant- directed speech were tried to be revealed by the comparison of speech directed toward infants and that toward other listeners. However, they share few characteristics with infants, while infants have many characteristics which may derive the features of IDS. In this study, to solve this problem, we will introduce a new approach that replaces the infant with an infant-like robot which is designed to control its motions and to imitate its appearance very similar to a real infant. We have now recorded both infant- and infant- like robot-directed speech and are constructing both corpora. Analysis of these corpora is expected to contribute to the studies of infant-directed speech. In this paper, we discuss the contents of this approach and the outline of the corpora.

Share on How does emphatic emotion emerge via human-robot rhythmic interaction? Authors: Hideyuki Takahashi Osaka University, Osaka, Japan Osaka University, Osaka, JapanView Profile , Nobutsuna Endo Osaka University, Osaka, Japan Osaka University, Osaka, JapanView Profile , Hiroki Yokoyama Osaka University, Osaka, Japan Osaka University, Osaka, JapanView Profile , Takato Horii Osaka University, Osaka, Japan Osaka University, Osaka, JapanView Profile , Tomoyo Morita Osaka University, Osaka, Japan Osaka University, Osaka, JapanView Profile , Minoru Asada Osaka University, Osaka, Japan Osaka University, Osaka, JapanView Profile Authors Info & Claims HAI '14: Proceedings of the second international conference on Human-agent interactionOctober 2014 Pages 273–276https://doi.org/10.1145/2658861.2658940Online:29 October 2014Publication History 2citation139DownloadsMetricsTotal Citations2Total Downloads139Last 12 Months7Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

It has been suggested that perceptual immaturity in early infancy enhances learning for various cognitive functions. This paper demonstrates the role of visual development triggered by self-organization in a learner's visual space in a case of the mirror neuron system (MNS). A robot learns a function of the MNS by associating self-induced motor commands with observed motions while the observed motions are gradually self-organized in the visual space. A temporal convergence of the self-organization triggers visual development, which improves spatiotemporal blur filters for the robot's vision and thus further advances self-organization in the visual space. Experimental results show that the self-triggered development enables the robot to adaptively change the speed of the development (i.e., slower in the early stage and faster in the later stage) and thus to acquire clearer correspondence between self and other (i.e., the MNS).

In order to develop skills, actions, behavior in a human symbiotic environment, a robot is going to learn something from behavior observation of predecessors or humans. Recently, robotic imitation methods based on many approaches have been proposed. We have proposed reinforcement learning based approaches for the imitation and investigated them under an assumption that the observer recognizes the body parts of the performer and maps them to the ones of its own. However, the assumption is not always applicable because the body of the performer is usually different from the observing robot. In order to learn various behaviors from the observation, the robot has to cluster the observed body area of the performer on the image and maps the clustered parts to its own body parts based on reasonable criterion for itself and feedback the data for the imitation. This paper shows that the clustering the body area on the camera image into the body parts of its own based on the estimation of the state value in the framework of reinforcement learning as well as it imitates the observed behavior based on the state value estimation. The clustering parameters are updated based on the temporal difference error, in an analogous way such that the parameters of the state value of the behavior are updated based on the temporal difference error. The validity of the proposed method is investigated by applying it to a imitation of a dynamic throwing motion of an inverted pendulum robot and human.

Robots are becoming part of people's everyday social lives - and will increasingly become so. In future years, robots may become caretaking assistants for the elderly, or academic tutors for our children, or medical assistants, day care assistants, or psychological counselors. Robots may become our co-workers in factories and offices, or maids in our homes. They may become our friends. As we move to create our future with robots, hard problems in HRI exist, both technically and socially. The Fifth Annual Conference on HRI seeks to take up grand technical and social challenges in the field - and speak to their integration. This panel brings together 4 leading experts in the field of HRI to speak on this topic.

Estimation of a caregiver's view is one of the most important capabilities for a child to understand the behavior demonstrated by the caregiver, that is, to infer the intention of behavior and/or to learn the observed behavior efficiently. We hypothesize that the child develops this ability in the same way as behavior learning motivated by an intrinsic reward, that is, he/she updates the model of the estimated view of his/her own during the behavior imitated from the observation of the behavior demonstrated by the caregiver based on minimizing the estimation error of the reward during the behavior. From this view, this paper shows a method for acquiring such a capability based on a value system from which values can be obtained by reinforcement learning. The parameters of the view estimation are updated based on the temporal difference error (hereafter TD error: estimation error of the state value), analogous to the way such that the parameters of the state value of the behavior are updated based on the TD error. Experiments with simple humanoid robots show the validity of the method, and the developmental process parallel to young children's estimation of its own view during the imitation of the observed behavior demonstrated by the caregiver is discussed.

A robot working with humans or other robots is supposed to be adaptive to changes in the environment. Reinforcement learning has been studied well for motor skill learning, robot behavior acquisition and adaptation of the behavior to the environmental changes. However, it is not practical that the robot learns and adapts its behavior only through trial and error by itself from scratch because huge exploration is needed. Fortunately, it is nothing unusual to have predecessors in the environment and it is reasonable to learn something from the observation of predecessors' behavior. In order to learn various behavior from the observation, the robot must segment the behavior based on reasonable criterion for itself and feedback the data to behavior learning by itself. This paper presents a case study for a robot to understand unfamiliar behavior shown by a human instructor through the collaboration between behavior acquisition and recognition of observed behavior, where the state value has an important role not simply for behavior acquisition (reinforcement learning) but also for behavior recognition (observation). The validity of the proposed method is shown by applying it to a dynamic environment where one robot and one human play soccer.

Life-time development of behavior learning seems based on not only self-learning architecture but also explicit/implicit teaching from other agents that is expected to accelerates the learning. This paper presents a method for a robot to understand unfamiliar behaviors shown by others through the collaboration between behavior acquisition and recognition of observed behaviors, where the state value has an important role not simply for behavior acquisition (reinforcement learning) but also for behavior recognition (observation). That is, the state value updates can be accelerated by observation without real trials and errors while the learned values enrich the recognition system since it is based on estimation of the state value of the observed behavior. The validity of the proposed method is shown by applying it to a dynamic environment where two robots play soccer.

A spiral response-cascade hypothesis is proposed to model the mechanism that enables human communication to emerge or be maintained among agents. In this hypothesis, we propose the existence of three cascades each of which indicates intrapersonal or interpersonal mutual facilitation in the formation of someone's feelings about one's communication partners and the exhibition of behaviors in communicating with them, i.e., responding. In this paper, we discuss our examination of an important part of the hypothesis, i.e., what we call an intrapersonal responding cascade, through an experiment where the gaze interactions between a participant and a communication robot were controlled not only by controlling the robot's gaze but also by signaling participants when to shift their gaze. We report that the participants' experiences in responding to the robot enable them to regard the robot as a communicative being, which partially supports the hypothesis of the intrapersonal responding cascade.

In performing various kinds of tasks, body representation is one of the most fundamental issues for physical agents (humans, primates, and robots). Especially during tool-use by Japanese macaque monkeys, neurophysiological evidence shows that the representation can be dynamically reconstructed by spatio-temporal integration of different sensor modalities so that it can be adaptive to environmental changes [1]. However, to construct such a representation, an issue to be solved is how to associate which information among various sensory data. This paper presents a method that constructs cross-modal body representation from vision, touch, and proprioception. When the robot touches something, the activation of tactile sense triggers the construction process of the visual receptive field for body parts that can be found by visual attention based on saliency map and consequently regarded as the end effector. Simultaneously, proprioceptive information is associated with this visual receptive field to construct the cross-modal body representation. The computer simulation results are comparable to the activities of parietal neurons found in the Japanese macaque monkeys. Various conditions are also investigated so that what kind of information is important to generate the same results as findings in neurophysiology.

Both self-learning architecture (embedded structure) and explicit/implicit teaching from other agents (environmental design issue) are necessary not only for one behavior learning but more seriously for life-time behavior learning. This paper presents a method for a robot to understand unfamiliar behavior shown by others through the collaboration between behavior acquisition and recognition of observed behavior, where the state value has an important role not simply for behavior acquisition (reinforcement learning) but also for behavior recognition (observation). That is, the state value updates can be accelerated by observation without real trial and error while the learned values enrich the recognition system since it is based on estimation of the state value of the observed behavior. The validity of the proposed method is shown by applying it to a dynamic environment where two robots play soccer.

We propose a novel approach for acquisition and development of behaviors through observation in multi-agent environment. Observed behaviors of others give fruitful hints for a learner to find a new situation, a new behavior for the situation, necessary information for the behavior acquisition. RoboCup scenario gives us a good test-bed multi-agent environment where a learner can observe behaviors of others during practices or games. It is more realistic, practical, and efficient to take advantages of observation of skilled players than to discover new skills and necessary information only through the interaction of a learner and an environment. The learner automatically detects state variables and a goal of the behavior through the observation based on mutual information. Reinforcement learning method is applied to acquire the discovered behavior suited to the robot. Experiments under RoboCup MSL scenario shows the validity of the proposed method.

In this paper, we propose a control system that changes the compliance based on the walking speed to stabilize biped walking on rough terrain. The proposed system does not use the inclination of the terrain. Instead, the system changes walking modes depends on its walking speed. In the downhill terrain, when the walking speed is increased, the stiffness of the ankle in the support phase is controlled so as to brake the increased speed. In the uphill terrain, when the walking speed is decreased, the stiffness of the waist joint is controlled and the desired trajectory for the supported leg is shifted so as not to falls down backward. To validate the efficiency of the proposed system, the stability of walking with the proposed system is examined in the two dimensional dynamics simulation. It is shown that the robot with the proposed system can walk in the more variable rough terrain and with the broader walking speed than without changing the stiffness of the joints.

Neurophysiology has revealed the existence of mirror neurons in brain of macaque monkeys and they shows similar activities during executing an observation of goal directed movements performed by self and other. The concept of the mirror neurons/systems[1] is very interesting and suggests that behavior acquisition and the inferring intention of other are related to each other. That is, the behavior learning modules might be used not only for behavior acquisition/execution but also for the understanding of the behavior/intention of other. We propose a novel method not only to learn and execute a variety of behaviors but also to understand behavior of others supposing that the observer has already acquired the utilities (state values in reinforcement learning scheme) of all kinds of behaviors the observed agent can do. The method does not need a precise world model or coordination transformation system to deal with view difference caused by different viewpoints. This paper shows that an observer can understand/recognize a behavior of other not by precise object trajectory in allocentric/egocentric coordinate space but by estimated utility transition during the observed behavior.

The existing reinforcement learning approaches have been suffering from the curse of dimension problem when they are applied to multiagent dynamic environments. One of the typical examples is a case of RoboCup competitions since other agents and their behaviors easily cause state and action space explosion. This paper presents a method of modular learning in a multiagent environment by which the learning agent can acquire cooperative behaviors with its team mates and competitive ones against its opponents. The key ideas to resolve the issue are as follows. First, a two-layer hierarchical system with multi learning modules is adopted to reduce the size of the sensor and action spaces. The state space of the top layer consists of the state values from the lower level, and the macro actions are used to reduce the size of the physical action space. Second, the state of the other to what extent it is close to its own goal is estimated by observation and used as a state value in the top layer state space to realize the cooperative/competitive behaviors. The method is applied to 4 (defense team) on 5 (offense team) game task, and the learning agent successfully acquired the teamwork plays (pass and shoot) within much shorter learning time (30 times quicker than the earlier work).

Due to differences in body structure between robots and humans, it is a formidable task for robots to show behaviors that correspond to human behaviors. As a simple case of this correspondence problem, this paper presents a robot that learns to vocalize vowels through interaction with its caregiver. Inspired by the findings in developmental psychology, we focus on the role of maternal imitation (i.e., imitation of a robot voice by a caregiver), which could play a role in guiding the correspondence of sounds. Furthermore, we suppose that it causes unconscious anchoring in which the imitated voice by the caregiver is approaching to one of his/her own vowels without his/her intension, and thereby works for guiding robot's utterances to be more vowel-like. We propose a method for vowel learning with an imitative caregiver under the assumption that the robot knows the desired categories of caregiver's vowels and the rough estimate of mapping between the region of sounds that the caregiver can generate and the region that the robot can generate. Through experiments with a Japanese imitative caregiver, we show that a robot succeeds in acquiring more vowel-like utterances than would be possible without such a caregiver, even when the robot is provided different mapping functions.

Optimization of robot dynamics for a given human motion is an intuitive way to approach the problem of learning complex human behavior by imitation. In this paper, we propose a methodology based on a learning approach that performs optimization of humanoid dynamics in a low-dimensional subspace. We compactly represent the kinematic information of humanoid motion in a low dimensional subspace. Motor commands in the low dimensional subspace are mapped to the expected sensory feedback. We select optimal motor commands based on sensory-motor mapping that also satisfy our kinematic constraints. Finally, we obtain a set of novel postures that result in superior motion dynamics compared to the initial motion. We demonstrate results of the optimized motion on both a dynamics simulator and a real humanoid robot

We present here a concept and realization of a dynamic walker that is stabilized by using a fast and heavy rotor, a gyro. The dynamics of a symmetric, fast rotating gyro is different from that of a non-rotating solid body, e.g. in the case of small disturbances it tends to stabilize the axis. This property is used and tested in a passive dynamic walker. We investigate the stability of a knee-less walker that is stabilized by a gyro in a 3D environment, and present also simulations. As the results of this work we show that the rotor enhances the stability of the walking in the simulations. We also summarize some concepts and simulations of actuated robots with a gyro. Finally we give an overview of experiments and prototypes that are realized so far

Observation study of human infants tells us that they can successfully acquire lexicon; understanding the relationship between the meaning and the uttered word from only one teaching by caregiver, even though there are many other possible mappings. This paper proposes a lexical acquisition model which makes use of curiosity to associate visual features of observed objects with the labels that are uttered by a caregiver. A robot changes its attention and learning rate based on curiosity. In the experiment with a real humanoid robot, the visual features are represented with self organizing maps which adaptively represents the shape of observed objects independent of the viewpoints.

Coevolution has been receiving increased attention as a method for simultaneously developing the control structures of multiple agents. Our ultimate goal is the mutual development of skills through coevolution. The coevolutionary process is, however, often prone to settle into suboptimal strategies. The key to successful coevolution has thus far been unclear. This paper discusses how several robots can emerge cooperative and competitive behavior through coevolutionary processes. In order to realize successful coevolution, we propose two ideas: multiple schedules for incremental evolution and fitness sharing based on the method of importance sampling. To examine this issue, we conducted a series of computer simulations. We have chosen a simplified soccer game consisting of two or three robots as a testbed for analyzing a problem in which both competitive and cooperative tasks are involved. We show that the proposed fitness evaluation allows robots to evolve robust behaviors in cooperative and competitive situations

A robot should have softness and many sensors to manipulate an object dexterously and to adapt various environments. However, many existing schemes where a designer calibrates the sensor output to the world coordinate frame are difficult to adapt for such the robot. This paper proposes a learning mechanism for a robot hand which consists of anthropomorphic fingertips. The sensor for the fingertip is difficult to calibrate because the sensor receptors are embedded randomly in the soft material. The effectiveness of the proposed mechanism is demonstrated by an experiment that the robot picks up an unknown weight object.

Existing reinforcement learning approaches have been suffering from the policy alternation of others in multiagent dynamic environments. A typical example is the case of RoboCup competitions because other agent behaviors may cause sudden changes in state transition probabilities in which constancy is needed for the learning to converge. The keys for simultaneous learning to acquire competitive behaviors in such an environment are: a modular learning system for adaptation to the policy alternation of others; and an introduction of macro actions for simultaneous learning to reduce the search space. This paper presents a method of modular learning in a multiagent environment in which the learning agents can simultaneously learn their behaviors and adapt themselves to the situations as a consequence of the others' behaviors.

Humans are supposed to utilize its joint elasticity to realize smooth and adaptive walking. Although such human-like biped walking is strongly affected by the terrain dynamics, it was not taken into account in robotic bipedalism since it is very difficult to model the dynamics formally. In this paper, instead of modeling the dynamics formally, we propose to estimate the relationship between actuation (air valve opening duration) and sensing (touch sensor information) by real walking trials, and to stabilize walking cycle by utilizing it. Since the terrain dynamics is involved in the relation, we can avoid to model it formally. We conducted walking experiments on various types of terrain to demonstrate the effectiveness of the proposed method.

Developing a humanoid robot that can learn to perform complex tasks by itself has become a major goal of robotics research. This paper proposes a new algorithm for the automatic segmentation of humanoid motion data. We use a simplified soccer game, RoboCup, as a prototype task. Motion data from a 20 degree-of-freedom humanoid soccer playing robot are reduced to their intrinsic dimensionality by nonlinear principal component analysis. The proposed algorithm operates in of two phases. The first phase automatically segments the motion data in the reduced sensorimotor space by incrementally generating nonlinear principal component analysis with a circular constraint networks and assigning data points based on their temporal order to these networks in a conquer-and-divide fashion. Then, the second phase of the algorithm removes repeated patterns based on the distance between redundant motion patterns in the reduced sensorimotor space. The networks abstracted five motion patterns without any prior information about the number or type of motion patterns. The learned networks can be used to recognize and generate humanoid actions.

Joint attention, a process to attend to the object that the other attends to is supposed to be important for human-robot communication as well as for human-human communication. We propose an architecture for acquiring joint attention within a certain time period for realizing natural human-robot interaction. The architecture has two featured modules: a self-organizing map that makes the leaning time shorter and an automatic visual attention selector that let the agent communicate with a human synchronously. We implemented the proposed architecture in a real robot agent and found that 30 minutes was enough for acquiring joint attention with two objects. We can conclude from preliminary experiments that even if the gaze preference of the robot is different from that of the human caregiver, it can acquire joint attention.

This paper proposes a method of behavior generation for humanoids in which a robot learns sensorimotor maps in each motion module as the forward and inverse relationships between optic flows in the robot's view and motion parameters. Humanoids use these maps to determine appropriate motion parameters that generate a desired flow given by a planner. Each module consists of a planner and sensorimotor maps of primitives and can accomplish a simple task. Using a predefined module transition rule, humanoids can accomplish a complex task. Passing a ball (face-to-face pass) between two humanoids which have different camera lens and body parameters is realized as an example task.

Joint attention is supposed to be a basis of the competence of communication with others. The authors have been attacking the issue how to learn joint attention only through interactions with caregivers, in other words, without external task evaluation from a viewpoint of a constructivist approach towards both establishing a design principle of communicative robots and understanding the developmental process of human communication. This paper presents a method for quick learning of joint attention with unfamiliar persons based on a hybrid architecture that consists of (1) a quick but person-dependent learning module and (2) a slow but person-independent learning module. Experimental results show the effectiveness of the proposed method.

In this paper, we propose a controller for stable lateral stepping of a humanoid which utilizes the passive phase and changes the swinging phase of torso by the ground contact of the swing leg. Although the passive phase makes use of the gravitational power, the existence of the passive phase makes it difficult to control the period of the motion. We utilize the relationship between the amplitude of swinging torso and the motion period so that the desired period of lateral stepping is realized. Combined with a simple controller for walking in the sagittal plane, our proposed controller can enable 3-D walking with short step length.

Finding the body in uninterpreted sensory data is one of the fundamental competences to construct the body representation that influences on adaptabilities of the robot to the changes in the environment and the robot body. The invariance of sensations in self-observation seems a promising key information to find the body. However, since each sensory attribute can be invariant only in the observation of a part of the body, the robot should complementarily utilize the invariance of the multiple sensory attributes. In this paper, we propose a method of body-nonbody discrimination by complementarily utilizing multiple sensory attributes based on a conjecture about the distribution of the variance of sensations for each observing posture, where it can be approximated by a mixture of two Gaussian distributions, which are for observing the body and the nonbody, respectively. By estimating the distribution, the robot can automatically find a discrimination hyperplane to judge whether it observes its body in the current observing posture. Simple experiments show the validity of the proposed method.

The research approaches utilizing ubiquitous sensors to support human activities have become of major interest lately. Sensor fusion is one of the fundamental issues to develop such intelligent environments. The sensor fusion in previous works is performed in the task-level layer through individual representations of the sensors. Therefore, it does not provide new information by fusing sensors. This paper proposes another method that fuses sensory signals based on mutual information maximization in the signal-level layer. The fused signal provides us new information that cannot be obtained from individual sensors. As an example, this paper also shows experimental results in an audio-visual fusion task.

This paper proposes a subjective map representation that enables a multiagent system to make decisions in a dynamic, hostile environment. A typical situation can be found in the Sony four-legged robot league of the RoboCup competition [M. Veloso, et al., 1998]. The subjective map is a map of the environment that each agent maintains regardless of the objective consistency of the representation among the agents. Owing to the map's subjectivity, it is not affected by incorrect information belonging to other agents. For example, it is not affected by non-negligible errors caused by dynamic changes in the environment, such as falling down or being picked up and brought to other places by the referee. A potential field is defined on the subjective map in terms of subtasks, such as approaching and shooting the ball, and the field is dynamically up-dated so that the robot can decide what to do next. This method is compared with conventional methods that involve sharing or not sharing information.

A human-like intelligent robot is expected to have the capability to develop its cognitive functions through experience without a priori knowledge or explicit teaching. In addition, the realization of this kind of robot leads us to understand the developmental mechanisms of human beings. This paper proposes a bootstrap learning model by which a robot acquires the ability of joint attention without a caregiver's evaluation or a controlled environment based on the robot's embedded mechanisms: visual attention and learning with self-evaluation. Through learning based on the proposed model, the robot finds a correlation in sensorimotor coordination when joint attention succeeds and consequently acquires the ability of joint attention by accumulating the appropriate correlation and losing the uncorrelated coordination as statistical outliers. The experimental results show the validity of the proposed model.

This paper presents a method for generating vision-based humanoid behaviors by reinforcement learning with rhythmic walking parameters. The walking is stabilized by a rhythmic motion controller such as CPG or neural oscillator. The learning process consists of two stages: the first one is building an action space with two parameters (a forward step length and a turning angle) that inhibits combinations that are not feasible. The second is reinforcement learning with the constructed action space and the state space consisting of visual features and posture parameters to find feasible actions. The method is applied to a situation of the RoboCupSoccer humanoid league [H. Kitano and M. Asada, Advanced Robotics, 2000], that is, to approach the ball and to shoot it into the goal. Instructions by human are given to start up the learning process and the rest is completely self-learning in real situations.

This paper describes the development of a human-like multi-modal soft finger and its ability to sense the texture of objects. This fingertip has two silicon rubber layers of different hardness; strain gauges and PVDF films are randomly distributed as tactile sensors. Owing to the dynamics of the silicon between sensors, the fingertip is supposed to have several sensor modalities. Preliminary experiments show that the fingertip can detect the difference between the textures of objects (paper and wood).

We propose a hierarchical multi-module learning system based on self-interpretation of instructions given by a coach. The proposed method enables a robot: (i) to decompose a long term task that needs various kinds of information into a sequence of short term subtasks that need much less information through its self-interpretation process for the instructions given by the coach; (ii) to select sensory information needed for each subtask; and (iii) to integrate the learned behaviors to accomplish the given long term task. We show a preliminary result from a simple soccer situation in the context of RoboCup [M. Asada, et al., 1999].

Without any explicit knowledge, human infants acquire the phonemes of adults who have different articulation parameters. By building a robot that reproduces this developmental process, we expect to model as yet unknown cognitive developmental processes and to discover fundamental design principles for machines capable of vocal communication. This paper proposes a model for acquiring Japanese vowels based on observations of imitation between a mother and an infant. We tested the validity of the proposed model by examining whether a real robot can acquire vowels through interactions with a caregiver.

This paper presents a constructive model by which a robot acquires the ability of joint attention with a human caregiver based on its embedded mechanisms of visual attention and learning with self-evaluation. The former is to look at a salient object in the robot's view, and the latter is to learn sensorimotor co-ordination when visual attention has succeeded. Since the success of visual attention does not always correspond to the success of joint attention, the robot has incorrect learning data for joint attention as well as correct data. However, the robot is expected statistically to lose incorrect data as outliers since such data do not have any correlation in the sensorimotor co-ordination while correct data have a correlation. The robot consequently acquires the ability of joint attention by finding the correlation in the sensorimotor co-ordination even if multiple objects are placed at random positions in an environment and a human caregiver does not provide any task evaluation to the robot. The experimental results show that the proposed model makes the robot reproduce the developmental process of infants' joint attention. Therefore, the proposed model could be one of the models to explain how infants develop the ability of joint attention.

Inspired by the observation that infants acquire phonemes common to adults without having the capability to articulate, nor having a priori knowledge about the relationship between the sensorimotor system and phonemes, a constructivist approach to building a robot that reproduces a similar developmental process is conducted. Two general issues are addressed: what are the interactive mechanisms involved and what should be the behaviour of the caregiver/teacher? Based on findings in developmental psychology, it is conjectured that: (a) the caregiver's vocalization in response to infants' cooing reinforces the infant's articulation along the caregiver's phonemic categories; and (b) the caregiver's repetition with adult phonemes helps to specify the correspondence between cooing and the caregiver's phonemes as well as determining the acoustic properties of the phonemes. The robot consists of an artificial articulatory system with a five-degrees of freedom mechanical system deforming a silicon vocal tract connected to an artificial larynx, an extractor of formants, and a learning mechanism with self-organizing auditory and articulatory layers. Starting off with random vocalizations, the system uses the caregiver's repetitive utterances to bootstrap its learning. In order to resolve the arbitrariness in determining proper articulations, the torque to deform the tract and its resultant deformation are minimized. The experimental results, discussion and future issues are given.

This paper presents a method for humanoid walking acquisition with less energy consumption based on a two-stage genetic algorithm. In the first phase of genetic algorithm, in order to acquire the continuous walking motion, the fitness function consists of a walking distance (longer is better). In the second phase, the fitness function consists of a walking distance (longer) and energy consumption (less) for acquisition of highly energy-efficient walking. Further, we restrain the relationship among some joints and keep knee joint straight on supporting leg in order to ensure the less energy consumption. We apply the method to our platform PINO which has low-torque actuators owing to the servo modules. In order to realize a genetic process, we encode the scaling parameter of the joint movements and the phase difference between the joints into the computational simulation which considers the characteristics of the servo module used in our platform, PINO. The evolved results are applied to a real PINO and its smooth and stable walking with less energy consumption is verified.

To build an adaptive autonomous robot, it must have a certain number of external sensors to observe the environment. One physical phenomenon is observed as sensor signals flow through these sensors. In this paper, we focus on a "slip" phenomenon and try to build a network representation of slip of an anthropomorphic robot hand. A robot hand with distributed tactile sensors and a vision sensor is built to demonstrate how it acquires the representation of "slip". At the beginning of leaning, only the vision sensor can sense the slip as the movement of target, but after a while the tactile sensors can sense the slip even if it is so small that the vision sensor cannot sense it.

This paper presents a method for simultaneous learning in multiagent environment to facilitate cooperative behavior. Each agent has one policy and one action value function: the former is for action execution based on the action value function updated in the previous stage, and the latter is for learning based on the episodes experienced by the current policy. This makes all agents behave based on the fixed policies, so that the non-Markovian problem can be avoided except for the update periods that depend on the learning progress of each agent. In order to avoid the local maxima due to such asynchronous renewal of action value functions, optimistic action values are given initially, which helps to avoid the exploration process being trapped in local maxima. The experimental results applied to one of the cooperative tasks in a dynamic, multiagent environment, RoboCup, is shown and a discussion is given.

Visual attention is one of the most important issues for a vision guided mobile robot. Methods have been proposed for visual attention control based on information criterion. However, the robot had to stop walking for observation and decision. This paper presents a method which enables observation and decision more efficiently and adaptively while it is walking. The method uses the expected information gain from future observations for attention control and action decision. It also proposes an image compensation method to handle the image changes due to the robot motion. Both are used to estimate observation probabilities from the observation while it is walking and then action probabilities are estimated from a decision tree based on the information criterion. The method is applied to a four legged robot. Discussions on the visual attention control in the method and the future issues are given.

This paper proposes a developmental learning model for joint attention between a robot and a human caregiver. The proposed model has abilities to accelerate the learning and improve the final task performance owing to two kinds of developments: a robot's development and a caregiver's one. The robot's development means that the sensing and actuating capabilities of the robot change from immaturity to maturity. On the other hand, the caregiver's development is defined as that the caregiver changes the task from easy situation to difficult one. The proposed model causes these developments according to the learning progress of the robot. The experimental results show what kinds of effects the developments bring to the learning.

The conventional reinforcement learning approaches have difficulties in handling the policy alternation of the opponents because it may cause dynamic changes of state transition probabilities of which stability is necessary for the learning to converge. A multiple learning module approach would provide one solution for this problem. If we can assign multiple learning modules to different situations in which each of the module can regard the state transition probabilities as consistent, then the system would provide reasonable performance. This paper presents a method of multi-module reinforcement learning in a multi-agent environment, by which the learning agent can adapt its behaviors to the situations as results of the other agent's behaviors. We show a preliminary result of a simple soccer situation.

This paper presents a framework of the energy consumption based control system for humanoid walking. Unlike the existing two approaches (One is based on precise control and powerful actuators (full control and wide applicability), and the other is passive dynamic walk (no control, therefore limited applicability)), the method aims at less energy consumption with more applicability. Knee stretching posture contributed to the former in general and a search algorithm based on the computer simulation enabled to find a feasible control to the given task.

In our previous work (2000), we have developed a method for visual imitation by recovering the demonstrator's view based on the stereo epipolar constraint. The method is applied to the stationary pair of the stereo cameras, therefore, the visual fields to observe the motions of both the demonstrator and the learner are limited. This paper presents a method to extend our previous work by adopting a pair of rotation invariant stereo cameras that has pan and tilt motions without changing the optical center, therefore, the stereo epipolar equation does not change. The spherical projection is used to represent the constraint. The experimental results obtained are shown.

It is necessary to coordinate multiple tasks in order to cope with larger-scaled and more complicated tasks. However, it seems very hard to accomplish the multiple tasks at the same time. The paper proposes a method to resolve a conflict between task modules through the processes of their executions. Based on the proposed method, the robot can select an appropriate module according to the priority. In addition, we apply the module conflict resolution to a multiagent environment. Consequently, multiple tasks are automatically allocated to the multiple robots. As a task example, a soccer game is selected to show the validity of the proposed method. Real experiments are shown, and a discussion is given.

Passive dynamic walking (PDW) has received an increasing attention as a simple walking method with no or very little control, thus requiring a small amount of energy consumption. To the best of our knowledge, there are no PDW models with a torso although there have already been many studies on PDW. This paper presents the first step towards applying the PDW principle to humanoid robots by adding a torso to a conventional PDW model. The computer simulation shows that the walking of the PDW robot converges at a stable gait cycle only with a simple PD control applied between the torso and the stance leg to stand the torso up. Three attempts have been tested to reduce the torque to stand the torso up by: changing the desired posture of the torso, adding the soft leg tips, and changing the curvature of the sole. Simulation results are shown and discussed.

To control a robot that has many degrees of freedom and various sensors, a method to dynamically assign the degrees for a task is proposed. First, a mechanism to estimate the relation between the sensor inputs and the control outputs is derived based on the least-mean-square method. Then, by observing the information matrix of the estimator, a method to find robot's redundancy with respect to a given task is derived. Applying the proposed scheme to the visual servoing task of a manipulator, we show several experimental results demonstrating that the method can find redundancy automatically, and can assign the redundant degrees to another task.

Visual attention is one of the key issues for robots to accomplish the given tasks, and the existing methods specify the image features and attention control scheme in advance according to the task and the robot. However, in order to cope with environmental changes and/or task variations, the robot should construct its own attention mechanism. As the first step towards selective attention, this paper presents a method for image feature generation by visio-motor map learning for a mobile robot. The teaching data construct the visio-motor mapping that constrains the image feature generation and state vector estimation as well. The resultant image feature and state vector are nothing but task-oriented. The method is applied to indoor navigation and soccer shooting tasks, and a discussion is given.

Visual attention is one of the most important issues for a vision guided mobile robot not simply because visual information brings a huge amount of data but also because the visual field is limited, therefore gaze control is necessary. The paper proposes a method of sensor space segmentation for visual attention control that enables mobile robots to realize efficient observation. The efficiency is considered from a viewpoint of not geometrical reconstruction but unique action selection based on information criterion regardless of localization uncertainty. The method builds a decision tree based on the information criterion while taking the time needed for observation into account, and attention control is done by following the tree. The tree is rebuilt by introducing contextual information for more efficient attention control. The method is applied to a four legged robot that tries to shoot a ball into the goal. Discussion on the visual attention control in the method is given and the future issues are shown.

Existing robotic approaches have focused on the behavior generation assuming the observation of the internal model of the demonstrator, but have not paid any attention on how to build such a model from the learner's perception. This paper presents a computational model of view-based imitation learning without any internal model of the demonstrator. Instead, based on the stereo epipolar constraint, the robot learns to imitate the demonstrator's motion by applying adaptive visual servoing that minimizes the residual between the recovered demonstrator's body parts supposed to be viewed by the demonstrator and the learner's ones in the learner's stereo image planes, and then reproducing the recovered demonstrator's trajectories without any reconstruction of the 3D trajectories. The computer simulation and real experiment are shown and discussion is given.

Describes the issues in robotics from a viewpoint of edutainment through a series of activities in the Robot World Cup Initiative and related events, such as the International Robot Games Festival (Robofesta) supported by the Japanese government to promote creative and imaginative education programs, RoboCup Jr. which is designed for kids and the younger generation to play RoboCup games with easily constructible platforms, development of small legged robots for pets in the house or games, and education projects in system engineering. Finally, concluding remarks for future activities are given.

Visual tracking is one of the key components for robots to accomplish a given task in a dynamic environment, especially when independently moving objects are included. This paper proposes an extension of adaptive visual servoing (AVS) for unknown moving object tracking. The method utilizes binocular stereo vision, but does not need the knowledge of camera parameters. Only one assumption is that the system need stationary references in the both images by which the system can predict the motion of unknown moving objects. The basic ideas how we extended the AVS method such that it can track unknown moving objects are given and formalized into a new AVS system. The experimental results with proposed control architecture are shown and a discussion is given.

The Japanese Government plans to hold the International Robot Games Festival in 2001, which is called "RoboFesta 2001". The RoboFesta 2001 will be held both in the Osaka area and Kanagawa Prefecture in Japan, and will consist of several authoritative robot games, an international forum, and lots of robot-related exhibitions during the summer and autumn in 2001. The article describes the aims and organisation of the festival, then describes the various games.

Self localization is necessary for mobile robot navigation. The conventional method such as geometric reconstruction from landmark observations is generally time-consuming and prone to errors. This paper proposes a method which constructs a decision tree and prediction trees of the landmark appearance that enable a mobile robot with a limited visual angle to observe efficiently and make decisions without global positioning in the environment. By constructing these trees based on information criterion, the robot can accomplish the given task efficiently. The validity of the method is shown with a four legged robot.

This paper proposes multi-layered reinforcement learning by which the control structure can be decomposed into smaller transportable chunks and therefore previously learned knowledge can be applied to related tasks in a newly encountered situation. The modules in the lower networks are organized as experts to move into different categories of sensor output regions and to learn, lower level behaviors using motor commands. In the meantime, the modules in the higher networks are organized as experts which learn higher level behavior using lower modules. We apply the method to a simple soccer situation in the context of RoboCup, show the experimental results, and provide a discussion.

RoboCup is an increasingly successful attempt to promote the full integration of robotics and AI research. The most prominent feature of RoboCup is that it provides the researchers with the opportunity to demonstrate their research results as a form of competition in a dynamically changing hostile environment, defined as the international standard game definition, in which the gamut of intelligent robotics research issues are naturally involved. The article describes what we have learned from the past RoboCup activities, and overview the future perspectives of RoboCup in the next century, mainly focusing on the real robot leagues. Finally, we introduce the new leagues, one of which will have been held at RoboCup-99 in Stockholm.

Proposes an active learning method by which the learner has capabilities of self-learning and understanding the instructions by coping with cross perceptual aliasing problem caused by the state space difference between the learner and the teacher. The learner asks the teacher to give an appropriate instruction when necessary to reduce the instruction frequency. Further, the learner finds taught data inconsistent with learner's state space caused by cross perceptual aliasing, and modifies its state space based on the clustering method C4.5 so that it can successfully achieve the goal by reinforcement learning. The method is applied to the domain of RoboCup, where a learning robot attempts to approach and shoot a ball into the goal with the help of instructions given by teacher on request. The experimental results are shown and a discussion is given.

This paper proposes a method which estimates the relationships between learner's behaviors and other agents' ones in the environment through interactions (observation and action) using the method of system identification. In order to identify the model of each agent, Akaike's Information Criterion is applied to the results of Canonical Variate Analysis for the relationship between the observed data in terms of action and future observation. Next, reinforcement learning based on the estimated state vectors is performed to obtain the optimal behavior. The proposed method is applied to a soccer playing situation, where a rolling ball and other moving agents are well modeled and the learner's behaviors are successfully acquired by the method. Computer simulations and real experiments are shown and a discussion is given.

This paper proposes a method that acquires robots' behaviors based on the estimation of the state vectors. In order to acquire the cooperative behaviors in multi-robot environments, each learning robot estimates the local predictive model between the learner and the other objects separately. Based on the local predictive models, the robots learn the desired behaviors using reinforcement learning. The proposed method is applied to a soccer playing situation, where a rolling ball and other moving robots are well modeled and the learner's behaviors are successfully acquired by the method. Computer simulations and real experiments are shown and a discussion is given.

Discusses how a robot can develop its state vector according to the complexity of the interactions with its environment. A method for controlling the complexity is proposed for a vision-based mobile robot whose task is to shoot a ball into a goal avoiding collisions with a goalkeeper. First, we provide the most difficult situation (the maximum speed of the goalkeeper with chasing-a-ball behavior), and the robot estimates the full set of state vectors with the order of the major vector components by a method of system identification. The environmental complexity is defined in terms of the speed of the goalkeeper while the complexity of the state vector is the number of the dimensions of the state vector. According to the increase of the speed of the goalkeeper, the dimension of the state vector is increased by taking a trade-off between the size of the state space (the dimension) and the learning time. Simulations are shown, and other issues for the complexity control are discussed.

The ultimate goal of the RoboCup Initiative is to build a humanoid soccer team which beats a human World Cup Champion team. In this paper, we presents reasons why this goal should be pursued, and analyze technical issues involved in, humanoid to play soccer game. The analysis demonstrates the breadth of technologies that need to be developed through the course of the Challenge, which has major impacts to industries in general.

Most of existing robot learning methods have considered the environment where their robots work unchanged, therefore, the robots have to learn from scratch if they encounter new environments. This paper proposes a method which adapts robots to environmental changes by efficiently transferring a learned policy in the previous environments into a new one and effectively modifying it to cope with these changes. The resultant policy (a part of state transition map) does not seem optimal in each individual environment, but may absorb the differences between multiple environments. We apply the method to a mobile robot navigation problem of which task is to reach the target avoiding obstacles based on uninterpreted sonar and visual information. Experimental results show the validity of the method and discussion is given.

Presents a hybrid structure of reflective gait control and adaptive visual servoing by which a vision guided legged robot realizes a reflective walk. The reflective gait consists of three steps: 1) select a leg to be lifted so as to increase the body stability, 2) move one of other legs to enable the selected leg lifted, and 3) move the selected leg. During these steps, adaptive visual servoing generates a swaying motion of the robot so that it can stabilize the visual target at the desired position in the image. Combining the reflective gait and the swaying, the robot attempts to track the visual target, and as a result a reflective walk emerges. The validity of the method is shown by computer simulation and a preliminary real experiment, and future work is described.

Co-evolution has been receiving increased attention as a method for multi agent simultaneous learning. This paper discusses how multiple robots can emerge cooperative behaviors through co-evolutionary processes. As an example task, a simplified soccer game with three learning robots is selected and a genetic programming method is applied to individual population corresponding to each robot so as to obtain cooperative and competitive behaviors. The complexity of the problem can be explained twofold: co-evolution for cooperative behaviors needs exact synchronization of mutual evolutions, and three robot co-evolution requires well-complicated environment setups that may gradually change from, simpler to more complicated situations. Simulation results are shown, and a discussion is given.

Sony has provided a remarkable platform for research and development in robotic agents, namely fully autonomous legged robots. In this paper, we describe our work using Sony's legged robots to participate at the RoboCup'98 legged robot demonstration and competition. Robotic soccer represents a very challenging environment for research into systems with multiple robots that need to achieve concrete objectives, particularly in the presence of an adversary. Furthermore RoboCup'98 offers an excellent opportunity for robot entertainment. We introduce the RoboCup context and briefly present Sony's legged robot. We developed a vision-based navigation and a Bayesian localization algorithm. Team strategy is achieved through pre-defined behaviors and learning by instruction.

ortant fact is that the debate continues. The debate concerns the reliability of a robot as a model of the animal under study. In parallel, the arti cial life community enjoys its own, similar debate: can simulation of the real world be reliable, or is embodiment a necessity in order to gain understanding about life? The discussion about simulation vs. embodiment is by no means new, and actually many researchers in both \\camps" have been players in the other. For instance, some of arti cial intelligence's great theoreticians have previously tried to build robots. In the late 1950s, Minsky and others tried to build a ball catching robot (derived from the initial wish to build a ping-pong playing robot), and in the early 1970s, McCarthy tried to build an assembly robot (that should assemble a television kit) [4]. None of the projects succeeded, and as is well-known, both researchers have moved away from embodied arti cial intelligence. A typical argument for going towards simulation

This paper describes a vision-based servoing control scheme for legged robots to achieve a vision-guided swaying task utilizing a visual servoing technique. According to the controller, motions of the legs are not pre-programmed by analyzing the kinematics/dynamics of the system, but are generated by the servoing scheme reactively. The vision-based servoing scheme is a hybrid one consisting of a controller to keep the distances between feet constant (a stance servoing controller), and a visual servoing controller. Some preliminary experimental results are shown to demonstrate the effectiveness of the proposed scheme.

This paper presents an overview of three technical challenges as the RoboCup Physical Agent Challenge Phase 1: (1) moving the ball to the specified area (shooting, passing, and dribbling) with no stationary or moving obstacles, (2) catching the ball from an opponent or a common side player (receiving, goal-keeping, and intercepting), and (3) passing the ball between two players. The first two are concerned with single agent skills while the third one is related to a simple cooperative behavior. Motivation for these challenges and evaluation methodology are given.

This paper describes a method to achieve a vision-cued swaying task in unknown environments utilizing adaptive visual servoing. The proposed method has a hybrid structure consisting of a controller to keep the distances between feet constant (a stance servoing controller), and an adaptive visual servoing controller. Making use of the method, the motion of each joint need not be pre-programmed, but is generated by the method according to the motion of visual cues. An experimental result demonstrates how the proposed method realizes a vision-cued swaying behavior of the legged robot.

In this paper, we proposed a method by which a stereo vision-based mobile robot learns to reach a target by detecting and avoiding occlusions. We call the internal representation that describes the learning behavior "stereo sketch". First, an input scene is segmented into homogeneous regions by the enhanced ISODATA algorithm with minimum description length principle in terms of image coordinates and disparity information obtained from the fast stereo matching unit based on the coarse-to-fine control method. Then, in terms of the segmented regions including the target area and their occlusion status identified during the stereo and motion disparity estimation process, we construct a state space for the reinforcement learning method to obtain a target reaching behavior. As a result the robot can avoid obstacles without explicitly describing them. We give the computer simulation results and real robot implementation to show the validity of our method.

Robot learning such as reinforcement learning generally needs a well-defined state space in order to converge. However, to build such a state space is one of the main issues of the robot learning because of the inter-dependence between state and action spaces, which resembles to the well known "chicken and egg" problem. This paper proposes a method of action-based state space construction for vision-based mobile robots. Basic ideas to cope with the inter-dependence are that we define a state as a cluster of input vectors from which the robot can reach the goal state or the state already obtained by a sequence of one kind action primitive regardless of its length, and that this sequence is defined as one action. To realize these ideas, we need many data (experiences) of the robot and cluster the input vectors as hyper ellipsoids so that the whole state space is segmented into a state transition map in terms of action from which the optimal action sequence is obtained. To show the validity of the method, we apply it to a soccer robot which tries to shoot a ball into a goal. The simulation and real experiments are shown.

This paper describes an adaptive hybrid visual servoing/force controller to realize visual servoing while the manipulator exerts contact force on a surface. The proposed controller has a hybrid structure of visual servoing control and force control. Because it has an online estimator for the parameters of the camera-manipulator system and the one for the parameters of the unknown constraint surface, it only needs a priori knowledge on the manipulator kinematics and nothing any more. First, we propose an estimator for an image Jacobian matrix which describes the relation between image features and the tip position/orientation of the manipulator. Second, a method to estimate the normal vector of the unknown constraint surface is introduced. Then, an adaptive hybrid visual servoing/force controller is proposed. Finally, experimental results are shown to demonstrate the effectiveness of the proposed scheme.

Many conventional methods for map generation by mobile robots have tried to reconstruct 3-D geometric representation of the environment, which are time-consuming, error-prone, and necessary to transform the map into the information available for the given task. This paper proposes a method to acquire a statistical map representation robust to sensor noise and directly usable for navigation task. The robot is equipped with a ring of ultrasonic ranging sensors and a collision avoidance behaviour is embedded in it. First, the mobile robot explores in the environment in order to store a set of sequences of sonar data, and the principle component analysis is applied to reduce the dimensionality of the sonar data. As a result, each sequence of sonar data can be described as a score pattern of principal components. Next, these patterns are classified into typical local structures of the environment in order for the robot to discriminate them. Finally, a graph representation of the environment is constructed in which nodes and arcs correspond to these local structures and the transition probabilities between them, respectively. The validity of the method is shown by computer simulations and real robot experiments.

Reinforcement learning has recently been receiving increased attention as a method for robot learning with little or no a priori knowledge and higher capability of reactive and adaptive behaviors. However, there are two major problems in applying it to real robot tasks: how to construct the state space, and how to reduce the learning time. This paper presents a method by which a robot learns purposive behavior within less learning time by incrementally segmenting the sensor space based on the experiences of the robot. The incremental segmentation is performed by constructing local models in the state space, which is based on the function approximation of the sensor outputs to reduce the learning time and on the reinforcement signal to emerge a purposive behavior. The method is applied to a soccer robot which tried to shoot a ball into a goal, The experiments with computer simulations and a real robot are shown. As a result, our real robot has learned a shooting behavior within less than one hour training by incrementally segmenting the state space.

Coordination of multiple behaviors independently obtained by a reinforcement learning method is one of the issues in order for the method to be scaled to larger and more complex robot learning tasks. Direct combination of all the state spaces for individual modules (subtasks) needs enormous learning time, and it causes hidden states. This paper presents a method of modular learning which coordinates multiple behaviors taking account of a trade-off between learning time and performance. First, in order to reduce the learning time the whole state space is classified into two categories based on the action values separately obtained by Q learning: the area where one of the learned behaviors is directly applicable (no more learning area), and the area where learning is necessary due to competition of multiple behaviors (re-learning area). Second, hidden states are detected by model fitting to the learned action values based on the information criterion. Finally, the initial action valves in the re-learning area are adjusted so that they can be consistent with the values in the no more learning area. The method is applied to one to one soccer playing robots. Computer simulation and real robot experiments are given, to show the validity of the proposed method.

This paper presents a method for spatiotemporal segmentation of long image sequences of scenes which include multiple independently moving objects, based on the minimum description length (MDL) principle. First, a family of motion models is constructed, each of which corresponds to a physically meaningful motion such as translation with constant velocity or a combination of translation and rotation. Then, the motion description length is formulated. When an object changes the type of the motion or a new part of an object appears, the corresponding temporal or spatial segmentation is carried out. Ambiguous segmentation of two consecutive images can be resolved by minimizing the motion description length in a long sequence of images. Experiments on several real image sequences show the validity of our method.

This paper presents a method of vision-based reinforcement learning by which a robot learns to shoot a ball into a goal, and discusses several issues in applying the reinforcement learning method to a real robot with vision sensor. First, a "state-action deviation" problem is found as a form of perceptual aliasing in constructing the state and action spaces that reflect the outputs from physical sensors and actuators, respectively. To cope with this, an action set is constructed in such a way that one action consists of a series of the same action primitive which is successively executed until the current state changes. Next, to speed up the learning time, a mechanism of learning form easy missions (or LEM) which is a similar technique to "shaping" in animal learning is implemented. LEM reduces the learning time from the exponential order in the size of the state space to about the linear order in the size of the state space. The results of computer simulations and real robot experiments are given.

A camera, which is used as an artificial vision in visual servoing control, often has a zoom mechanism. A zoom mechanism cannot realize fast motion while an arm mechanism can, and it has only one degree of freedom. On the other hand the arm mechanism cannot cover wide range of change of images while the zoom mechanism can. In this paper, we propose a complementary visual servoing controller of zoom and arm mechanisms. First we discuss on the condition that camera position and zoom setting are considered as redundant. Then a visual servoing controller is proposed making use of complementary characteristics of the both mechanisms. To show the effectiveness of the proposed controller, experimental results are shown.

In this paper, a trajectory generator for a visual servoing system is proposed to make the system accomplish obstacle avoidance tasks in unknown environments. Using an estimated epipolar constraint, the proposed scheme can generate trajectories for the visual servoing system on the 2D image planes by a simple obstacle avoidance method without reconstructing 3D geometry. The proposed scheme is based on the idea, "as long as one of the projected trajectories does not intersect with projected obstacles, the trajectory in 3D space can avoid the obstacles". An experimental result is shown to demonstrate the validity of a combination of the proposed trajectory generator and the visual servoing control scheme.

This paper presents a method for spatiotemporal segmentation of long sequences of images which include multiple independently moving objects, based on the Minimum Description Length (MDL) principle. Spatiotemporal (ST) segments in the image sequence are extracted, each of which consists of edge segments having similar motions. First, we construct a family of motion models, each of which is completely determined by its specified set of equations. Then we formulate the motion description length in a long sequence based on these sets of equations. The motion state of an object at a given moment is determined by finding the model with shortest description length. Temporal segmentation is carried out when the motion state is found to have changed. At the same time, the spatial segmentation is globally optimized in such a way that the motion description of the entire scene reaches a minimum.>

A method is proposed which accomplishes a whole task consisting of plural subtasks by coordinating multiple behaviors acquired by a vision-based reinforcement learning. First, individual behaviors which achieve the corresponding subtasks are independently acquired by Q-learning, a widely used reinforcement learning method. Each learned behavior can be represented by an action-value function in terms of state of the environment and robot action. Next, three kinds of coordinations of multiple behaviors are considered; simple summation of different action-value functions, switching action-value functions according to situations, and learning with previously obtained action-value functions as initial values of a new action-value function. A task of shooting a ball into the goal avoiding collisions with an enemy is examined. The task can be decomposed into a ball shooting subtask and a collision avoiding subtask. These subtasks should be accomplished simultaneously, but they are not independent of each other.>

Proposes a versatile visual servoing control scheme with a Jacobian matrix estimator. The Jacobian matrix estimator does not need a priori knowledge of the kinematic structure and parameters of the robot system, such as camera and link parameters. The proposed visual servoing control scheme ensures the convergence of the image-features to desired trajectories, by using the estimated Jacobian matrix, which is proved by the Lyapunov stability theory. To show the effectiveness of the proposed scheme, simulation and experimental results are presented.>

An optical flow determination method is proposed which derives at a location more than two gradient constraint equations with a set of orientation-selective spatial Gaussian filters. The uncertainty measure of an optical flow is also obtained from the equations.>

A method to obtain the optimal motion description from two consecutive images including multiple moving parts is presented. It copes with segmentation and motion estimation problems. Segmentation is necessary for motion estimation of each part, and vice versa. The authors propose to use an information measure approach, based on comparisons between an individual (or pixel) and a class (or set of pixels). First, the motion of an edge segment is optimally modeled. Next, merging and splitting processes are iterated until the minimum description is obtained for the whole image. As a result, the image is segmented into several regions, each of which is represented by an edge segment list, and, at the same time, the maximum likelihood motion estimation is obtained for each region. Experiments performed on real images are shown.>

A qualitative approach to quantitatively recovering the shape and pose of a straight homogeneous generalized cylinder (SHGC) based on a weak Lambertian assumption is described. This assumption relaxes the strict cosine law of the Lambertian reflection model. The method does not need to know the lighting condition or surface albedo. The image of the projection of the axis of an SHGC is extracted. The slant angle of the SHGC is estimated using the weak Lambertian assumption along an extremal cross-section curve. The location of the SHGC's 3-D axis on other parallels is located. As a result, the pose (slant and tilt) and the shape (the shape of the cross section, the location of the axis, and the sweeping function) of an SHGC are recovered. Experimental results for both synthesized and real images are shown.>

Weak Lambertian assumption is proposed and used to determine shape and pose of cylindrical objects from a monocular intensity image. The method does not require the knowledge of lighting conditions (light intensity and lighting direction), surface properties, or albedos. Experimental results for both synthesized and real images showing the validity of the method are presented.>

A framework for bottom-up initial segmentation of color images is proposed in which the role of initial segmentation is restricted to generating indices into a knowledge base of object models. The importance of primitive knowledge, feature integration and feature salience is discussed in the case of color and texture.>

The authors describe a reliable feature-based stereo matching method which determines more reliable pairs of matching edges earlier than less reliable ones and makes use of previous matching results. The contrast of an edge point is used as the measure of reliability of the match because an edge with high contrast has high positional directional accuracy. The decisions on whether there are correspondences between pairs of lower contrast edges or not become easier because previously found more reliable matching results are available to reduce the occurrences of ambiguous matches. Experiments with complicated indoor scenes proved that the proposed method is better than those which do not consider reliability.>

In this paper, we present a method which interprets a monocular color image using three dimensional( 3-D) information extracted dim ing the interpretation process. First, an inpiit im- age is segmented into regions each of which has uniform brightness and color. Then the regions are interpreted utilizing knowledge about objects such as color, size and shape, and candidate re- gions for the objects are determined. For some candidate regions, the 3-D information can be ex- tracted such as position, direction, or size. Once the 3-D information is obtained, it can improve the current result of interpretation and constrains the interpretation for other regions. By iterating this process, the current interpretation result is improved and more accurate one is obtained. As the final result, the interpretation labels for the re- gions in the image and 3-D relationships between the objects are obtained. Experiment for several road scenes has proved the validity of the method.

A qualitative approach to quantita- tive recovery of cylindrical shape, pose and illumi- nant cpditiop is described. ''Weask Lambertian gs- sumption is introduced as a qualitative constraint on shading, which can relax the requirements for shape from shading methods in two ways: one is that it does not strictly constrain the property of the perfectly Lambertian surface of objects, and the other is that object surfaces may include spec- ular component in addition to diffused one. The method does not need to know the lighting con- dition (light intensity and lighting direction) or surface albedos. Input scenes include cylindrical objects each of which has a cylindrical surface and planar one as its cross-section. First, an actual lighting condition is transformed into a normal- ized lighting one, and an equation which relates the cross-section contour on the image plane to the shape parameter of the cylindrical object is derived based on the Weak Lambertian Assumption. In the case of scenes including plural cylindrical ob- jects, we can estimate the actual lighting direction and albedos of both diffused and specular com- ponents for each surface. Moreover, we can infer relative configuration of plural objects by making a comparison between an input image and a recon- structed image synthesized by using the estimated actual lighting direction and albedos. Experimen- tal results for both synthesized and real images are shown.

We introduce utilizations of T- junctions for stereo matching and detecition of T-junctions. By identifying T-junctions we can improve matchings near occluding con4ours. T- junction is also useful for a horizontal line to de- termtine whether or not the disparity can be inter- polated because the interpolation is allowed only if none of the terminals of the horizontal line relate with T-junctions. Edges are extracted as zero-crossings of a Lapla- cian Gaussian filtered image. We analyze the re- sponse of a Laplacian of a Gaussian filtered image at ideal T-junctions and obtain a method to de- tected T-junctions from a monocular image:.

We propose a method for model- based scene segmentation which separates object instances from the background in the scene. An object model is composed of parameterized geo- metric primitives constrained to each other accord- ing to its shape and size variations, and the rela- tionships to other object classes. As sensory data, we use a pair of color stereo images which provides color and disparity information. Using these infor- mation and the constraints in model descriptions, we approach to the problems of scene segmenta- tion and object recognition. First, we construct a height map, which shows a top view of the input scene, transforming the disparity information into 3-D space based on the height and tilt of the camera system. Using the size constraints of the object model, we separate candidate areas for object instances from the back- ground. Next, parts of the model are extracted from the scene descriptions inside each candidate area, and free parameters in the object model are determined so that the instance can be best fitted to the model. Finally, uncertainty measure for pla- nar patches is defined and applied to the extracted parts. We show the preliminary results using car models.

An approach to geometrical reasoning for world model representations that are based on planar surfaces from range and video images is described. The geometrical reasoning is regarded as an inferring process of the spatial extent of primal surfaces derived from the range image at an early stage. The inferring process has two subprocesses: expansion of a primal surface using a directional uncertainty defined by the moments around the axes on the plane fitted to it, and the determination of the boundary shape of the expanded surface using constraints on the spatial relationships between the observed data. Experimental results that were applied to road scenes in which the inferring process proved useful for the integration process of video and range image sequences are discussed.>

Describes two topics: (1) a reliable feature-based stereo matching method which determines the more reliable matching pairs earlier and then determines the less reliable ones later using the previously obtained results; and (2) a reliable calibration method for small vertical displacements. The feature point is an edge. The contrast, the position, and the direction of the edge are used to evaluate the matching reliability. Experiments with complicated indoor scenes proved that the proposed method is better than conventional methods.>

Presents an approach to build a world model representation based on planar surfaces from range and video images. The method consists of three sub-processes: (1) to extract the primal surface patches from range and video images; (2) to define the directional uncertainty for the extracted surface patch; and (3) to match the model surfaces of the object expected in the scene with the extracted surface patches using the defined directional uncertainty, model constraints, and a certainty measure of matching, which is defined by directional uncertainty. The preliminary results applied to road scenes are shown.>

An approach is presented for building a 3-D world model from range image sequences derived from road scenes including moving objects. First, a range image is transformed into a height map representing the height information from the assumed ground plane, and then it is segmented into the ground plane and objects on it. In order to capture the resolution and accuracy of the range information and to represent the consistency of the height information between different height maps, the authors define a reliability of the height information for each point on the height map. Using the reliability, the system finds the correspondences of both static and moving objects between different observations, and successively refines the height information and its reliability with newly acquired data, dealing with inconsistent data. The authors show the results obtained using landscape models and a range finder based on the structured light. >

A method for building a three-dimensional (3-D) world model for a mobile robot from sensory data derived from outdoor scenes is presented. The 3-D world model consists of four kinds of maps: a physical sensor map, a virtual sensor map, a local map, and a global map. First, a range image (physical sensor map) is transformed to a height map (virtual sensor map) relative to the mobile robot. Next, the height map is segmented into unexplored, occluded, traversable and obstacle regions from the height information. Moreover, obstacle regions are classified into artificial objects or natural objects according to their geometrical properties such as slope and curvature. A drawback of the height map (recovery of planes vertical to the ground plane) is overcome by using multiple-height maps that include the maximum and minimum height for each point on the ground plane. Multiple-height maps are useful not only for finding vertical planes but also for mapping obstacle regions into video images for segmentation. Finally, the height maps are integrated into a local map by matching geometrical parameters and by updating region labels. The results obtained using landscape models and the autonomous land vehicle simulator of the University of Maryland are shown, and constructing a global map with local maps is discussed.>

A method for representing a global map consisting of local map representations and relations between them has been developed. Sensor maps viewed at locations close to each other are integrated into a local map representation in a Cartesian coordinate system fixed to an object. First, three-dimensional (3-D) information of the edges on the floor is obtained at each sensor map by assuming the camera model and the flatness of the floor. A reliable feature is selected as a reference in the sensor map; this feature is used in finding the correspondence between the current sensor map and the following ones and in building a local map with these sensor maps. During the motion of the robot, the local map is updated by the motion stereo method unless the current reference point disappears from the sensor map. Farther edges are also represented in other local maps when the robot approaches them since the precise estimation of their locations in the current local map is difficult. Finally, the relation between local maps that represents the relative orientation and the approximate distance from the previous local map to the current one is included in the global map. The method was tested in an indoor environment; the experimental results are shown.>

A method is proposed for representing a global map for a mobile robot by using the descriptions of local maps and their relation. Sensor maps viewed at different locations close to each other are transferred into a local map represented in the object-centered coordinate system. First, the 3-D information of the edges on the floor is obtained at each sensor map (a view) by assuming the camera model and the flatness of the floor. A reliable feature is selected as a reference in the local map on which other edges are mapped. During the motion of the robot, the local map is updated by a motion stereo method until the current reference point disappears from a view. Farther edges must be represented in other local maps when the robot approaches them, since they cannot be located as precisely as closer edges can. Finally, the relationship between local maps in the context of the global map is described. The method has been tested on an indoor scene, and the experimental results are shown.>

The author presents a method for building a 3D world model for a mobile robot from sensory data. The model consists of three kinds of maps: a sensor map, a local map, and a global map. A range image (sensor map) is transformed to a height map (local map) with respect to a mobile robot. The height map is segmented into four categories (unexplored, occluded, traversable, and obstacle regions) for obstacle detection and path planning. Obstacle regions are classified into artificial objects or natural objects using both the height image and video image. One drawback of height map-the recovery of vertical planes-is overcome by the utilization of multiple height maps which include the maximum and minimum heights of each point, and the number of points in the range image mapped into one point in the height map. The multiple height map is useful not only for finding vertical planes in the height map but also for segmentation of the video image. Height maps are integrated into a global map by matching geometrical properties and updating region labels.>

A method is presented for determining the surface orientations of an object by projecting a stripe pattern on to it. Assuming orthographical projection as a camera model and parallel light projection of the stripe pattern, the method obtains a 2 1/2-D representation of objects by estimating surface normals from the slopes and intervals of the stripes in the image. The 2 1/2-D image is further divided into planar or singly curved surfaces by examining the distribution of the surface normals in gradient space. A simple application to finding a planar surface and determining its orientation and shape is shown. The error in surface orientation is discussed.>

This paper presents a simple method which determines the surface properties of an object by projecting a stripe pattern on to it. Assuming orthographical projection as a camera model and parallel light projection of the stripe pattern, the method obtains a 2(1/2)D representation of objects by estimating surface normals from the slopes and intervals of the stripes in the image. The 2(1/2)D image is further divided into planar or singly curved surfaces by examining the distribution of the surface normals in gradient space. Evaluation of the error in surface orientation is also described.

This correspondence presents a motion stereo method based on coarse-to-fine control strategy. A camera sliding straight takes images that form a set of stereo pairs. The matching proceeds from the shortest baseline pair to the longest baseline pair, using the disparity map already obtained to guide in searching for the next pair.

Stereo images taken by a mobile robot are analyzed and integrated into a world model. The knowledge on the properties of the environment, such as the flatness of floor and the richness in vertical surfaces, is arranged into constraints on matching and interpreting images. First, the stereo matcher detects edge points on the floor by predicting disparities from a camera model and testing them in actual images. Next, the correspondences of vertical edges starting at the floor edge points are established. By iterating similar procedures, vertical edges starting from edge points at different levels in scene are also detected. The hypothesis that a region between two vertical edges in scene is a vertical planar surface is proposed and tested by examining whether the disparities predicted from the 3-D geometry of the hypothesized surface contradict with those of patterns appeared in the region.

Analysis of scene viewed continuously from a robot moving in a man-made environment, such as a building or a plant, yields useful information for the navigation. The knowledge on the environment, richness of the scene in vertical edges and the flatness of the floor, is arranged in constraints for the dynamic scene analysis. The rotational component of camera motion is estimated first from image points invarient from translation. After compensating for movements by the rotation between the consecutive images, the foci of expansion of translational motion of both the robot and moving objects are determined.

Cine film and videotape are used to record a variety of natural processes in biology, medicine, meteorology, etc. This paper describes a system which detects and tracks moving objects from these records to obtain meaningful measures of their movements, such as linear and angular velocities. Features of the system are as follows. 1) In order to detect moving objects that are usually blurred, temporal differences of gray values (differences between consecutive frames) are used to separate moving objects from stationary objects, in addition to spatial differences of gray values. 2) The results of previous frames are used to guide feature extraction process of the next frame so that efficient processing of moving pictures which consists of a large number of frames is possible. 3) Uncertain parts in the current frame, such as occluded objects, are deduced using information of previous frames. 4) Misinterpreted or unknown parts in previous frames are reanalyzed using the results of later frames where those parts could be found.