Musculoskeletal quadruped robots driven by pneumatic artificial muscles (PAMs) have great softness. Due to the softness, the proprioceptive information of PAMs (e.g. tension) reflects the environmental information. However, how to utilize this information for stable quadrupedal gait has been rarely explored. In this work, we utilized PAM tension for stable locomotion control over uneven terrain. We newly developed a durable tension sensor and proposed tension feedback control for quadruped locomotion over uneven terrain. Our proposed controller stabilizes the trunk posture by modulating the phase of the leg. To verify the effectiveness of the proposed controller, we implement it in a simple quadrupedal model and a musculoskeletal quadruped robot driven by PAMs. Through experiments, with tension feedback, the trunk posture oscillated more stably than that without the feedback. Furthermore, over uneven terrain, the running velocity with tension feedback was higher than that without the feedback in the robot experiment. These successful results will lead to more robust musculoskeletal quadruped robots that can be employed in the real-world environment.

As a strategy to address the difficulties encountered when modeling and controlling a musculoskeletal system, we present a straightforward implementation of an autonomous decentralized motion control system in this paper; the system is inspired by the spinal reflex system of animals. We developed an artificial receptor, a muscle, and a neuron to mechanically implement the reflex mechanisms of animals. Among the reflex mechanisms, this paper presents a reflex system with reciprocal innervation for a musculoskeletal quasi-quadruped robot, including antagonist muscles. In the experiments, the robot autonomously generated a leg trajectory and a gait pattern with smooth alternating motions of the antagonist muscles through the interaction between the body, the ground, and the artificial reflex systems. To evaluate the reciprocal innervation, we compared the developed robot with one that does not include antagonist muscles. The reciprocal innervation allows for twice as many muscle implementations as those offered by the robot without antagonist muscles. Moreover, it improves the running speed by 5% on average and the flexion and extension velocities of all joints by 28% on average at around touchdowns and liftoffs of the foot. This successful result lead to implement more advanced nervous systems by solely mechanical devices.

We used cells, which are the units that make up a living body, as building blocks to design a biomachine hybrid system and develop a tactile sensor that uses living cells as sensor receptors. We fabricated a novel cell tactile sensor with the electrodes formed using printed electronics technology. This sensor comprises elastic electrodes mounted on a soft material to acquire tactile information; similar to a conventional cell tactile sensor, it acquires signals through mechanical stimulation. Further, self-organization of cells can be induced, and logical processing such as selective responses to stimuli can be performed directly by the physical system, without any coding using programming languages. The proposed novel cell tactile sensor that uses printed electrodes is small enough to mount on robots. Interestingly, we confirmed the self-healing properties of the proposed sensor after cells were injured mechanically.

Musculoskeletal robots that are based on pneumatic actuation have a variety of properties, such as compliance and back-drivability, that render them particularly appealing for human-robot collaboration. However, programming interactive and responsive behaviors for such systems is extremely challenging due to the nonlinearity and uncertainty inherent to their control. In this paper, we propose an approach for learning Bayesian Interaction Primitives for musculoskeletal robots given a limited set of example demonstrations. We show that this approach is capable of real-time state estimation and response generation for interaction with a robot for which no analytical model exists. Human-robot interaction experiments on a 'handshake' task show that the approach generalizes to new positions, interaction partners, and movement velocities.

A lot of research has been conducted on swarm and modular robots and their reconfiguration. For this kind of robot system, it is best to simplify each individual module in order to avoid making the swarm system too complex in the interest of reducing the computing power necessary for a task.This paper proposes a swarm robot system capable of self-segregation by changing the size of each agent, utilising the Brazil Nut Effect. The self-segregation allowed by this effect lead to the simplification of the control protocol necessary for the swarm.We simulated a swarm robot system consisting of circular modules capable of changing their radius and, with the aid of externally supplied vibrations, utilising the Brazil Nut Effect to achieve segregation of the swarm. The effect also allowed for the migration of individual modules through the swarm. Additionally, we harnessed this segregation effect in simulation to create a flow of modules within the swarm, whilst the swarm was confined within a vibrating container.Lastly, we took the first steps toward physical verification of the experimental results by designing and building a prototype for a circular robot module capable of changing its radius in a manner inspired by a spiral torsion spring.

Motor babbling and goal babbling has been used for sensorimotor learning of highly redundant systems in soft robotics. Recent works in goal babbling have demonstrated successful learning of inverse kinematics (IK) on such systems, and suggest that babbling in the goal space better resolves motor redundancy by learning as few yet efficient sensorimotor mappings as possible. However, for musculoskeletal robot systems, motor redundancy can provide useful information to explain muscle activation patterns, thus the term motor abundance. In this work, we introduce some simple heuristics to empirically define the unknown goal space, and learn the IK of a 10 DoF musculoskeletal robot arm using directed goal babbling. We then further propose local online motor babbling guided by Covariance Matrix Adaptation Evolution Strategy (CMA-ES), which bootstraps on the goal babbling samples for initialization, such that motor abundance can be queried online for any static goal. Our approach leverages the resolving of redundancies and the efficient guided exploration of motor abundance in two stages of learning, allowing both kinematic accuracy and motor variability at the queried goal. The result shows that local online motor babbling guided by CMA-ES can efficiently explore motor abundance at queried goal positions on a musculoskeletal robot system and gives useful insights in terms of muscle stiffness and synergy.

It has been widely considered that a distinctive feature of musculoskeletal structures is that both the joint angle and stiffness can be changed by exploiting the agonistantagonist driving of the joint. However, musculoskeletal systems in animals and humans are typically highly complex, and the simple agonist-antagonist driving is rarely found. Therefore, in accordance with the increasing complexity of musculoskeletal robots, the feature that causes the robot to assume a posture with different stiffness values becomes difficult to achieve, owing to the difficulty in modeling the kinematics. Although datadriven approaches such as the neural network are regarded as suitable for modeling complex relationships, the training data are difficult to obtain because measuring joint stiffness is typically extremely difficult in contrast to measuring an actuator's state and posture. Hence, we herein propose the common dimensional autoencoder where the encoded feature exhibits identical dimensions to the original input vector. In the proposed network, in parallel with the original unsupervised training using the data of the actuators' states, supervised training at part of the encoded features is performed using posture data. Consequently, features expressing the redundancy of inverse kinematics appear at the remaining part of the encoded features without using data such as joint stiffness. The validity of the proposed method was confirmed successfully through an experiment using a 10 degrees-of-freedom complex musculoskeletal robot arm driven by pneumatic artificial muscles.

A number of researches have been focusing on the development and control of robots with soft structures such as flexible musculoskeletal systems. Thus far, it has been reported that these robots can achieve high adaptability to environments despite their extremely simple controllers. However, because these robots are difficult to model mathematically, there is still no systematic design policy, in which control theory has been playing a role in conventional robotics, for constituting simple controllers. To tackle this problem, we propose a new approach using a neural network to obtain mathematical models. In particular, with this method, the control theory is applied to linear system models extracted from a network trained to express the forward dynamics of a robot. Through simulations, the validity and advantage of the proposed method was successfully confirmed.

Stochastic resonance (SR) is a phenomenon occurring in some nonlinear systems by which a signal provided as input that is too small in magnitude to normally influence the system's output can actually influence the system's output once a non-zero level of noise is provided. SR has been extensively studied both theoretically and experimentally, and noise has been exploited for improving the performance in both biological and artificial systems. In addition to its scientific importance, the use of noise has attracted interest because of its potential to overcome the present limitations in engineering applications. In this study, we investigate the use of a universal approximator exploiting SR as a new realization of well-established feed-forward neural networks. The proposed universal approximator consists of groups of threshold elements. Although the approximation universality of a network consisting of threshold elements has been proven in terms of extreme learning machine implementations, once SR is taken into account, the system can be modeled in a form identical to that of a classic generic three-layered neural network, for which the universal approximation capability has been proven. The capability of the proposed approximator for serving as a universal approximator is first proven theoretically in the limit of an infinite number of hidden units. Subsequently, the performance achieved by the backpropagation type and the extreme learning machine type learning algorithms is experimentally evaluated for cases involving limited numbers of hidden units, highlighting the SR effect occurring in the proposed system.

The aim of this research is to develop a musculoskeletal robot arm, which has a similar mechanical structure to that of a human, to simply make the robot move similarly to a human based on his/her electromyographic signals. In recent years, many musculoskeletal robots have been developed to show advantages of their bio-inspired designs. In this research, we propose a new perspective of their advantages that the control using biological signals can be simplified thanks to the similarity of mechanical structure. To this end, the shoulder complex, which consists of several bones and joints, is focused because it is the most difficult part to develop in humanlike musculoskeletal robot arms. In particular, we develop a linkage mechanism, which can realize similar function to the shoulder complex, to make a surface electromyography(sEMG) based posture control applicable for this complex system. The advantage, that a simple mapping is still available to control the posture to be the same to that of a human, has been successfully shown in an experiment.

Humans can walk stably and adaptively in the presence of various environmental changes. Their bodies have very complex musculoskeletal structures that contribute to their walking stability and adaptability. In this paper, we focus on the gluteus medius, one of muscles contributing to the support of the pelvis. The gluteus medius supports the pelvis when the leg is in a standing position, but it should not inhibit its smooth swing motion. This mechanism is assumed to be realized by the musculoskeletal structure. This paper is devoted to explaining the mechanism by reproducing it using artificial pneumatic muscles. We developed a musculoskeletal robot with a gluteus medius, and showed that the mechanism is well-designed for realizing two seemingly contradictory functions: being stiff to support the pelvis while standing and being smooth to allow swinging. This finding can be utilized for adaptive walking of musculoskeletal humanoid robots.

Direct teaching is suitable for generating motions of robot arms which have complex kinematics. So far, we have proposed a direct teaching method specialized for musculoskeletal robot arms actuated by pneumatic artificial muscles (PAMs) based on their pressure and tension information. In the method, it is important to prevent slacks and excessive tensions of PAMs to efficiently obtain the pressure and the tension information during the teaching phase. In this research, we propose a method for generating active behavior of musculoskeletal robots driven by PAMs to effectively receive human's direct teaching. The method, which is naturally derived from the simple analytical model of PAMs, requires only pressure and tension information of PAMs in musculoskeletal robot arms and does not need to cope with complex inverse kinematics problem. The validity of the method has been confirmed in the experiment using a minimalistic 2DOFs anthropomorphic musculoskeletal robot arm actuated by three pairs of agonist/antagonist PAMs.

Recently, the tendon-driven mechanism with variable joint stiffness has received attention for use in the development of a humanoid robot operated in an uncertain environment with physical contact. In this paper, we propose a mechanism to control the position and joint stiffness of a tendon-driven manipulator independently, using dedicated actuators. This mechanism consists of two parts: a component that transforms the movements of the tendons to activate the actuators, and a component that applies tensile forces to adjust the joint stiffness. We named this mechanism “tendon routing resolving inverse kinematics” (TRIK). The methodology for designing this mechanism for various tendon-driven manipulators is presented with several examples. We designed TRIK for a manipulator with one degree of freedom and nonconstant-moment arms. Finally, experiments of variable joint stiffness with nonlinearly elastic components were conducted to validate the proposed mechanism.

Sarcomere is a functional unit constituting a skeletal muscle, which can only contract and relax in response to changes in Ca+concentration. In order from the simple to the most complex, it builds structures corresponding to myofibrils, muscle fibers, muscle fiber bundles and the skeletal muscle. This distinctive hierarchical structure of skeletal muscles has been intensively studied in interdisciplinary research fields. In engineering, how the system efficiently controls a large number of sarcomeres to express continuous output force, is a point that has been focused. In this research, we propose a new decentralized control which is very simple but can manage many binary functional units by exploiting environmental noise. The validity of method is confirmed in both numerical simulation and a developed biologically inspired actuator.

Feline locomotion combines great acrobatic proficiency, unparalleled balance and higher accelerations than other animals. Capable of accelerating from 0 to 100 km h-1in three seconds, the cheetah (Acinonyx jubatus) is still a mystery which intrigues scientists. Aiming for a better understanding of the source of such higher speeds, we develop a biomimetic platform, where musculoskeletal parameters (range of motion and moment arms) from the biological system can be evaluated with air muscles within a lightweight robotic structure. We performed experiments validating the muscular structure during a treadmill walk, successfully reproducing animal locomotion while adopting an EMG based control method.

This paper presents a direct teaching method for musculoskeletal robots driven by pneumatic artificial muscles (PAMs). In order to reproduce motions which are directly taught by a human, it is necessary to reproduce the lengths of PAMs because they geometrically determine the posture assumed by the robot. However, it is difficult to measure the lengths of PAMs because mounting length sensors is space-consuming. Additionally, estimating lengths is also difficult because it is required to know the intrinsic parameters of PAMs which are extremely difficult to measure for each muscle. In order to overcome the above problems, the proposed method calculates the desired internal pressures or the desired axial tensions of the PAMs under a specific constraint, which forces PAM's lengths in the reproducing phase to be similar to the lengths during the teaching phase. In this way, it is possible to reproduce the motion by controlling the internal pressures or the axial tensions instead of the lengths. The validity was confirmed through an experiment using a real musculoskeletal robot arm.

In recent years, musculoskeletal robots are being intensively studied to exploit the advantages of biological musculoskeletal systems for robot developments. In these robots, it is very important to assure engineering, biomechanical, and anatomical plausibility at the same time. However, these requirements are often in contradiction. Especially, in the human's shoulder complex, mimicking the glenohumeral joint and the scapulothoracic joint has been difficult because of the need to assure a wide range of movement and the joint's stability at the same time. In this paper, we propose mechanical structures to realize the functions of the glenohumeral joints and scapulothoracic joint. These structures were used to develop a musculoskeletal robot arm driven by pneumatic artificial muscles. In addition, in order to verify the feasibility of the robot arm, we present a dynamic motion in which the robot arm throws a ball by a simple control strategy.

Stochastic resonance (SR) is a phenomenon by which the immeasurable input signals of a non-linear system can be observed in the output signals by adding a non-zero level of noise. So far, this phenomenon has been intensively studied, but no methodology for its use in engineering applications has been established yet. To exploit SR in engineering, and, in particular, to determine the appropriate noise variance that optimizes the SR performance, remains an open problem. In this study, we propose a method, which exploits the non- linear correlation between outputs from a set of redundant sensors subject to different noise sources, to tune the noise variance. Because the proposed method does not require the input signal information, it allows the exploitation of SR in realistic engineering problems. The proposed method is validated by information theory and numerical simulations. Based on the results, we developed a tactile sensing system that utilizes the method. Experimental results demonstrate that the tactile sensing system can sense immeasurable signals which are smaller than the quantization error of the sensing system.

Human's spinal reflexes response rapidly to the change of states of muscles, therefore, they are supposed to play an important role for dynamic motion. In order to realize three dimensional stable jumping, rapid control system is indispensable. In this paper, we investigate contribution of the stretch reflex for stabilizing roll motion of continuous jumping by constructing a musculoskeletal biped robot. By conducting numerical simulation and robot experiments, we show that the stretch reflex in the Soleus muscle effectively stabilizes the roll motion of the robot. Through investigation, we confirmed that roll motion was partially stabilized by implementation of the stretch reflex in the soleus muscle. This study may lead to understanding the role of spinal reflexes in human jumping and to realization of robust jumping of bipedal robots.

This paper aims to clarify the mechanism of an infant's locomotive development from the viewpoint of cognitive developmental robotics. We built up an infant-sized musculoskeletal robot driven by McKibben pneumatic actuators, which enable the robot to interact with its environment without any problems of mechanical damage and excessive heat in long-term experiments. We applied a learning algorithm based on central pattern generators and optimization method as a way for the robot to acquire its crawling motion. As a result of a developmental experiment, an efficient forward motion was acquired with the proposed method. We discuss how its musculoskeltal body, spinal structure and tonus of the artificial muscles affect its development of crawling behavior.

The human musculoskeletal system is supposed to play an important role in doing various static and dynamic tasks. From this standpoint, some musculoskeletal humanoid robots have been developed in recent years. However, existing musculoskeletal robots did not have upper body with several DOFs to balance their bodies statically or did not have enough power to perform dynamic tasks. We think the musculoskeletal structure has two significant properties: whole-body flexibility and whole-body coordination. Using these two properties can enable us to make robots' performance better than before. In this study, we developed a humanoid robot with a musculoskeletal system that is driven by pneumatic artificial muscles. To demonstrate the robot's capability in static and dynamic tasks, we conducted two experiments. As a static task, we conducted a standing experiment using a simple feedback control and evaluated the stability by applying an impulse to the robot. As a dynamic task, we conducted a walking experiment using a feedforward controller with human muscle activation patterns and confirmed that the robot was able to perform the dynamic task.

This paper describes learning of robust haptic recognition by Bionic Hand, a human-like robot hand, through dynamic interaction with the object. Bionic hand is a human-like hand that has soft skin with distributed receptors and is driven by artificial pneumatic muscles. At the beginning of learning, it utilizes the result of physical interaction with the object: thanks to the hand compliance, regrasping will leads object's posture to stable one in the hand. This result can be successively used as object classification for learning dynamic interaction between the hand and the object by a recurrent neural network. We conduct experiments and show that the proposed method is effective for robust and fast object recognition.

Most of traditional biped walkers based on passive dynamic walking have arc feet and locked ankle joints. In this paper, we propose the method to substitute the arc feet with flat feet. We hypothesize that the shape of the arc feet corresponds to a circular roll-over shape (ROS), which is a shape of a trajectory of center of pressure in the shank-fixed frame. Firstly, we show that ankle joints driven by flexible muscles antagonistically can generate a circular ROS by simple simulation and real robot experiments. Radius of the ROS can be controlled by tension of the muscles. Then, we demonstrate stable 3D limit cycle walking by a biped robot with flat feet using the proposed ROS controlling method. We also investigate its behavior and stability when extra load is added to the robot and verified that the stability of the robot is maintained by keeping the ROS. The results suggest that the ROS can be a stability measure for limit cycle walkers to realize adaptive walking.

Structure of a human’s muscular-skeleton system is supposed to play an essential role for realizing dynamic locomotion such as jumping and running. This paper investigates the effectiveness of the human-like morphology for dynamic bouncing. We designed a monopod that had a bio-mimetic muscular-skeleton system utilizing pneumatic artificial muscles. Through the experiments, we confirmed that the biarticular muscles strongly governed the coordinated movement of its body. Utilizing such synergy brought by the muscles, a simple controller could realize stable bouncing even though the pneumatic artificial muscles have complicated characteristics.

Skin is an essential component of artificial hands. It enables the use of object affordance for recognition and control, but due to its intrinsic locality and low density of current tactile sensors, stable and proper manual contacts with the objects are indispensable. Recently, design of hand structure have shown to be effective for adaptive grasping. However, such adaptive design are only introduced to the fingers in existing works of haptics and their role in recognition remains unclear. This paper introduces the design of the Bionic Hand; an anthropomorphic hand with adaptive design introduced to the whole hand and fully covered with sensitive skin. The experiment shows that anthropomorphic design of hand structure enables robust haptic recognition by convergence of object contact conditions into stable representative states through repetitive grasping. The structure of the human hand is found to solve the issue of narrowing down the sensor space for haptic object recognition by morphological computation.

Thermal sense plays an important role in object recognition. The idea to identify materials by heat flow from the skin to the touched materials have been introduced to artificial thermal sensors and proved effective. Moreover, the addition of tactile sensors is found to improve the performance of material identification by obtaining further properties of objects such as roughness and stiffness. However, the application of these sensors to autonomous recognition by a robot is not a well investigated topic. Contact with the objects will vary in such cases, but no previous sensor has shown the ability of material recognition robust to change in contact conditions with the required size and softness. The limitation is partly due to the design of current thermal sensors where the skin and the sensing devices are separated. This paper introduces a soft anthropomorphic finger with both thermal and tactile sensing elements embedded in soft polyurethane material. Experimental results show the ability to discriminate materials robust to variety in contact condition by virtue of the deformable tissue and compensation by the tactile sensors. Recognition robust to change of temperature is also realized by an additional thermal sensor.

It is expected that variation of leg compliance, that is determined not only by joint compliance but by joint angle, contributes to provide multi-modal locomotion such as walking, jumping and running by single robot. This paper describes a design of bipedal robot and feed forward controller that realize multi-modal locomotion by tuning appropriate leg compliance on individual locomotion. In order to tune physical leg compliance, we adopt rotational joints driven by antagonistic McKibben pneumatic muscles. Experimental results show that proper leg compliance provided by joint compliance and leg posture achieves a walking from standing and a sequential jumping. These results suggest that tunable leg compliance is powerful mechanism for bipedal robot to provide human-like multi-modal locomotion.

Human walking emerges from synergy of whole body dynamics: not only legs, but also a torso, arms, and a head are compliantly connected with each other by antagonistic muscles. Although change of activation of a muscle affects whole body motion, such synergy is supposed to play a great role for realizing stable walking. This paper investigates synergistic 3D limit cycle walking of an anthropomorphic biped robot whose joints are driven by artificial pneumatic muscles antagonistically. Since its walking emerges from the synergy, we cannot design the desired trajectory in a top-down manner, but can change an activation pattern of the muscles and figure out appropriate parameters for stable walking. We experimentally demonstrate that the biped walks stably with a simple limit cycle controller. This is the pioneering work for investigating synergistic stable walking of a whole body humanoid.

The passive dynamic walker (PDW) has a remarkable characteristic that it realizes cyclic locomotion without planning the joint trajectories. However, it cannot control the walking behavior because it is dominated by the fixed body dynamics. Observing the human cyclic locomotion emerged by elastic muscles, we add the compliant hip joint on PDW, and we propose a "phasic dynamics tuner" that changes the body dynamics by tuning the joint compliance in order to control the walking behavior. The joint compliance is obtained by driving the joint utilizing antagonistic and agonistic McKibben pneumatic actuators. This paper shows that PDW with the compliant joint and the phasic dynamics tuner enhances the walking performance than present PDW with passive free joints. The phasic dynamics tuner can change the walking velocity by tuning the joint compliance. Experimental results show the effectiveness of the joint compliance and the phasic dynamics tuner.

To realize adaptive and robust manipulation, a robot should have several sensing modalities and coordinate their outputs to achieve the given task based on underlying constraint in the real environment. This paper discusses on acquisition of multi-modal expression of slip consisting of vibration, pressure, and vision sensations through pick-up experiences. A sensor network is proposed to acquire the expression, whose learning ability is demonstrated by a real experiment. The applicability of the learned network is also demonstrated by experiments to realize adaptive picking

Recently, many researchers on humanoid robotics are interested in quasi-passive-dynamic walking (quasi-PDW) which is similar to human walking. It is desirable that control parameters in quasi-PDW are automatically adjusted because robots often suffer from changes in their physical parameters and the surrounding environment. Reinforcement learning (RL) can be a key technology to this adaptability, and it has been shown that RL realizes quasi-PDW in a simulation study. To apply the existing method to controlling real robots, however, requires further improvement to accelerate its learning, otherwise the robots will break down before acquiring appropriate controls. To accelerate the learning, this study employs off-policy natural actor-critic (off-NAC), and applies it to an acquisition problem of quasi-PDW. The most important feature of the off-NAC is that it reuses the samples that has already been obtained by previous controllers. This study also shows an adaptive method of the learning rate. Simulation as well as real experiments demonstrate that fast and stable learning of quasi-PDW of an unstable biped robot can be realized by our modified off-NAC

A robot should have softness and many sensors to manipulate an object dexterously and to adapt various environments. However, many existing schemes where a designer calibrates the sensor output to the world coordinate frame are difficult to adapt for such the robot. This paper proposes a learning mechanism for a robot hand which consists of anthropomorphic fingertips. The sensor for the fingertip is difficult to calibrate because the sensor receptors are embedded randomly in the soft material. The effectiveness of the proposed mechanism is demonstrated by an experiment that the robot picks up an unknown weight object.

Humans are supposed to utilize its joint elasticity to realize smooth and adaptive walking. Although such human-like biped walking is strongly affected by the terrain dynamics, it was not taken into account in robotic bipedalism since it is very difficult to model the dynamics formally. In this paper, instead of modeling the dynamics formally, we propose to estimate the relationship between actuation (air valve opening duration) and sensing (touch sensor information) by real walking trials, and to stabilize walking cycle by utilizing it. Since the terrain dynamics is involved in the relation, we can avoid to model it formally. We conducted walking experiments on various types of terrain to demonstrate the effectiveness of the proposed method.

Joint attention, a process to attend to the object that the other attends to is supposed to be important for human-robot communication as well as for human-human communication. We propose an architecture for acquiring joint attention within a certain time period for realizing natural human-robot interaction. The architecture has two featured modules: a self-organizing map that makes the leaning time shorter and an automatic visual attention selector that let the agent communicate with a human synchronously. We implemented the proposed architecture in a real robot agent and found that 30 minutes was enough for acquiring joint attention with two objects. We can conclude from preliminary experiments that even if the gaze preference of the robot is different from that of the human caregiver, it can acquire joint attention.

Joint attention is supposed to be a basis of the competence of communication with others. The authors have been attacking the issue how to learn joint attention only through interactions with caregivers, in other words, without external task evaluation from a viewpoint of a constructivist approach towards both establishing a design principle of communicative robots and understanding the developmental process of human communication. This paper presents a method for quick learning of joint attention with unfamiliar persons based on a hybrid architecture that consists of (1) a quick but person-dependent learning module and (2) a slow but person-independent learning module. Experimental results show the effectiveness of the proposed method.

In this paper, we propose a controller for stable lateral stepping of a humanoid which utilizes the passive phase and changes the swinging phase of torso by the ground contact of the swing leg. Although the passive phase makes use of the gravitational power, the existence of the passive phase makes it difficult to control the period of the motion. We utilize the relationship between the amplitude of swinging torso and the motion period so that the desired period of lateral stepping is realized. Combined with a simple controller for walking in the sagittal plane, our proposed controller can enable 3-D walking with short step length.

Finding the body in uninterpreted sensory data is one of the fundamental competences to construct the body representation that influences on adaptabilities of the robot to the changes in the environment and the robot body. The invariance of sensations in self-observation seems a promising key information to find the body. However, since each sensory attribute can be invariant only in the observation of a part of the body, the robot should complementarily utilize the invariance of the multiple sensory attributes. In this paper, we propose a method of body-nonbody discrimination by complementarily utilizing multiple sensory attributes based on a conjecture about the distribution of the variance of sensations for each observing posture, where it can be approximated by a mixture of two Gaussian distributions, which are for observing the body and the nonbody, respectively. By estimating the distribution, the robot can automatically find a discrimination hyperplane to judge whether it observes its body in the current observing posture. Simple experiments show the validity of the proposed method.

A human-like intelligent robot is expected to have the capability to develop its cognitive functions through experience without a priori knowledge or explicit teaching. In addition, the realization of this kind of robot leads us to understand the developmental mechanisms of human beings. This paper proposes a bootstrap learning model by which a robot acquires the ability of joint attention without a caregiver's evaluation or a controlled environment based on the robot's embedded mechanisms: visual attention and learning with self-evaluation. Through learning based on the proposed model, the robot finds a correlation in sensorimotor coordination when joint attention succeeds and consequently acquires the ability of joint attention by accumulating the appropriate correlation and losing the uncorrelated coordination as statistical outliers. The experimental results show the validity of the proposed model.

This paper presents a method for generating vision-based humanoid behaviors by reinforcement learning with rhythmic walking parameters. The walking is stabilized by a rhythmic motion controller such as CPG or neural oscillator. The learning process consists of two stages: the first one is building an action space with two parameters (a forward step length and a turning angle) that inhibits combinations that are not feasible. The second is reinforcement learning with the constructed action space and the state space consisting of visual features and posture parameters to find feasible actions. The method is applied to a situation of the RoboCupSoccer humanoid league [H. Kitano and M. Asada, Advanced Robotics, 2000], that is, to approach the ball and to shoot it into the goal. Instructions by human are given to start up the learning process and the rest is completely self-learning in real situations.

This paper describes the development of a human-like multi-modal soft finger and its ability to sense the texture of objects. This fingertip has two silicon rubber layers of different hardness; strain gauges and PVDF films are randomly distributed as tactile sensors. Owing to the dynamics of the silicon between sensors, the fingertip is supposed to have several sensor modalities. Preliminary experiments show that the fingertip can detect the difference between the textures of objects (paper and wood).

Without any explicit knowledge, human infants acquire the phonemes of adults who have different articulation parameters. By building a robot that reproduces this developmental process, we expect to model as yet unknown cognitive developmental processes and to discover fundamental design principles for machines capable of vocal communication. This paper proposes a model for acquiring Japanese vowels based on observations of imitation between a mother and an infant. We tested the validity of the proposed model by examining whether a real robot can acquire vowels through interactions with a caregiver.

This paper presents a constructive model by which a robot acquires the ability of joint attention with a human caregiver based on its embedded mechanisms of visual attention and learning with self-evaluation. The former is to look at a salient object in the robot's view, and the latter is to learn sensorimotor co-ordination when visual attention has succeeded. Since the success of visual attention does not always correspond to the success of joint attention, the robot has incorrect learning data for joint attention as well as correct data. However, the robot is expected statistically to lose incorrect data as outliers since such data do not have any correlation in the sensorimotor co-ordination while correct data have a correlation. The robot consequently acquires the ability of joint attention by finding the correlation in the sensorimotor co-ordination even if multiple objects are placed at random positions in an environment and a human caregiver does not provide any task evaluation to the robot. The experimental results show that the proposed model makes the robot reproduce the developmental process of infants' joint attention. Therefore, the proposed model could be one of the models to explain how infants develop the ability of joint attention.

Inspired by the observation that infants acquire phonemes common to adults without having the capability to articulate, nor having a priori knowledge about the relationship between the sensorimotor system and phonemes, a constructivist approach to building a robot that reproduces a similar developmental process is conducted. Two general issues are addressed: what are the interactive mechanisms involved and what should be the behaviour of the caregiver/teacher? Based on findings in developmental psychology, it is conjectured that: (a) the caregiver's vocalization in response to infants' cooing reinforces the infant's articulation along the caregiver's phonemic categories; and (b) the caregiver's repetition with adult phonemes helps to specify the correspondence between cooing and the caregiver's phonemes as well as determining the acoustic properties of the phonemes. The robot consists of an artificial articulatory system with a five-degrees of freedom mechanical system deforming a silicon vocal tract connected to an artificial larynx, an extractor of formants, and a learning mechanism with self-organizing auditory and articulatory layers. Starting off with random vocalizations, the system uses the caregiver's repetitive utterances to bootstrap its learning. In order to resolve the arbitrariness in determining proper articulations, the torque to deform the tract and its resultant deformation are minimized. The experimental results, discussion and future issues are given.

To build an adaptive autonomous robot, it must have a certain number of external sensors to observe the environment. One physical phenomenon is observed as sensor signals flow through these sensors. In this paper, we focus on a "slip" phenomenon and try to build a network representation of slip of an anthropomorphic robot hand. A robot hand with distributed tactile sensors and a vision sensor is built to demonstrate how it acquires the representation of "slip". At the beginning of leaning, only the vision sensor can sense the slip as the movement of target, but after a while the tactile sensors can sense the slip even if it is so small that the vision sensor cannot sense it.

This paper presents a method for simultaneous learning in multiagent environment to facilitate cooperative behavior. Each agent has one policy and one action value function: the former is for action execution based on the action value function updated in the previous stage, and the latter is for learning based on the episodes experienced by the current policy. This makes all agents behave based on the fixed policies, so that the non-Markovian problem can be avoided except for the update periods that depend on the learning progress of each agent. In order to avoid the local maxima due to such asynchronous renewal of action value functions, optimistic action values are given initially, which helps to avoid the exploration process being trapped in local maxima. The experimental results applied to one of the cooperative tasks in a dynamic, multiagent environment, RoboCup, is shown and a discussion is given.

This paper proposes a developmental learning model for joint attention between a robot and a human caregiver. The proposed model has abilities to accelerate the learning and improve the final task performance owing to two kinds of developments: a robot's development and a caregiver's one. The robot's development means that the sensing and actuating capabilities of the robot change from immaturity to maturity. On the other hand, the caregiver's development is defined as that the caregiver changes the task from easy situation to difficult one. The proposed model causes these developments according to the learning progress of the robot. The experimental results show what kinds of effects the developments bring to the learning.

This paper presents a framework of the energy consumption based control system for humanoid walking. Unlike the existing two approaches (One is based on precise control and powerful actuators (full control and wide applicability), and the other is passive dynamic walk (no control, therefore limited applicability)), the method aims at less energy consumption with more applicability. Knee stretching posture contributed to the former in general and a search algorithm based on the computer simulation enabled to find a feasible control to the given task.

In our previous work (2000), we have developed a method for visual imitation by recovering the demonstrator's view based on the stereo epipolar constraint. The method is applied to the stationary pair of the stereo cameras, therefore, the visual fields to observe the motions of both the demonstrator and the learner are limited. This paper presents a method to extend our previous work by adopting a pair of rotation invariant stereo cameras that has pan and tilt motions without changing the optical center, therefore, the stereo epipolar equation does not change. The spherical projection is used to represent the constraint. The experimental results obtained are shown.

It is necessary to coordinate multiple tasks in order to cope with larger-scaled and more complicated tasks. However, it seems very hard to accomplish the multiple tasks at the same time. The paper proposes a method to resolve a conflict between task modules through the processes of their executions. Based on the proposed method, the robot can select an appropriate module according to the priority. In addition, we apply the module conflict resolution to a multiagent environment. Consequently, multiple tasks are automatically allocated to the multiple robots. As a task example, a soccer game is selected to show the validity of the proposed method. Real experiments are shown, and a discussion is given.

Passive dynamic walking (PDW) has received an increasing attention as a simple walking method with no or very little control, thus requiring a small amount of energy consumption. To the best of our knowledge, there are no PDW models with a torso although there have already been many studies on PDW. This paper presents the first step towards applying the PDW principle to humanoid robots by adding a torso to a conventional PDW model. The computer simulation shows that the walking of the PDW robot converges at a stable gait cycle only with a simple PD control applied between the torso and the stance leg to stand the torso up. Three attempts have been tested to reduce the torque to stand the torso up by: changing the desired posture of the torso, adding the soft leg tips, and changing the curvature of the sole. Simulation results are shown and discussed.

To control a robot that has many degrees of freedom and various sensors, a method to dynamically assign the degrees for a task is proposed. First, a mechanism to estimate the relation between the sensor inputs and the control outputs is derived based on the least-mean-square method. Then, by observing the information matrix of the estimator, a method to find robot's redundancy with respect to a given task is derived. Applying the proposed scheme to the visual servoing task of a manipulator, we show several experimental results demonstrating that the method can find redundancy automatically, and can assign the redundant degrees to another task.

Visual tracking is one of the key components for robots to accomplish a given task in a dynamic environment, especially when independently moving objects are included. This paper proposes an extension of adaptive visual servoing (AVS) for unknown moving object tracking. The method utilizes binocular stereo vision, but does not need the knowledge of camera parameters. Only one assumption is that the system need stationary references in the both images by which the system can predict the motion of unknown moving objects. The basic ideas how we extended the AVS method such that it can track unknown moving objects are given and formalized into a new AVS system. The experimental results with proposed control architecture are shown and a discussion is given.

This paper proposes a method which estimates the relationships between learner's behaviors and other agents' ones in the environment through interactions (observation and action) using the method of system identification. In order to identify the model of each agent, Akaike's Information Criterion is applied to the results of Canonical Variate Analysis for the relationship between the observed data in terms of action and future observation. Next, reinforcement learning based on the estimated state vectors is performed to obtain the optimal behavior. The proposed method is applied to a soccer playing situation, where a rolling ball and other moving agents are well modeled and the learner's behaviors are successfully acquired by the method. Computer simulations and real experiments are shown and a discussion is given.

This paper proposes a method that acquires robots' behaviors based on the estimation of the state vectors. In order to acquire the cooperative behaviors in multi-robot environments, each learning robot estimates the local predictive model between the learner and the other objects separately. Based on the local predictive models, the robots learn the desired behaviors using reinforcement learning. The proposed method is applied to a soccer playing situation, where a rolling ball and other moving robots are well modeled and the learner's behaviors are successfully acquired by the method. Computer simulations and real experiments are shown and a discussion is given.

Discusses how a robot can develop its state vector according to the complexity of the interactions with its environment. A method for controlling the complexity is proposed for a vision-based mobile robot whose task is to shoot a ball into a goal avoiding collisions with a goalkeeper. First, we provide the most difficult situation (the maximum speed of the goalkeeper with chasing-a-ball behavior), and the robot estimates the full set of state vectors with the order of the major vector components by a method of system identification. The environmental complexity is defined in terms of the speed of the goalkeeper while the complexity of the state vector is the number of the dimensions of the state vector. According to the increase of the speed of the goalkeeper, the dimension of the state vector is increased by taking a trade-off between the size of the state space (the dimension) and the learning time. Simulations are shown, and other issues for the complexity control are discussed.

Presents a hybrid structure of reflective gait control and adaptive visual servoing by which a vision guided legged robot realizes a reflective walk. The reflective gait consists of three steps: 1) select a leg to be lifted so as to increase the body stability, 2) move one of other legs to enable the selected leg lifted, and 3) move the selected leg. During these steps, adaptive visual servoing generates a swaying motion of the robot so that it can stabilize the visual target at the desired position in the image. Combining the reflective gait and the swaying, the robot attempts to track the visual target, and as a result a reflective walk emerges. The validity of the method is shown by computer simulation and a preliminary real experiment, and future work is described.

This paper describes a vision-based servoing control scheme for legged robots to achieve a vision-guided swaying task utilizing a visual servoing technique. According to the controller, motions of the legs are not pre-programmed by analyzing the kinematics/dynamics of the system, but are generated by the servoing scheme reactively. The vision-based servoing scheme is a hybrid one consisting of a controller to keep the distances between feet constant (a stance servoing controller), and a visual servoing controller. Some preliminary experimental results are shown to demonstrate the effectiveness of the proposed scheme.

This paper describes a method to achieve a vision-cued swaying task in unknown environments utilizing adaptive visual servoing. The proposed method has a hybrid structure consisting of a controller to keep the distances between feet constant (a stance servoing controller), and an adaptive visual servoing controller. Making use of the method, the motion of each joint need not be pre-programmed, but is generated by the method according to the motion of visual cues. An experimental result demonstrates how the proposed method realizes a vision-cued swaying behavior of the legged robot.

Robot learning such as reinforcement learning generally needs a well-defined state space in order to converge. However, to build such a state space is one of the main issues of the robot learning because of the inter-dependence between state and action spaces, which resembles to the well known "chicken and egg" problem. This paper proposes a method of action-based state space construction for vision-based mobile robots. Basic ideas to cope with the inter-dependence are that we define a state as a cluster of input vectors from which the robot can reach the goal state or the state already obtained by a sequence of one kind action primitive regardless of its length, and that this sequence is defined as one action. To realize these ideas, we need many data (experiences) of the robot and cluster the input vectors as hyper ellipsoids so that the whole state space is segmented into a state transition map in terms of action from which the optimal action sequence is obtained. To show the validity of the method, we apply it to a soccer robot which tries to shoot a ball into a goal. The simulation and real experiments are shown.

This paper describes an adaptive hybrid visual servoing/force controller to realize visual servoing while the manipulator exerts contact force on a surface. The proposed controller has a hybrid structure of visual servoing control and force control. Because it has an online estimator for the parameters of the camera-manipulator system and the one for the parameters of the unknown constraint surface, it only needs a priori knowledge on the manipulator kinematics and nothing any more. First, we propose an estimator for an image Jacobian matrix which describes the relation between image features and the tip position/orientation of the manipulator. Second, a method to estimate the normal vector of the unknown constraint surface is introduced. Then, an adaptive hybrid visual servoing/force controller is proposed. Finally, experimental results are shown to demonstrate the effectiveness of the proposed scheme.

Reinforcement learning has recently been receiving increased attention as a method for robot learning with little or no a priori knowledge and higher capability of reactive and adaptive behaviors. However, there are two major problems in applying it to real robot tasks: how to construct the state space, and how to reduce the learning time. This paper presents a method by which a robot learns purposive behavior within less learning time by incrementally segmenting the sensor space based on the experiences of the robot. The incremental segmentation is performed by constructing local models in the state space, which is based on the function approximation of the sensor outputs to reduce the learning time and on the reinforcement signal to emerge a purposive behavior. The method is applied to a soccer robot which tried to shoot a ball into a goal, The experiments with computer simulations and a real robot are shown. As a result, our real robot has learned a shooting behavior within less than one hour training by incrementally segmenting the state space.

Coordination of multiple behaviors independently obtained by a reinforcement learning method is one of the issues in order for the method to be scaled to larger and more complex robot learning tasks. Direct combination of all the state spaces for individual modules (subtasks) needs enormous learning time, and it causes hidden states. This paper presents a method of modular learning which coordinates multiple behaviors taking account of a trade-off between learning time and performance. First, in order to reduce the learning time the whole state space is classified into two categories based on the action values separately obtained by Q learning: the area where one of the learned behaviors is directly applicable (no more learning area), and the area where learning is necessary due to competition of multiple behaviors (re-learning area). Second, hidden states are detected by model fitting to the learned action values based on the information criterion. Finally, the initial action valves in the re-learning area are adjusted so that they can be consistent with the values in the no more learning area. The method is applied to one to one soccer playing robots. Computer simulation and real robot experiments are given, to show the validity of the proposed method.

This paper presents a method of vision-based reinforcement learning by which a robot learns to shoot a ball into a goal, and discusses several issues in applying the reinforcement learning method to a real robot with vision sensor. First, a "state-action deviation" problem is found as a form of perceptual aliasing in constructing the state and action spaces that reflect the outputs from physical sensors and actuators, respectively. To cope with this, an action set is constructed in such a way that one action consists of a series of the same action primitive which is successively executed until the current state changes. Next, to speed up the learning time, a mechanism of learning form easy missions (or LEM) which is a similar technique to "shaping" in animal learning is implemented. LEM reduces the learning time from the exponential order in the size of the state space to about the linear order in the size of the state space. The results of computer simulations and real robot experiments are given.

A camera, which is used as an artificial vision in visual servoing control, often has a zoom mechanism. A zoom mechanism cannot realize fast motion while an arm mechanism can, and it has only one degree of freedom. On the other hand the arm mechanism cannot cover wide range of change of images while the zoom mechanism can. In this paper, we propose a complementary visual servoing controller of zoom and arm mechanisms. First we discuss on the condition that camera position and zoom setting are considered as redundant. Then a visual servoing controller is proposed making use of complementary characteristics of the both mechanisms. To show the effectiveness of the proposed controller, experimental results are shown.

In this paper, a trajectory generator for a visual servoing system is proposed to make the system accomplish obstacle avoidance tasks in unknown environments. Using an estimated epipolar constraint, the proposed scheme can generate trajectories for the visual servoing system on the 2D image planes by a simple obstacle avoidance method without reconstructing 3D geometry. The proposed scheme is based on the idea, "as long as one of the projected trajectories does not intersect with projected obstacles, the trajectory in 3D space can avoid the obstacles". An experimental result is shown to demonstrate the validity of a combination of the proposed trajectory generator and the visual servoing control scheme.

A macro-micro manipulator system has been proposed with the purpose of overcoming the difficulty to realize a trajectory tracking control of a flexible manipulator. In this paper, a trajectory tracking controller of macro-micro manipulator system is proposed which takes the dynamics of the system into consideration. This control law consists of a simple PD feedback law for the macro flexible manipulator and a dynamic trajectory tracking control law for the micro rigid manipulator. Experimental results show the effectiveness of the proposed scheme.>

In this paper, to realize the hybrid position/force control of a flexible manipulator, the macro-micro manipulator system is used. The macro manipulator can move widely, but can not realize fast and precise motion because of its flexibility. To compensate the position and force errors, a micro manipulator is attached at the tip of the macro manipulator. The kinematic equations of the macro-micro manipulator system are introduced. By utilizing the redundancy and flexibility of the system, the trajectory planning is discussed. A quasi-static hybrid position/force control algorithm and a dynamic hybrid position/force control algorithm are proposed. Lastly, experimental results are shown to verify the effectiveness of the proposed control algorithms.>

A method is proposed which accomplishes a whole task consisting of plural subtasks by coordinating multiple behaviors acquired by a vision-based reinforcement learning. First, individual behaviors which achieve the corresponding subtasks are independently acquired by Q-learning, a widely used reinforcement learning method. Each learned behavior can be represented by an action-value function in terms of state of the environment and robot action. Next, three kinds of coordinations of multiple behaviors are considered; simple summation of different action-value functions, switching action-value functions according to situations, and learning with previously obtained action-value functions as initial values of a new action-value function. A task of shooting a ball into the goal avoiding collisions with an enemy is examined. The task can be decomposed into a ball shooting subtask and a collision avoiding subtask. These subtasks should be accomplished simultaneously, but they are not independent of each other.>

Proposes a versatile visual servoing control scheme with a Jacobian matrix estimator. The Jacobian matrix estimator does not need a priori knowledge of the kinematic structure and parameters of the robot system, such as camera and link parameters. The proposed visual servoing control scheme ensures the convergence of the image-features to desired trajectories, by using the estimated Jacobian matrix, which is proved by the Lyapunov stability theory. To show the effectiveness of the proposed scheme, simulation and experimental results are presented.>

In this paper, a method for PTP and trajectory control of Cartesian type three DOF industrial manipulator with flexible joints are proposed. The elasticity of the manipulator is modeled using the spring-mass model. To establish the equation of motion, the Lagrangian formulation is used. The state equation is derived. Then it is linearized approximately. The PTP control system is formulated. A minimal order observer is used in this PTP control system, and the trajectory control is realized based on the optimal control theory. Furthermore, the desired trajectory is modified to improve the tracking performance. Finally, computer simulation results are given to show the validity of the proposed tracking controller.

For developing a dynamic model of a flexible manipulator, a modeling scheme using virtual rigid links and passive joints is proposed. The parameters of the model are identified by measured data of the real arm, so it is expected that the dynamic behavior of the model is similar to that of the real arm. The equations of motion of an N degree of freedom flexible arm using this model are derived to simulate its dynamic behavior. The results of simulation and experiment show the validity of this modeling scheme.>