A novel method for replaying behavior of a mobile robot from its memory of past experiences is presented in this paper. The method is a version of a particle filter on episode (PFoE), which applies a particle filter on the memory so as to efficiently find some similar situations with the current one. Though the original PFoE was proposed as a reinforcement learning method, we once removed the reward system from the original one so as to apply it to task teaching. In the experiment, we gave several kinds of motion to a micromouse type robot with the proposed method through a gamepad. The robot replayed the behaviors robustly with sensor feedback after several number of repetitive teaching.

In this paper, we investigate characteristics of a dynamic programming (DP) algorithm when it is applied to a control problem of a chaotic system. We choose a height task of the Acrobot, which is an underactuated robot. For various reasons, a straightforward method of DP is thought that it is not suitable for such a system. However, DP is attractive because of its conciseness, versatility, and flexibility toward boundary conditions. Our simulation results suggest that to apply DP methods to the task is possible under some conditions. Moreover, even when the conditions are not fulfilled, some interesting behavior of the Acrobot can be found.

We propose a method for generating robot motions by integrating multiple policies effective for task achievement. The method obtains a new policy efficiently based on the applied policies. However, there might be some states in which the applied policies fail to achieve the task. The failing states are found by means of a decrease in the state values. The policies for the states are then modified. In this paper, we applied this method to clutch assembly in order to demonstrate its validity for assembly tasks. We integrated insertion motion and search motion for the task and finally obtained the effective motion that accords to the task states.

To solve decision making problems of multi-agent systems, researchers have devised complicated methods, which are expected to solve curse of dimensionality. In this paper, we go to the opposite extreme. We generate cooperative behavior of two soccer robots with a simple dynamic programming (DP), which was proposed in '50s. Through the example, the ability of DP on a recent computer is measured and evaluated both qualitatively and quantitatively. We then show that the simple structure of DP is useful in obtaining behavior of robots in a convincing way. In the implementation of DP, space that is spanned by eight variables for decision making is divided into 610 million states. DP solves the optimal actions of two robots in every division and creates a look-up table, which is called a state-action map. The ability of this state-action map is measured by simulation and the result is discussed.

In this paper, the sampling real-time QMDP value method is proposed and applied to a goalkeeper task of a soccer robot. Soccer is a challenging task for an autonomous robot that has poor computing resources and sensors, and a good subject in the study of decision making in dynamic environments. A robot frequently decides its behavior without enough observation of the environment. In the proposed method, the risk of a score is solved beforehand toward every set of position of the goalkeeper and motion of the ball. The shortage of observation is calculated and represented by particle filters. The proposed method uses the risk function and the particle filters so as to choose appropriate behavior of the goalkeeper. The experiment with an actual robot suggests that the method can decide actions reflexively toward the motion of the ball.

Time optimal control method for quadruped walking robots are developed and installed into a practical robot system. Each leg is modeled as a two link manipulator whose time optimal control theory has already been established by Bobrow. However, in legged robot systems, each leg supports their body weight, and the reaction forces from its ground must be inside of their friction corn. Moreover, the ZMP (zero moment point) of the robot is constrained for stable walk. Therefore, time optimal control inputs must be designed considering these constraints. SONY ERS-7 is used as a quadruped walking robot and a fundamental experiment is done. From the experimental results, the effectiveness of the developed control algorithm is verified

The Acrobot is an underactuated robot and provides challenging control problems due to its nonlinearity. We try creating global control policies depending on the limitation of memory that is available for recording them. A swinging up task of the Acrobot is solved by dynamic programming (DP) without heuristics. The result of DP is compressed to an arbitrary size of control policy by our vector quantization (VQ) method. A chaotic but performable control policy can be obtained in the global state space of the Acrobot by DP. Moreover, we can verify that our VQ algorithm can reduce the dimension of the control policy from four to two in the process of compression

This paper proposes a method for planning robot motions by integrating multiple knowledge that is effective in task achievement. The method efficiently obtains a new policy, which is a mapping from states to actions, on the basis of the knowledge presented in a state-action map. However, in some states, the applied knowledge fails to achieve a given task. In our method, the failing states are found by using the decrease in the state values, and the policy for these states is then modified. In order to demonstrate the validity of our method, we applied it to rearrangement tasks of multiple objects. The appropriate policies were obtained by integrating programs for similar tasks and a simple rule for the task process; moreover, a new knowledge that is effective in the rearrangement tasks was extracted from the obtained policies

We have proposed the real-time QMDP method for decision making of a robot under uncertain state recognition. This method evaluates every action and chooses the best one with a particle filter for estimation and a state-value function of dynamic programming. Different from our past work, this paper applies it to a complicated decision making task that yields local maxima and discontinuity on the state-value function. We then verify whether the method can choose proper actions or not in such a condition. As an example, total behavior of a goalkeeper for robot soccer is planned by using value iteration. This task contains three strategies, which are related to three kinds of local maxima respectively. Simulations, experiments and actual games have suggested that the method can decide actions effectively according as uncertain result of state estimation.

We have proposed to use vector quantization (VQ) for compressing a decision making rule (a policy) on a multi-dimensional memory array. Our VQ method can reduce the amount of memory for recording a policy. In this paper, we compare this method with other techniques that economize the amount of memory in order to measure the ability of this method more rigidly than ever. One of the techniques is tile coding, which is frequently used for reinforcement learning. The other is the mere reduction of the resolution of a policy. Moreover, we try applying VQ to already compressed policies. As a result, vector quantized policies and double vector quantized policies could mark better performance than the others.

Vector quantization (VQ) is a useful method for compressing data in an array such as images and sounds. We have applied it to motion planning for robots. Dynamic programming is used for planning and outputs the result as a large multi-dimensional array. The array is compressed with VQ, and installed on a robot whose memory space is smaller than the array. Some experiments have shown the potential of this method. In this paper, we improve this method with value iteration algorithms that are applied to a compressed map. Simulations verified the improvement of the compression ratio with the value iteration algorithms.

Though Monte Carlo localization is a popular method for mobile robot localization, it requires a method for recovery of large estimation error in itself. In this paper, a recovery method, which is named an expansion resetting method, is newly proposed. A blending of the expansion resetting method and another, which is called the sensor resetting method, is also proposed. We then compared our methods and others in a simulated RoboCup environment. Typical accidents for mobile robots were produced in the simulator during trials. We could grasp the characteristics of each method. Especially, the blending method was robust against the kidnapped robot problems.

It sounds clever to achieve intelligence of a mobile robot by means of pre-computed algorithm, because it can cut down computation on a small computer installed on the robot. However, the amount of pre-computed results is usually too large to store. This paper proposes a compression method for pre-computed data of dynamic programming. A vector quantization method is proposed with the studies on entropy evaluation. Robot motions in RoboCup are planned by means of dynamic programming. States on the optimal state-action map are once bounded into a neighboring group and then compressed into a tiny number of state-action. The distortion, the bad side effect of compression, is evaluated and minimized. The proposed method is verified on both simulations and experiments of robots.

Uncertainty of self-localization is one of the most serious problems related to navigation of autonomous mobile robots. There are some studies that deal with this problem. However, most of them assume that the extent of the uncertainty is known or has been measured previously, though it changes easily with some trivial changes of the environment. To avoid this assumption, we take the following approach: 1) a robot uses a self- localization method that is quite robust against sensing noise and the environment change, 2) the plan for robot behavior is based on the assumption that the robot can recognize the environment perfectly, 3) a novel decision-making algorithm computes proper behavior from the planning result and self-localization results in real time. These algorithms were implemented on a soccer robot of the RoboCup four-legged robot league, and their efficiency was verified with experiments.

Seamless and high resolution images that copy a room is expected to be used for many applications (e.g. a guard system, amusement, remote control, and so forth). We are developing a mobile robot system for obtaining such images automatically. This system utilizes single kind of line marker, which is adhesive tape and is attached horizontally by a user beforehand. A mobile robot goes round a room and builds map. The map is used to collect images of above part of each marker. It means that the mobile robot does not need to know prior information of the room. Therefor, the underlying problems in this paper are localization and exploration by a mobile robot without prior information of the environment.

In this paper, we describe a novel self-localization algorithm. Self-localization methods are required for lowering the computational cost and handling vague sensor data. Thus, we propose to use only the uniform distribution to represent probability distributions in Monte Carlo localization, and name this method a uniform Monte Carlo localization (Uniform MCL). We manifest the low computational cost and robustness of Uniform MCL in the environment of RoboCup Sony legged robot league.