This paper presents a parallel permutation algorithm that achieves linear full-resolution reconfiguration of sliding-only cubic modular robots. We assume the use of a cubic module that can only slide across other modules' surfaces. The idea of a cubic modular robot with sliding-only motion primitive is a new concept that has advantages in simplifying the mechanisms of module hardware and space saving in its heterogeneous operations compared with previously studied cubic modules, such as those with sliding and convex motion primitives, or rotating motion primitives. However, because of its limited mobility, there are difficulties in managing the connectivity and scalability of the heterogeneous reconfiguration algorithm for it. To overcome these disadvantages, we introduce a parallel heterogeneous permutation method with linear operating time cost that can be incorporated into our previous full-resolution reconfiguration algorithm. We prove the correctness and completeness of the proposed algorithm. Simulation results show that the full-resolution reconfiguration algorithm that incorporates the proposed permutation algorithm reconfigures the robot structure with sliding-only cubic modules in linear operating-time cost.

Two types of reconfiguration-heterogeneous reconfiguration, in which all the modules are considered to be nonidentical, and homogeneous reconfiguration, in which they are considered to be identical-are widely studied in modular robotics research. Homogeneous reconfiguration can manage the transformation of the whole configuration of the robot structure, whereas heterogeneous reconfiguration can manage not only this transformation, but also the permutation process for navigating all modules to their exact target positions in the transformed configuration. However, the transformation process and time-consuming permutation process cannot always be executed simultaneously in heterogeneous reconfiguration. As a result, it takes much more time to reconfigure heterogeneous modular robots than it does to reconfigure homogeneous ones. In the previous research, linear homogeneous transformation algorithms for modular robots have been proposed. However, only quadratic permutation algorithms are available for heterogeneous lattice modular robots. This article studies a reconfiguration algorithm for heterogeneous lattice modular robots with linear operation time cost. The algorithm is based on simultaneous tunneling and permutation; a robot transforms its configuration via homogeneous tunneling motion, but permutation of each module's position is carried out simultaneously during the tunneling transformation. The approach also makes the time cost for the whole permutation process linear. To achieve this, the idea of a transparent meta-module is introduced, which allows the modules belonging to a meta-module to pass through the spaces occupied by other meta-modules. The algorithm is provided in both centralized and distributed form. We prove the correctness and completeness of the proposed algorithm. We also show some results of the reconfiguration simulation of heterogeneous modular robots with three-dimensional 2 × 2 × 2 cubic meta-modules by the proposed algorithm.

Reconfiguring heterogeneous modular robots in which all modules are not identical is much more time consuming than reconfiguring homogeneous ones, because ordinary heterogeneous reconfiguration is a combination of homogeneous transformation and heterogeneous permutation. While linear homogeneous transformation has been accomplished in previous research, linear heterogeneous permutation has not. This paper studies a reconfiguration algorithm for heterogeneous lattice modular robots with linear operation time cost. The algorithm is based on simultaneous tunneling and permutation, where a robot transforms its configuration via tunneling motion while permutation of each module's position is performed simultaneously during the tunneling transformation. To achieve this, we introduce the idea of a transparent meta-module that allows modules belonging to a meta-module to pass through the spaces occupied by other meta-modules. We prove the correctness and completeness of the proposed algorithm for a 2×2×2 cubic meta-module-based connected robot structure. We also show examples of the reconfiguration simulations of heterogeneous modular robots by the proposed algorithm.

This paper studies a reconfiguration algorithm for heterogeneous cubic modular robots in environments with obstacles. Tunneling is suitable for the transformation of cubic modular robots in such an environment because a tunneling robot only passes spaces that are occupied by the robot in the start and goal configurations. We have also designed a permutation algorithm that can be executed in the space used by the tunneling robot. Considering the application in environments with obstacles, we assumed a motion primitive with only sliding motion along another module's surface. This differs from the previously studied one, which allows convex motions. The proposed algorithm uses a three-dimensional 2 × 2 × 2 meta-module to guarantee the existence of mobile modules and maintain the connectivity of the robot structure during the tunneling and permutation processes. We show that the algorithm is complete for a connected robot structure with more than one meta-module and that the reconfiguration in an environment with obstacles is executed in quadratic operating time cost.

This paper presents a full-resolution reconfiguration algorithm for a heterogeneous modular robot composed of sliding cube modules with a limited motion primitive. We assume that a cube module is not allowed to make convex motions around other modules but is allowed to slide across other modules' surfaces. The availability of reconfiguration algorithm for such a module with limited mobility overcomes restrictions in the design of modular robots. To overcome the limited mobility of the assumed cube module, we introduce a method that compresses a robot with virtual walls and executes a reconfiguration and heterogeneous permutation of the robot structure in the compressed configuration. We prove the correctness and completeness of the proposed algorithm for three-dimensional connected structures. Simulation results show that the reconfiguration in free space is executed in quadratic operating time cost.

This paper presents a reconfiguration algorithm for a robot composed of sliding cube modules with a limited motion primitive. We assume that a cube module is not allowed to make convex motions around other modules but is allowed to slide across other modules' surfaces. Such a motion primitive limitation complicates the development of the reconfiguration algorithm; however, it simplifies the design of module hardware and reduces the size and manufacturing costs of modules. The algorithm uses a three-dimensional 2 × 2 × 2 meta-module to guarantee the existence of mobile modules and maintain the connectivity of the robot structure during the reconfiguration process. The transition of the meta-module is executed by the tunneling of cube modules. The tunneling movement is efficiently controlled by focusing on the motion of voids that are invoked through the transition of meta-modules. We show that the algorithm is complete for a connected robot structure with more than a certain number of meta-modules and that the reconfiguration in an environment with obstacles is executed in quadratic operating time cost.

In applying reinforcement learning (RL) to multi-robot control, the size of the learning state space easily explodes because the state space has a high dimension. Hierarchical reinforcement learning (HRL) is one of the most practical approaches to solve the problem; however, automatically decomposing a plain MDP state space into sub-spaces has not been studied thoroughly enough to be applied to practical robotics problems. We propose a method that automatically forms hierarchical sub-tasks for multi-robot delivery missions. The method executes sub-task decomposition and the learning process in a step-by-step manner, by widening the robot's range of movements around the load and gradually decreasing the domain of the load position. The method automatically detects the state in which cooperative motion among the robots is needed for them to accomplish the mission. The performance of the method is demonstrated by simulations.

Multi-agent reinforcement learning (MARL) is one of the most promising methods for solving the problem of multi-robot control. One approach for MARL is cooperative Q-learning (CoQ), which uses learning state space containing states and actions of all agents. Inspite of the mathematical foundation for learning convergence, CoQ often suffers from a state space explosion caused by the increase in the number of agents. Another approach to MARL is distributed Q-learning (DiQ), in which each agent uses learning state space not containing the states and actions of other agents. The state space for DiQ can easily be kept compact. Therefore, DiQ seems suitable for solving multi-robot control problems. However, there is no mathematical guarantee for learning convergence in DiQ and it is difficult to apply DiQ to a multi-robot control problem in which definite appointments among working robots must be considered for accomplishing a mission. To solve these problems in applying DiQ for multi-robot control, we treat the work operated by robots as a new agent that regulates robots' motion. We assume that the work has braking ability for its motion. The work stops its motion when the robot attempts to push the work in an inappropriate direction. The policy for the work braking is obtained via dynamic programming of a Markov decision process by using a map of the environment and the work's geometry. By virtue of this, DiQ without joint state space shows convergence. Simulation results also show the high performance of the proposed method in learning speed.

We have been investigating a speech processing system for ensuring safety and security, namely, acoustic-based security system. Focusing on indoor security, we have been studying for an advanced security system which can discriminate emergency shout from the other acoustic sound events based on automatic understanding of speech events.In this paper, we present our investigations, and describe fundamental results.

Blimp-type unmanned aerial vehicles (BUAVs) does not consume any energy in keeping their longitudinal position in the air, so their use as a platform for mine searching missions is anticipated. For mine searching missions, a BUAV must fly near the ground. Because there is severe limitation on the weight of equipment, such as sensors and actuators, most BUAVs are so-called under-actuated robots. For a BUAV to fly near the ground safely, a sophisticated obstacle avoidance algorithm that considers the kinematical constraints of under-actuated BUAV is needed. In developing the obstacle avoidance algorithm for under actuated robot, establishing a motion planning method is essential. To carry out applicable motion planning calculation for a BUAV, detail information about the wind velocity condition and geometry of obstacles in the mission environment is needed; however, it is very difficult to accurately estimate wind velocity distribution that is disturbed by the obstacles. In this paper, a method for estimating a rough wind condition near the ground considering the geometrical property of obstacles is proposed. The estimated rough wind condition is applied to the stochastic motion planning calculation based on dynamic programming (DP) in Markov decision process (MDP). The performance of the method is examined by computational fluid dynamical (CFD) simulation.

A blimp-type unmanned aerial vehicle (BUAV) maintains its longitudinal motion using buoyancy provided by the air around it. This means the density of a BUAV equals that of the surrounding air. Because of this, the motion of a BUAV is seriously affected by flow disturbances, whose distribution is usually non-uniform and unknown. In addition, the inertia in the heading motion is very large. There is also a strict limitation on the weight of equipment in the BUAV, so most BUAVs are so-called under-actuated robots. From this situation, it can be said that the motion planning of the BUAV considering the stochastic property of the disturbance is needed for obstacle avoidance. In this paper, we propose an approach to the motion planning of the BUAV via the application of Markov decision process (MDP). The proposed approach consists of a method to prepare a discrete MDP model of the BUAV motion and a method to maintain the effect of the unknown wind on the BUAVs motion. The performance of the methods is examined by dynamical simulation of the BUAV in the environment with wind disturbance

A method for real-time collision avoidance for underactuated autonomous underwater vehicles (AUVs) in the sea environment with unknown obstacle configurations and unknown vortex sea flow is proposed. The proposed method is an application of the Markov-decision-process- (MDP) based motion planning method. It is essential to adopt the MDP-based method for managing both the kinematics and dynamics of an AUV affected by sea flow. The real-time obstacle avoidance needs replanning when a new obstacle is discovered so that the AUV can find a suitable path around concave obstacles; however, MDP-based re-planning calculations require high computational power and are hard to be used in real time. In addition, it is impossible to know the whole distribution of the changing sea flow velocity in real time. Therefore, MDP-based re-planning calculation during the mission is not efficient. As a solution, the proposed method combines a rough re-planning considering only the geometrical properties of obstacles with rough target path tracking considering the kinematical and dynamical property of the AUV. Because there is no guarantee that the AUV can track the target path with high accuracy, the AUV is controlled to avoid collisions with obstacles at the same time. The rough target path tracking and obstacle avoidance is done using the result of off-line MDP based motion planning. The performance of the proposed method is examined by simulation of an AUV cruising in a vortex sea flow

The development of a practical motion planning and control algorithm for under-actuated robots in an unknown disturbance is a very important issue in robotics research. In the case of under actuated underwater vehicles, developing such an algorithm has been particularly problematic for several reasons. First, not only the kinematical characteristics of the motion but also the dynamical characteristics of the underwater vehicle must be considered in the motion planning calculation. Second, it is very difficult to ascertain the exact distribution of the velocity of non-uniform sea flow around obstacles on the seabed before the mission. Third, the effect of the sea flow on the motion of an underwater vehicle is very large because the speed of sea flow is very high compared with the vehicle's. This paper proposes a new method based on the application of reinforcement learning to solve these problems. Reinforcement learning based on the Markov decision process (MDP) is known to be suitable for acquiring motion control algorithms for robots acting in a stochastic environment with disturbance. However, to apply reinforcement learning method, the robot's motion must be suitably digitized and the learning environment must be equal to the robot's mission environment. This paper introduces a motion digitizing method based on an artificial neuron model and a method for making up for the difference between learning and mission environments. The performance of the proposed algorithm is examined by the dynamical simulation of an under-actuated underwater vehicle cruising in an environment with an obstacle and an unknown non-uniform flow simulated by potential flow.

A multi-degree-of-freedom (DOF) ultrasonic motor can rotate in three DOFs and does not generate noise. In addition, with an appropriate preloading mechanism, it can generate high torque for its size. The multi-DOF ultrasonic motor is, therefore, anticipated for use as a servomotor in the next generation of robots. However, for several reasons, there have been few applications of multi-DOF ultrasonic motors. One reason is the difficulty in designing a proper preloading mechanism for the motor and the limitation of the size of the stators. Another is the difficulty in developing a control algorithm, due to the motor's complex and changing dynamical characteristics, and the serious jaggy motion caused by its very quick response. This paper proposes a preloading mechanism and control algorithm for a multi-DOF ultrasonic motor, considering the motor's application to an actual auditory tele-existence robot, TeleHead. TeleHead is an elaborate dummy head robot that has a 3-DOF neck mechanism. The proposed methods achieve smooth and fast multi-DOF rotating motion of the dummy head with little time delay. In the preloading method, tensioned springs generate high preloading force and make up for the lack of torque by compensating for the resistance torque generated by the inclining motion of the dummy head. In the control algorithm, high-DOF motion is managed by high-frequency switching of the rotating axis, and smooth and quick trajectory tracking motion is achieved by introducing feed-forward control using an inverse model, with multiresolution acquired by feedback-error learning. Experimental results verify the high performance of these methods.

A multi degree of freedom (DOF) ultrasonic motor can rotate in three degrees of freedom and does not generate noise. In addition, it generates high torque with an appropriate pre-loading mechanism. This motor is therefore anticipated for use as a servomotor in robots in the next generation. This paper proposes a pre-loading mechanism and control algorithm for the multi-DOF ultrasonic motor for its application to an actual auditory tele-existence robot, TeleHead. The proposed methods realize fast multi-DOF rotating motion with little time delay. The pre-loading method generates pre-loading force by using tensioned springs, and compensates for the torque generated by the weight load on the motor by using these springs and a counter weight. The control algorithm uses feed forward control based on online feedback error learning. Experimental results show the high performance of these proposed methods.

A multi degree of freedom (multi-DOF) ultrasonic motor is anticipated as an actuator in the next generation, because it can drive three-DOF motion with high torque and low noise that can not be perceived by the human auditory system. Using a multi-DOF ultrasonic motor as an actuator that constitutes the mechanism of the neck of the auditory tele-existence robot "TeleHead" is a good example of its applicability. However getting enough torque requires a suitable pre-loading method, and properly controlling three-DOF motion requires a suitable measurement method of the three-DOF angular position of the rotor and a control algorithm that takes into account the high-DOF motion and complex physical characteristics of the motor. This paper proposes both a hardware system for pre-loading and measuring 3-DOF angular position and a software system for controlling 3-DOF motion of the motor from the view point of application to TeleHead (see video). The proposed pre-loading method can generate enough pre-loading force and reduce resistance torque when the dummy head of "TeleHead" is in the inclined state. The proposed measuring method for three-DOF motion achieves a wide range of angular position of the motion of the dummy head in each rotating direction, especially in the yaw direction. The proposed control algorithm adopts a serial switching method with a very high frequency, and does not need any dynamics model of the physical characteristics of the motor for control signal calculation. Experimental results show the high performance of the proposed system when a 1-kg load is attached to the rotor of the multi-DOF ultrasonic motor.

A training algorithm for motion planning of a nonholonomic autonomous underwater vehicle (AUV) in the strong water current is proposed in this paper. The proposed algorithm can be applied in the environment with obstacles placed in arbitrary configuration. In order to realize these functions, the Q-learning and teaching method are introduced and a multilayer structure is proposed. By introducing Q-learning, the motion of the nonholonomic AUV can be suitably treated. Taking advantage of the Baysian net, a motion planning algorithm in the case of an existence of obstacles, is derived automatically from the learned knowledge. The multilayer structure accelerates the learning process. Results of the demonstration by the simulation of control of R-One robot show the high performance of proposed algorithm.

This paper proposes a fast reinforcement learning (RL) algorithm for motion planning algorithm of nonholonomic autonomous underwater vehicle (AUV) in the strong water current with high reliability. The proposed algorithm can also be applied in an underwater environment with obstacles which are placed in arbitrary configuration. The algorithm has the ability to complete a learning process within a practical time limit. In order to accomplish a high learning speed, this paper proposes a hierarchical RL algorithm that copes with the curse of dimensionality, which comes from high complexity of dynamics of an AUV. The higher level of the algorithm refers only the position of the AUV, and learns a motion planning algorithm. The lower level of the algorithm refers to the velocity of the AUV and compensates for the undesirable nonMarkovian effects by stabilizing the dynamics motion of the AUV. The proposed hierarchical algorithm also uses a dynamics model described in the form of Bayesian-network. The dynamics model is very useful for increasing the efficiency of searching in learning process. The proposed algorithm is demonstrated by simulation. The result of the simulation shows high learning speed and performance of the proposed algorithm.

A training algorithm for dynamics control of nonholonomic AUV (autonomous underwater vehicle) is proposed in this paper which can recover from thruster failure during cruising mission. It is based on Q-learning and teaching method. The back up data that represents dynamics model expressed in the form of Bayesian net can be used effectively in this case. In order to overcome difficulties due to, making discrete expression of continuous state space of AUV, the algorithm uses multiresolution Q-value tables which is combined in the form of subsumption architecture. Simulation results show high performance of the proposed algorithm for a vertical ascent mission in a severe current condition. It is shown that AUV users can conveniently and quickly train the control algorithm of the AUV by using simulation of dynamics of the vehicle.

No abstract available.