Navigating unknown three-dimensional (3D) rugged environments is challenging for multi-robot systems. Traditional discrete systems struggle with rough terrain due to limited individual mobility, while modular systems—where rigid, controllable constraints link robot units—improve traversal but suffer from high control complexity and reduced flexibility. To address these limitations, we propose the Multi-Robot System with Controllable Weak Constraints (MRS-CWC), where robot units are connected by constraints with dynamically adjustable stiffness. This adaptive mechanism softens or stiffens in real time during environmental interactions, ensuring a balance between flexibility and mobility. We formulate the system’s dynamics and control model and evaluate MRS-CWC against six baseline methods and an ablation variant in a benchmark dataset with 100 different simulation terrains. Results show that MRS-CWC achieves the highest navigation completion rate and ranks second in success rate, efficiency, and energy cost in the highly rugged terrain group, outperforming all baseline methods without relying on environmental modeling, path planning, or complex control. Even where MRS-CWC ranks second, its performance is only slightly behind a more complex ablation variant with environmental modeling and path planning. Finally, we develop a physical prototype and validate its feasibility in a constructed rugged environment. For videos, simulation benchmarks, and code, please visit https://wyd0817.github.io/project-mrs-cwc/.

We present a novel acoustic camera simulator (ACSim) that generates realistic sonar images by incorporating recursive ray tracing and sonar artifact modeling and provides various ground truth labels, enabling benchmarking and learning purposes. The 2D forward-looking sonar (FLS), also known as the acoustic camera, produces high-quality 2D images. Conducting real-world underwater experiments is challenging, making realistic sonar image simulation a necessary alternative. However, existing simulators often lack sufficient realism or are limited to specific scenes and phenomena. As a result, training on simulations and testing on real sonar images (i.e., sim-to-real) remains an open problem for deep learning-based applications. Our work introduces a novel sonar simulator with a customized rendering engine. We use recursive ray tracing to model multipath reflections in arbitrary scenes and propose physics-based shading for intensity computation. We propose a resampling method for anti-aliasing and model significant artifacts such as rolling shutter distortions and cross-talk noise. The simulator provides various ground truths for benchmarking and deep learning applications. We tested several tasks by training on synthetic images and demonstrated that the models also work on real images. We developed a Blender add-on for an enhanced user interface and will make the simulator open-source to advance future research.

We approach designing a state-space model for deep learning applications through its dual representation, the transfer function, and uncover a highly efficient sequence parallel inference algorithm that is state-free: unlike other proposed algorithms, state-free inference does not incur any significant memory or computational cost with an increase in state size. We achieve this using properties of the proposed frequency domain transfer function parametrization, which enables direct computation of its corresponding convolutional kernel’s spectrum via a single Fast Fourier Transform. Our experimental results across multiple sequence lengths and state sizes illustrates, on average, a 35% training speed improvement over S4 layers – parametrized in time-domain – on the Long Range Arena benchmark, while delivering state-of-the-art downstream performances over other attention-free approaches. Moreover, we report improved perplexity in language modeling over a long convolutional Hyena baseline, by simply introducing our transfer function parametrization. Our code is available at https://github.com/ruke1ire/RTF.

A system is expected to facilitate coordination among multiple construction machines or robots, enabling them to adaptively perform various tasks in disaster sites and unknown environments. Prior research has generally adopted a model-based approach to designing cooperative behavior. However, it is difficult to adapt to environments and scenarios that cannot be predicted by the model. In recent years, it has been reported that a robot equipped with foundation models can adapt to unknown (open) environments and unpredictable situations. However, there has been little discussion on foundation models for multiple robot systems; a flow that cooperatively handles unexpected events does not exist. In this paper, we propose the system flow that enables multiple robots to adaptively coordinate to unforeseen scenarios based on the functional expressions of each other and environment understanding utilizing GPT-4 and GPT-4V. Through experimentation, we verify that the proposed flow is able to adapt to an unforeseen environment, particularly path obstruction via robot experiments. Furthermore, we examine the validity of the proposed flow by varying the robots’ functional expressions and sensor information for the environment.

Mobile robots functioning in human environments should behave with a secure and socially-compliant manner. Although many studies have revealed the effectiveness of Deep Reinforcement Learning (DRL) in robot navigation, most of them can only handle the presence of human as independent individuals. Failing to consider groups may lead to the robot getting stuck or behaving rudely, and omitting to separately handle obstacles from pedestrians will cause low efficiency. In this work, we present a novel all-aware neural network that utilizes DRL to process groups, obstacles, and individuals simultaneously. The proposed solution employs a new Group–Robot Interaction (GRI) subnetwork to encode the mutual effects between groups and the robot, and a modified Obstacle–Robot Unilateral interaction (ORU) subnetwork is presented to avoid obstacle collisions caused by sensing noises or motion uncertainties. In addition, the influences of a pedestrian, obstacle, and group on other pedestrians or groups, that indirectly affect the robot, are also integrated into the Human–Robot Interaction (HRI) subnetwork or GRI subnetwork respectively by using map tensors. Finally, the GRI, ORU, and HRI subnetworks are aggregated into a planning subnetwork to train and derive an all-aware robot navigation policy based on DRL. Evaluation results in both real-world and simulation experiments show that the proposed approach outperforms the current cutting-edge methods.

2D forward-looking sonar is a crucial sensor for underwater robotic perception. A well-known problem in this field is estimating missing information in the elevation direction during sonar imaging. There are demands to estimate 3D information per image for 3D mapping and robot navigation during fly-through missions. Recent learning-based methods have demonstrated their strengths, but there are still drawbacks. Supervised learning methods have achieved high-quality results but may require further efforts to acquire 3D ground-truth labels. The existing self-supervised method requires pretraining using synthetic images with 3D supervision. This study aims to realize stable self-supervised learning of elevation angle estimation without pretraining using synthetic images. Failures during self-supervised learning may be caused by motion degeneracy problems. We first analyze the motion field of 2D forward-looking sonar, which is related to the main supervision signal. We utilize a modern learning framework and prove that if the training dataset is built with effective motions, the network can be trained in a self-supervised manner without the knowledge of synthetic data. Both simulation and real experiments validate the proposed method.

Mobile robot navigation in a human-populated environment has been of great interest to the research community in recent years, referred to as crowd navigation. Currently, offline reinforcement learning (RL)-based method has been introduced to this domain, for its ability to alleviate the sim2real gap brought by online RL which relies on simulators to execute training, and its scalability to use the same dataset to train for differently customized rewards. However, the performance of the navigation policy suffered from the distributional shift between the training data and the input during deployment, since when it gets an input out of the training data distribution, the learned policy has the risk of choosing an erroneous action that leads to catastrophic failure such as colliding with a human. To realize risk sensitivity and improve the safety of the offline RL agent during deployment, this work proposes a multipolicy control framework that combines offline RL navigation policy with a risk detector and a force-based risk-avoiding policy. In particular, a Lyapunov density model is learned using the latent feature of the offline RL policy and works as a risk detector to switch the control to the risk-avoiding policy when the robot has a tendency to go out of the area supported by the training data. Experimental results showed that the proposed method was able to learn navigation in a crowded scene from the offline trajectory dataset and the risk detector substantially reduces the collision rate of the vanilla offline RL agent while maintaining the navigation efficiency outperforming the state-of-the-art methods.

Retrieving the missing dimension information in acoustic images from 2D forward-looking sonar is a well-known problem in the field of underwater robotics. There are works attempting to retrieve 3D information from a single image which allows the robot to generate 3D maps with fly-through motion. However, owing to the unique image formulation principle, estimating 3D information from a single image faces severe ambiguity problems. Classical methods of multi-view stereo can avoid the ambiguity problems, but may require a large number of viewpoints to generate an accurate model. In this work, we propose a novel learning-based multi-view stereo method to estimate 3D information. To better utilize the information from multiple frames, an elevation plane sweeping method is proposed to generate the depth-azimuth-elevation cost volume. The volume after regularization can be considered as a probabilistic volumetric representation of the target. Instead of performing regression on the elevation angles, we use pseudo front depth from the cost volume to represent the 3D information which can avoid the 2D-3D problem in acoustic imaging. High-accuracy results can be generated with only two or three images. Synthetic datasets were generated to simulate various underwater targets. We also built the first real dataset with accurate ground truth in a large scale water tank. Experimental results demonstrate the superiority of our method, compared to other state-of-the-art methods.

We detail a novel class of implicit neural models. Leveraging time-parallel methods for differential equations, Multiple Shooting Layers (MSLs) seek solutions of initial value problems via parallelizable root-finding algorithms. MSLs broadly serve as drop-in replacements for neural ordinary differential equations (Neural ODEs) with improved efficiency in number of function evaluations (NFEs) and wall-clock inference time. We develop the algorithmic framework of MSLs, analyzing the different choices of solution methods from a theoretical and computational perspective. MSLs are showcased in long horizon optimal control of ODEs and PDEs and as latent models for sequence generation. Finally, we investigate the speedups obtained through application of MSL inference in neural controlled differential equations (Neural CDEs) for time series classification of medical data.

Effective control and prediction of dynamical systems require appropriate handling of continuous-time and discrete, event-triggered processes. Stochastic hybrid systems (SHSs), common across engineering domains, provide a formalism for dynamical systems subject to discrete, possibly stochastic, state jumps and multi-modal continuous-time flows. Despite the versatility and importance of SHSs across applications, a general procedure for the explicit learning of both discrete events and multi-mode continuous dynamics remains an open problem. This work introduces Neural Hybrid Automata (NHAs), a recipe for learning SHS dynamics without a priori knowledge on the number, mode parameters, and inter-modal transition dynamics. NHAs provide a systematic inference method based on normalizing flows, neural differential equations, and self-supervision. We showcase NHAs on several tasks, including mode recovery and flow learning in systems with stochastic transitions, and end-to-end learning of hierarchical robot controllers.

In this paper, we propose a novel method for 3D measurement of large structures that have sparse features. The proposed method uses a structured-light approach based on a spherical camera and an omnidirectional ring laser. The spherical camera can capture omnidirectional images which enable it to view all sparse feature points existing in the target environment. The omnidirectional ring laser can enable dense 3D measurement of cross-sections of the environment via the structured light method. Structure from Motion (SfM) is used to measure the motion of the camera to integrate the laser cross sections to obtain a dense 3D model.However, the result of SfM does not contain real-world scale information. The novelty of this research lies in a new method to obtain the real-world scale. The real-world scale is determined by comparing a mesh generated from the resultant SfM point cloud and the integrated laser sections in terms of each scale.In a simulated environment, the proposed method was found to be accurate up to 1 mm. It was also able to accurately measure the 3D shape of a real environment.

In this study, we present a method for all-around depth estimation from multiple omnidirectional images for indoor environments. In particular, we focus on plane-sweeping stereo as the method for depth estimation from the images. We propose a new icosahedron-based representation and ConvNets for omnidirectional images, which we name "CrownConv" because the representation resembles a crown made of origami. CrownConv can be applied to both fisheye images and equirect- angular images to extract features. Furthermore, we propose icosahedron-based spherical sweeping for generating the cost volume on an icosahedron from the extracted features. The cost volume is regularized using the three-dimensional CrownConv, and the final depth is obtained by depth regression from the cost volume. Our proposed method is robust to camera alignments by using the extrinsic camera parameters; therefore, it can achieve precise depth estimation even when the camera alignment differs from that in the training dataset. We evaluate the proposed model on synthetic datasets and demonstrate its effectiveness. As our proposed method is computationally efficient, the depth is estimated from four fisheye images in less than a second using a laptop with a GPU. Therefore, it is suitable for real-world robotics applications. Our source code is available at https://github.com/matsuren/crownconv360depth.

Continuous deep learning architectures have recently re-emerged as Neural Ordinary Differential Equations (Neural ODEs). This infinite-depth approach theoretically bridges the gap between deep learning and dynamical systems, offering a novel perspective. However, deciphering the inner working of these models is still an open challenge, as most applications apply them as generic black-box modules. In this work we ``open the box'', further developing the continuous-depth formulation with the aim of clarifying the influence of several design choices on the underlying dynamics.

The infinite-depth paradigm pioneered by Neural ODEs has launched a renaissance in the search for novel dynamical system-inspired deep learning primitives; however, their utilization in problems of non-trivial size has often proved impossible due to poor computational scalability. This work paves the way for scalable Neural ODEs with time-to-prediction comparable to traditional discrete networks. We introduce hypersolvers, neural networks designed to solve ODEs with low overhead and theoretical guarantees on accuracy. The synergistic combination of hypersolvers and Neural ODEs allows for cheap inference and unlocks a new frontier for practical application of continuous-depth models. Experimental evaluations on standard benchmarks, such as sampling for continuous normalizing flows, reveal consistent pareto efficiency over classical numerical methods.

Spherical cameras, which can acquire all-round information, are effective to estimate rotation for robotic applications. Recently, Convolutional Neural Networks have shown great robustness in solving such regression problems. However they are designed for planar images and cannot deal with the non-uniform distortion present in spherical images, when expressed in the planar equirectangular projection. This can lower the accuracy of motion estimation. In this research, we propose an Equirectangular-Convolutional Neural Network (E-CNN) to solve this issue. This novel network regresses 3D spherical camera rotation by uniformizing distorted optical flow patterns in the equirectangular projection. We experimentally show that this results in consistently lower error as opposed to learning from the distorted optical flow.

Comparing to perspective cameras, the all-round 3D measurement of the environment can be done by spherical cameras in a more efficient way. However, the measurement using binocular spherical stereo has two singularity points at the epipoles of each spherical camera, where the measurement result gets extremely sensitive to the error when getting close to the epipoles and along the epipolar directions. This affects the accuracy of 3D reconstruction along with the epipolar directions. A three-way measurement method using three spherical cameras with trinocular spherical stereo setup is proposed in this paper to achieve accurate all-round 3D measurement. The improved accuracy of 3D measurement by the implementation of weighted reprojection error optimization was verified in experiments.

In this paper, we propose a novel motion planning method for off-road unmanned ground vehicles, based on three-dimensional (3-D) terrain map information. Previous studies on the motion planning of a vehicle traveling on rough terrain dealt only with a relatively small environment. Furthermore, unique vehicle characteristics were not considered, and it was also impossible to incorporate regulations, such as maintaining driving speed and suppressing posture change. The proposed method enables vehicles to adaptively generate a path by considering vehicle characteristics and the regulations, in a large-scale environment, with rough terrain. A random sampling based scheme was applied to carrying out global path planning, based on a 3-D environmental model. Experimental results showed that the proposed off-road motion planner could generate an appropriate path, which satisfies vehicle characteristics and predefined regulations.

This paper presents a dense method of real-time registration of RGB-D image pair. So far, we have proposed the “see-through system”, in which multiple images acquired from RGB-D sensors are integrated to present images that is useful for confirming the shape or the positions of objects behind obstacles. However, errors of positional relation of sensors result in see-through images in which some objects are doubled or appear at incorrect positions. It is difficult to align the images because they are captured from distant viewpoints and there are few shared field of view. In the proposed method, positional relation of two RGB-D sensors is corrected using a new IRLS framework, fast and robust minimization strategy.

Conventional techniques for frame-to-frame camera motion estimation rely on tracking a set of sparse feature points. However, images taken from spherical cameras have high distortion which can induce mistakes in feature point tracking, offsetting the advantage of their large fields-of-view. Hence, in this research, we attempt a novel approach of using dense optical flow for distortion-robust spherical camera motion estimation. Dense optical flow incorporates smoothing terms and is free of local outliers. It encodes the camera motion as well as dense 3D information. Our approach decomposes dense optical flow into epipolar geometry and the dense disparity map, and reprojects this disparity map to estimate 6 DoF camera motion. The approach handles spherical image distortion in a natural way. We experimentally demonstrate its accuracy and robustness.

Localization is an important task for mobile service robots in indoor spaces. In this research, we propose a novel technique for indoor localization using a spherical camera. Spherical cameras can obtain a complete view of the surroundings allowing the use of global environmental information. We take advantage of this in order to estimate camera position and the orientation with respect to a known 3D line map of an indoor environment, using a single image. We robustly extract 2D line information from the spherical image via spherical-gradient filtering and match it to 3D line information in the line map. Our method requires no information about the 3D-2D line correspondences. In order to avoid a complicated six degrees of freedom (6 DoF) search for position and orientation, we use a Manhattan world assumption to decompose the line information in the image. The 6 DoF localization process is divided into two phases. First, we estimate the orientation by extracting the three principle directions from the image. Then, the position is estimated by robustly matching the distribution of lines between the image and the 3D model via a spherical Hough representation. This decoupled search can robustly localize a spherical camera using a single image, as we demonstrate experimentally.

In camera-equipped teleoperated robots, it is often tedious for the operator to manage both the viewpoint and the shaky/unstable navigation, leading to disorientation. Our proposal is to create a virtual, freely rotatable camera that is decoupled from the robot's rotation. It is implemented using a complete spherical camera and removing its rotation in-image with a novel algorithm based on aligning the dense spherical optical flow field along the epipolar direction. Finally, any area on the rotation-less image sequence can be undistorted, resulting in the desired decoupled camera. We illustrate the concept by showing the effect on some videos taken from a spherical camera under different robot motions.

This paper introduces a new methodology of robotic hammering inspection for the maintenance of social infrastructures. In particular, the estimation of material defect conditions, such as delamination depth of concrete, is focused upon. Development of an automated diagnosis methodology is necessary for the maintenance of superannuated social infrastructures. The hammering test, which is an efficient inspection method, has attracted considerable attention in the context of automated inspection using robots. In this study, to apply the hammering test to robotic inspection, in which material conditions of infrastructures must be diagnosed in detail, an estimation method of the defect conditions is proposed, and an integration technique of plural classifiers for improving the inspection accuracy is introduced. Furthermore, an inspection system that can decrease the influence of the mechanical running-noise is implemented. Our experimental results using concrete test pieces demonstrate the effectiveness of the proposed method; the accuracy of the defect detection and defect condition estimation was validated.

Indoor robot localization systems using wireless signal measurements have gained popularity in recent years, as wireless Local Area Networks can be found practically everywhere. In this field, a popular approach is the use of fingerprinting techniques, such as Gaussian Processes. In our approach, we improve Gaussian Processes based mapping using path loss models as priors. Path loss models encode information regarding the signal propagation phenomena into the mapping. Our approach first fits training data to a simple path loss model, and then trains a zero-mean Gaussian Process with the mismatches between the models and the data. Signal strength mean predictions are done using both the path loss model and the Gaussian Process output, while variances are calculated by bounding the Gaussian Process variance using the path loss models. Notably, the main improvement generated by our approach is not an enhanced mean value prediction, but rather a better model variance prediction. This translates into better likelihood estimations, leading to higher localization accuracy. Experiments using data acquired in an indoor environment and our approach as the perceptual likelihood of a dual Monte Carlo localization algorithm are used to demonstrate this improvement. Furthermore, this idea can be extrapolated to other fingerprinting techniques and to applications other than wireless-based localization.

We propose a prediction method for lane changes in other vehicles. According to previous research, over 90 % of car crashes are caused by human mistakes, and lane changes are the main factor. Therefore, if an intelligent system can predict a lane change and alarm a driver before another vehicle crosses the center line, this can contribute to reducing the accident rate. The main contribution of this work is to propose a new feature describing the relationship of a vehicle to adjacent vehicles. We represent the new feature using a dynamic characteristic potential field that changes the distribution depending on the relative number of adjacent vehicles. The new feature addresses numerous situations in which lane changes are made. Adding the new feature can be expected to improve prediction performance. We trained the prediction model and evaluated the performance using a real traffic dataset with over 900 lane changes, and we confirmed that the proposed method outperforms previous methods in terms of both accuracy and prediction time.

It is important to understand the mechanism of human standing-up motion to improve the declined physical ability of the elderly people. This study employs the concept of muscle synergies (modular structure of coordinative muscle activation) to understand how humans coordinate their muscles to achieve the standing-up motion. Neuro-musculoskeletal model was developed to represent human body to generate standing-up motion. Using the developed model, forward dynamic simulation was used to analyze how humans utilized the muscle synergies to realize the motion. Results showed that the developed model could generate the standing-up motion with four muscle synergies rather than controlling individual muscles. Moreover, further analysis showed that three different strategies of the standing-up motion could be generated only by changing the start time of the particular muscle synergy.

Three-dimensional (3D) measurement is an important means for robots to acquire information about their environment. Structure from Motion is one of these 3D measurement methods. The 3D reconstruction of objects in the environment can be obtained from pictures captured with single camera in Structure from Motion. Furthermore, the camera motion can be obtained simultaneously. Because of its simplicity, Structure from Motion has been implemented in various ways. However there is an essential problem in that the scale of the measured objects cannot be computed by Structure from Motion. In order to compute the absolute scale, other information is required. However this is difficult for robots in an unknown situation. In this paper, we propose a method that can reconstruct the absolute scale of objects using refraction. Refraction changes the light ray path between the objects and the camera. This method is implemented using only a refractive plate and single camera. The results of simulations show the effectiveness of the proposed method in both air and other media (e.g., water).

We present a novel rough terrain traversability analysis method for mobile robot navigation. We focused on the scenario of mobile robot operation in a disaster environment with limited sensor data. The robot builds a map in real time and analyzes the terrain traversability of its surrounding environment. The proposed method is based on fuzzy inference so that it can handle uncertainties in the sensor data. Two values associated with the terrain traversability, roughness and slope, are calculated from an elevation map built by a laser range finder mounted on the mobile robot. These two values are inputted to the fuzzy inference, and the traversability is analyzed. Based on the traversability output from the fuzzy inference, a vector field histogram (VFH) is generated. The mobile robot course is determined according to the VFH. We demonstrated our algorithm on an artificial environment. The experimental results showed that the mobile robot was able to reach the target position safely while avoiding untraversable areas.

An automated diagnosis methodology is necessary for the maintenance of superannuated social infrastructures. In this context, the hammering test is an efficient inspection method, and it has been widely used because of the resulting accuracy and efficiency of operation. While robotic automation of the hammering inspection method is highly desirable, the development of an automatic diagnostic algorithm that can operate at actual inspection sites is essential. Furthermore, portability of the diagnostic algorithm is also highly desirable. In this study, in order to construct reliable detectors and to improve their portability for the performance of the hammering test, we propose a boosting-based defect detector that is robust against variations in environmental conditions. In particular, we present the construction of a noise-robust classifier with a refinement of the feature values extracted from hammering sounds and an updating rule of template vectors of its evaluation function. Our experimental results in a concrete tunnel demonstrate the effectiveness of the proposed method; the accuracy of the classifier at an actual site and adaptivity to environmental noise are confirmed.

The standing-up motion is one of the most important activities of daily livings. In order to understand the strategy to achieve the standing-up motion, muscle synergy analysis is applied to the measured data during human standing-up motion. In addition, musculoskeletal model which consists of three body segments and nine muscles in lower limb is developed to ensure that the standing-up motion can be generated by muscle synergies. As a result, three muscle synergies have been extracted from the human standing-up motion, and each synergy strongly corresponded to characteristic kinematic events: momentum flexion, momentum transfer, and posture stabilization. Results of forward dynamic simulation show that the standing-up motion can be achieved by controlling time-varying weighting coefficient of three muscle synergies instead of controlling individual nine muscles.

In this paper, novel reinforcement learning method with intrinsic motivation for reproducibility of the past successful experience is presented. The experience is extracted as skill, which is composed of action sequence and abstract knowledge about observed sensor input. Utilizing the collected skills, reproduction of the successful experience is attempted in novel and unknown environment. Consistent exploration and active reduction of search space are realized by learning with intrinsic motivation for reproducibility of experience. Simulation experiments in grid world demonstrate that proposed method significantly accelerate speed of learning.

Although standing-up motion is an important activity of daily living, it remains unclear how people perform the motion in different situations. As described in this paper, muscle synergy analysis is applied to standing-up motions performed at different circumstances, such as two different heights and at three different speeds. Results elucidated three invariant groups of synchronized muscle activations: The first synergy pulls the ankle and raises the hip. The second synergy extends the upper body. The third synergy stabilizes posture. Results also show that people controlled the activation coefficient of each synergy differently during all motions. The slower the standing-up motion is, the longer each synergy activates to adapt to the slower motion speed. Results of this study show that people use the same group of synchronized muscle activation and only control the activation coefficient to achieve adaptive standing-up motion.

As the population continues to age, the number of elderly people requiring healthcare is increasing. In order to improve their physical function, they need to get physical training. There are activities which require integrated arm movements and lower body movements. However there are no quantitative testing methods of the degree of recovery for the coordination between arm movements and lower body movements. In this study, we focus on dart throwing motion as arm movements in lower body movements and suggest the quantitative evaluation of the coordination between arm movements and lower body movements in dart throwing motion. Normalized correlation coefficient (NCC) between arm and lower body was computed at different throwing distances. In addition the standard deviation of the NCC was computed in order to investigate the stability of the joint correlation evaluation. This analysis shows that the correlation between elbow and ankle, or between elbow and knee, are increased at throwing long distance. We suggest that the NCC between elbow angle and right knee angle may be used for the evaluation of the joint correlation between arm movements and lower body movements in dart throwing motion.

In this paper, we propose a 3-D measurement method of underwater objects using a fish-eye stereo camera. Sensing in aquatic environments is important to maintain underwater structures and research aquatic lives. A 3-D measurement method of objects in water using a fish-eye stereo camera enables underwater robots to execute its task safely with a wide field of view. However, images taken by fish-eye camera have large distortion following projection model. Also, sensing in aquatic environments meets the difficulty that image distortion occurs by refraction of light due to the difference of refractive indices of air, watertight container and water. As to these problems, we introduce a method to correct fish-eye image and a ray tracing method to remove the effect of refraction. Experimental results show the effectiveness of the proposed method.

In this paper, we propose an efficient estimation method of an omnidirectional camera movement. The proposed method is based on Structure from Motion utilizing a constraint of parallel lines. In an environment having manmade structures, parallel lines can be extracted from an omnidirectional image easily and constantly, because of its wide field of view. Parallel lines provides a valuable constraint for camera movement estimation. The proposed method can estimate 3-D camera movements by solving one degree of freedom problem three times without regard to the number of viewpoints. Experimental results show the effectiveness of our proposed method.

View occluders influence the image quality of a subject when occluders exist between a camera and a subject. For example, a blurred fence image interrupts a subject when an image of a scene is captured by a camera through a fence. In this paper, we propose an automatic removal method of foreground occluders from images using multiple focusing. Our method automatically detects foreground occluder regions by using two images with and without flashlight. The influence of foreground occluders is estimated and overlapping effects of foreground occluders are removed by using multiple focus images. Experimental results show the effectiveness of the proposed method.

In order for robots to safely move in human-robot coexisting environment, they must be able to predict their surrounding people's behavior. In this study, a pedestrian behavior model that produces humanlike behavior was developed. The model takes into account the pedestrian's intention. Based on the intention, the model pedestrian sets its subgoal and moves toward the subgoal according to virtual forces affected by other pedestrian and environment. The proposed model was verified through pedestrian observation experiments.

Recently healthcare of the elderly people has become a serious issue in medical and rehabilitation areas. In order to know their functional mobility and provide sufficient medical treatment, it is important to measure their body state precisely and objectively. Therefore we developed a wearable and wireless sensor of gyroscope and accelerometer (PocketIMU2) as an easy and precise measurement of human motions. In the sensor, we employed a small and high accurate LiNbO3 crystal to achieve joint angle computation with simple integration of angular velocity. In the current paper, we evaluate the accuracy of the sensor in two important basic motion, such as a walking and sit-to-stand motions. Computed joint angles of ankle, knee, and hip are compared to the reference data measured from a optical motion capture system in term of coefficients of correlation and root mean square error. As a result, coefficient of correlation showed very high value for all joint angles, and root mean square error was adequately small. This strongly supports the usage of our developed gyroscope and accelerometer sensor for monitoring human body movement for medical usage.

To achieve a balanced walking for a humanoid, it is necessary to estimate the dynamic stability of the system. However, in a small size humanoid with restricted system resource, it is hard to satisfy the performance level desired by dynamics analysis. Therefore, in this paper, we propose the feasible methods to generate gait pattern and stabilize walking based on coupled oscillators which have a clear correlation between oscillator parameters and system dynamics without a real time ZMP calculation. The proposed method was tested on the open humanoid platform DARwIn-OP for the evaluation, and the result showed that a real time gait pattern generation and stabilization were realized.

The IEEE/RSJ 2010 International Conference on Intelligent Robots and Systems (IROS 2010) will be held on October 18–22, 2010 at the Taipei International Conference Center (TICC), Taipei, Taiwan. IROS 2010 is the 23rd year of the conference, since its inception in 1988. The conference theme is “Intelligent Robotics in the Next Transition” The theme of IROS 2010 is chosen such that the conference would continue reflect the ever increasing interest in interactions, co-works, and co-existence of robots with human, cognitive robotics and other forms of intelligent machines and systems, with the goal for improving the better quality of human life. IROS 2010 continues to attract a large number of participants, paper submissions, and accepted papers whose quality remains high. IROS 2010 had 1798 papers submitted from more than 50 countries. We will have 828 papers for oral presentation and 144 for interactive discussions at the conference after first going through a rigorous review process, then by second going through a quality control process conducted at the Program Committee (PC) meeting in May 2010. The technical Program of IROS 2010 consists of 1 Keynote Talk, 3 Plenary Talks, one day Robot Industry Forum, half day Robotics Education Forum, 140 technical sessions in 14 tracks, 17 workshops/tutorials, and 16 video presentations, 14 exhibitions, and Robot Competitions.

In order for service robots to safely coexist with humans, collision avoidance with humans is the most important issue. On the other hand, working efficiencies are also important and cannot be ignored. In this paper, we propose a method to estimate a pedestrian's behavior. Based on the estimation, we realize smooth collision avoidances between a robot and a human. A robot detects pedestrians by using a laser range finder and tracks them by a Kalman filter. We apply the social force model to the observed trajectory for a determination whether the pedestrian intends to avoid a collision with the robot or not. The robot selects an appropriate behavior based on the estimation results. We conducted experiments that a robot and a person pass each other. Through the experiments, the usefulness of the proposed method was demonstrated.

For effective trajectory formation of robots, we propose a method to extract candidate points of the destination of walking people from their walking trajectories. The method is useful for setting the candidate points of the destination automatically which is necessary to estimate the pedestrians' destination. We assume that potential fields influence pedestrians. After estimating parameters of the potential fields and evaluating their minimum points, we extracted candidate points of the respective destinations of pedestrians. To estimate the parameters of the potential fields accurately, we classified data of pedestrian' walking trajectories into groups. Thereby, the trajectories leading to the same destination are gathered in one group. We designed an evaluation function that indicates the conformity between the trajectory and the destination to classify the data correctly. Results obtained from experiments in an actual environment to verify the availability of the proposed method show that appropriate candidate points of the destination are extracted.

Division of Labor Control is advanced function for distributed autonomous robotic systems. Many studies focussing division of labor control inspired biological phenomenon have been reported. However, the optimality is not discussed because decentralized control is typically complicated. In this study, we propose the division of labor control method for robot group that enable adaptively select whether homogeneous state or heterogeneous state against working conditions and address the optimality by mathematical analysis. To evaluate the effectiveness of the proposed method, the computer simulations are carried out and we confirm that robot group implemented the proposed method inevitably organize the division of labor state with group performance improvement.

Near-infrared spectroscopy (NIRS) is a non-invasive technology for measuring brain activity. Recently, the number of research papers on brain machine interface (BMI) based on NIRS technology is increasing. NIRS is a safe and convenient technique but its measurement results are unstable. To improve reliability of NIRS-based BMI, methods to extract stable data from NIRS signals are necessary. This paper describes a reliable NIRS-based BMI system we have developed. The feasibility of the method was demonstrated through generating motion of a humanoid robot.

Prevention of dementia is a crucial issue in this aged society. We propose coimagination method for prevention of dementia through supporting interactive communication with images. Coimagination method aims to activate three cognitive functions: episode memory, division of attention, and planning function, which decline at mild cognitive impairment (MCI). Participants of the coimagination program bring images according to the theme and communicate with them. The objective of the program is to make participants to focus on present and future rather than past, which is major difference between coimagination and reminiscence. We measure frequency of comments by others for each participant in order to evaluate interactivity of conversation. They take memory task whether they remember the owner or theme of images after the series of sessions. We held coimagination program successfully at the welfare institution for elderly people in Kashiwa city, Japan. Each session was held one hour per week for five times. The result of the task indicates that the participants showed empathy with each other. The effectiveness of the proposed method was validated through the experiment.

In this paper, we propose the technique classifying background area automatically by calculating the color variance in the RGB space of the picture. By this technique, the background classification robust to change of the background brightness and the background color was enabled. Moreover, we show the effectiveness of this technique by performing particle detection and verifying the result of counting. We aim at development of the particle detector which performs particle counting automatically about the “dispersion staining method” which is one of the processes of asbestos analysis. We used the pictures taken by phase contrast microscopy used for a “dispersion staining method.”

This paper presents open brain simulator, which estimates the neural state of human through external measurement for the purpose of improving motor and social skills. Macroscopic anatomical nervous systems model was built which can be connected to the musculoskeletal model. Microscopic anatomical and physiological neural models were interfaced to the macroscopic model. Neural activities of somatosensory area and Purkinje cell were calculated from motion capture data. The simulator provides technical infrastructure for human biomechatronics, which is promising for the novel diagnosis of neurological disorders and their treatments through medication and movement therapy, and for motor learning support system supporting acquisition of motor skill considering neural mechanism.

This paper presents a method for designing classifier to automate an evaluation process of protein crystallization growth states. The classifier is designed by binary decision tree and its classification sequence is decided in descending order of classification accuracy. Furthermore, for applying to automated crystallization system, some functions reducing mis-classification are appended to the classifier. The performance of this method has been evaluated by comparing classification result by using proposed method with the result by manual classification

In our current research, we are developing a holonomic mobile system which is capable of running over the step. This system realizes omni-directional motion on flat floor using special wheels and passes over non-flat ground in forward or backward direction using the passive suspension mechanism. In order to realize the high mobile performance during step climbing, it is required to reduce wheel slippage for maximizing wheel traction. This paper proposes a new plural wheel control method based on wheel slip estimation. Our key idea is estimation of wheel slippage comparing with the loads and rotation velocities of all actuated wheels, and using this result for wheel control of the vehicle for reducing wheel slippage. The controller can adjust control tractions of plural wheels when the wheel begins to slip and can improve the mobile performance of the vehicle. The performance of our proposed control scheme is experimented by computer simulations and experiments

We are conducting research and development in technology for unifying the information of a large number of intelligent sensors distributed in a disaster-stricken area. Operating effectively as a system in real time, network household electric appliances, a personal digital assistant (PDA), a robot, etc. provided information and an action plan for a stricken area's situation. In this paper we introduce ubiquitous device "intelligent data carriers for rescue (IDC-R)" and formation of a dynamic network, and development of a means of disaster information communication (a group of people, a robot, an adhoc network, a home network, etc.). Target experiment for actual proof is also introduced

In our current research, we are developing a holonomic mobile system which is capable of running over the step. This system realizes omni-directional motion on flat floor using special wheels and passes over non-flat ground in forward or backward direction using the passive suspension mechanism. The vehicle changes its body shape according to ground condition when it runs over the rough terrain. This paper proposes a new wheel control method according to the change of the vehicle's body shape for passing over the step. Our key idea is the modification of its kinematic model referring to the body configuration dynamically and using this model for wheel control of the vehicle. The controller can adjust the wheel control references when the vehicle passes over the rough terrain changing the body shape and can reduce the slippage and the rotation error of the wheels. The performance of our proposed method is experimented by computer simulations and experiments.

Presents the welcome message from the conference proceedings.

This paper proposes a technique of the extended template matching that detects the object with two-dimensional standard shape under the outdoor environment. Under the outdoor environment, the intensity distribution of the object changes variously by the change in the lighting condition and the change in the surface properties of the object. On the other hand, the conventional template matching assumes the intensity distribution of the object does not change. Therefore, it proposes the technique for evaluating plural similarities calculated with partial and whole templates by the linear discriminant function. The effectiveness of the proposed technique was confirmed by the detection experiment of the corner fitting of the container under the outdoor environment.

In recent years, several projects have advanced the research and development related to the automation of the protein crystallization process. However, the evaluation of crystallization states has not been completely automated yet. In the usual crystallization process, the researchers evaluate the crystallization growth states of the protein solution samples based on one's visual impressions and assign them a score over and over again. Then it is required to make the work more efficient. The method presented here automates such evaluations. This method attempts to categorize the individual crystallization droplet images into five classes from A to E, based on their crystallization states. The algorithm is comprised of pre-processing, feature extraction from images using texture analysis and a categorization process utilizing linear discriminant analysis. The performance of this method has been tested on our experiments using actual protein solution images.

In this paper, we propose a new path and viewpoint planning method for a mobile robot with multiple observation strategies. When a mobile robot works in the constructed environments such as indoor, it is very effective and reasonable to attach landmarks on the environment for the vision-based navigation. In that case, it is important for the robot to decide its motion automatically. Therefore, we propose a motion planning method that optimizes the efficiency of the task, the danger of colliding with obstacles, and the accuracy and the ease of the observation according to the situation and the performance of the robots. We also introduce multiple landmark-observation strategies to optimize the motion depending on the number and the configuration of visible landmarks in each place.

In this study, we propose a new holonomic mobile mechanism which is capable of step-climbing. This mechanism realizes to move in any direction on the flat floor and pass over steps and slopes in one (fore and hind) direction. The vehicle equips with seven omni-directional wheels with cylindrical free rollers and a passive body axis that provides to change the shape of the body on the rough terrain. There is no need for additional actuators or sensors for the passive body axis. In our previous work, we constructed a prototype mobile mechanism and analyzed its kinematic characteristics. In this paper, we design a new control scheme and improve the mechanism for passing over the steps more stably, The performance of the prototype system is verified through experiments and computer simulations.

In this paper, region exploration path planning algorithm is proposed. In order for a mobile robot to perform this task, appropriate measures with the shape of the working environment, which may be intricate or curved, is necessary. In addition, a robot must be able to react and be flexible when confronted with obstacles. With this algorithm, these challenges can be met by approximately expressing the working environment in grid points and regenerating the path using one that was planned beforehand. Simulations are used to demonstrate proposed exploration path planning and re-planning algorithm.

In this paper, we propose cooperative exploration method for mobile robots in a working area. The method will generate and share the minimal-cost path for each mobile robot. Each mobile robot goes through several observation points to accomplish the exploration. The observation points should be arranged at a fixed distance from at least on corresponding observation point. In addition, the number of observation points and the lengths of paths for each mobile robot are to be minimized. The proposed method should have the efficiency in computational cost concurrently with the adaptability to dynamic environmental changes.

This is a review paper on Mobiligence research. In order to solve the mechanism of intelligence with which higher organisms can behave even in unknown environment, and to discover design principle of artificial systems that have such intelligence, a new concept called Mobiligence is introduced based on assumption that the intelligence must be emerged through the interaction of the system with the environment caused by its mobility. The research on the Mobiligence is characterized by embodied plasticity, co-embodiment with environment, and internal generation of abductive information. The framework of the research on Mobiligence is presented in this paper as well as the originality of the research.

An inevitable collision state for a robotic system can be defined as a state for which, no matter what the future trajectory followed by the system is, a collision with an obstacle eventually occurs. An inevitable collision state takes into account both the dynamics of the system and the obstacles, fixed or moving. The main contribution of this paper is to lay down and explore this novel concept (and the companion concept of inevitable collision obstacle). Formal definitions of the inevitable collision states and obstacles are given. Properties fundamental for their characterisation are established. This concept is very general and can be useful both for navigation and motion planning purposes (for its own safety, a robotic system should never find itself in an inevitable collision state). The interest of this concept is illustrated by a safe motion planning example.

This paper offers a proposal for an algorithm of controlling a mobile robot that searches for and rearranges objects with unknown locations and shape. In this paper, we divide the task into two parts: exploration task and rearrangement task. The algorithms for each part of the task are presented with respect to the effectiveness of the path length and computational cost. Additionally integration algorithm that effectively combines exploration and rearrangement is presented. Experiments with a real robot are conducted to demonstrate the effectiveness of the proposed algorithm.

In this paper, we propose a motion-planning method of multiple mobile robots for cooperative transportation of a large object in a three-dimensional environment. This task has various kinds of problems, such as obstacle avoidance and stable manipulation. All of these problems cannot be solved at once, since it would result in a dramatic increase of the computational time. Accordingly, we divided the motion planner into a global path planner and a local manipulation planner, designed them, and integrated them. The aim was to integrate a gross motion planner and a fine motion planner. Concerning the global path planner, we reduced the dimensions of the configuration space (C-space) using the feature of transportation by mobile robots. We used the potential field to find the solution by searching in this smaller-dimension reconstructed C-space. In the global path planner, the constraints of the object manipulation are considered as the cost function and the heuristic function in the A/sup */ search. For the local manipulation planner, we developed a manipulation technique, which is suitable for mobile robots by position control. We computed the conditions in which the object becomes unstable during manipulation and generated each robot's motion, considering the robots' motion errors and indefinite factors from the planning stage. We verified the effectiveness of our proposed motion planning method through simulations.

We propose a decentralized motion control algorithm of multiple mobile robots transporting a single object in cooperation with a human based on a path generated from an environment. Each mobile robot is controlled as if it had caster-like dynamics around a representative point attached to the object and its representative point moves along the path based on the intentional force applied by a human. The proposed decentralized control algorithm is experimentally applied to three omni-directional mobile robots referred to as DR Helper. Experimental results illustrate the validity of the proposed control algorithm.

Considering that intelligent robotic systems work in a real environment, it is important to equip the function of self-condition recognition in such systems. From this point of view, we discuss the way to diagnose such systems and construct a self-diagnostic system for an autonomous mobile robot. The system must have a self-contained type diagnosing system. In our previous work (2001), we constructed a state sensory system and examined the coping behaviors to self-condition. In this paper, we describe an extension of the state sensory system and diagnosing experiment using an autonomous mobile robot. The prototype of our sensory system consists of internal and external sensors. We show the experimental results using an omnidirectional mobile robot and confirm the effectiveness of our proposed system.

In this paper, we focus on the procedure whereby an autonomous multiple robot system constructs feasible paths. We formulate optimal solutions in the following two cases; (i) a robot makes the decision by itself, (ii) an environment (or specific locations in an environment) gathers information and makes suggestions. We also propose learning algorithms for appropriate path construction in both cases. Then we discuss the efficiency of the two cases based on optimal solutions and learning results of simulations.

This paper deals with a task-assignment architecture for cooperative transport by multiple mobile robots in an unknown static environment. The architecture should satisfy three features: deal with a variety of tasks in time and space, deal with a large number of tasks compared with the number of robots, and decide the behavior in real time. The authors propose the following approach: we consider the unit of task (task instance) as the job that should be done in a short time by one robot. Based on the environmental information, task instances are dynamically generated using task templates. The priority of task instances is evaluated dynamically based on the number of robots and the configuration in the workspace. In addition, we avoid generating too many task instances by suppressing object motion. The main part of the architecture consists of two real-time planners: a priority-based task-assignment planner solved by using the linear programming method, and motion planners based on short-time estimation. The effectiveness of the proposed architecture is verified by a cooperative transport simulation in an unknown environment.

We propose a decentralized motion control algorithm of multiple mobile robots transporting a single object in cooperation with a human based on a path generated from an environment. Each mobile robot is controlled as if it has a wheel with a free rotational joint and moves along the path based on the intentional force applied by a human. The proposed decentralized control algorithm is experimentally applied to the omni-directional mobile robots referred to as DR Helpers. Experimental results illustrate the validity of the proposed control algorithm.

We propose a decentralized control system of multiple mobile robots transporting a single object in coordination without using force/torque sensors. In this system, each robot is controlled as if it has a specified impedance dynamics without using a force/torque sensor. The robots transport an object in coordination using a leader-follower type control algorithm. The effect of parameter identification errors is discussed and a method to reduce it is proposed. The proposed control algorithm is experimentally applied to two mobile robots, and the experimental results illustrate the validity of the proposed control algorithm.

This paper describes our research work towards the development of an optical guidance system for multiple mobile robots in an indoor environment. The guidance system operates with an environment model, communicates with mobile robots and indicates their target positions by means of a light projection from a laser pointer onto the ground. Processing the image data from a CCD color camera mounted on the mobile robot allows it to detect the laser light beacon on the ground and estimate its relative coordinates. The robot's control system ensures the accurate motion of the robot to the indicated target position. The guidance system subsequently indicates target positions corresponding to a desired route for a specified mobile robot in the fleet. The concept of the optical guidance system, its implementation and experimental results obtained are discussed.

We propose a decentralized control algorithm of multiple robot helpers, called DR Helpers, which have been developed for manipulating a large object in cooperation with humans. In this algorithm, each robot is controlled as if it has a caster-like dynamics around a representative point attached to the object, and manipulation of a large or long object in cooperation with a human can be realized easily. The proposed control algorithm is experimentally applied to two DR Helpers and experimental results illustrate the validity of the proposed control algorithm.

We propose information assistance for a victim search system in a rescue activity. We have developed an intelligent data carrier system (IDC) that provides the environment with information storage, sensing, and processing functionality. Each IDC unit has its own CPU, memory and radio communication module. We have incorporated the auditory functionality into IDC units and applied them to rescue activity support. IDC units attached in buildings are activated by broadcast of a wake-up signal, and then they repeat calling to victims and recording their answers independently. Rescue parties perform effectively by seeking records of victims. We have also built an autonomous dispatch system of autonomous robots based on the information obtained by the IDC system.

In this paper, we propose a device for cooperation of multiple autonomous mobile robots. A device which enables local communication, named "intelligent data carrier (IDC)", is proposed. In this paper, we propose an algorithm to improve efficiency of iterative transportation tasks by autonomous mobile robots. An autonomous mobile robot gives name of its latest visited destination to an IDC which is located on a junction in a work area. An IDC summarizes the information from robots and tells a direction that it effects best way to a destination which a robot wants to go. By the proposed algorithm and system, mobile robots construct information of navigation and improve the efficiency of a task automatically. We verified the effect of the proposed algorithm by simulations and experiments.

This paper deals with a task-assignment architecture for cooperative transport by multiple mobile robots in unknown environment. The planner should satisfy three features: to deal with the variety of tasks in time and space; do deal with a large amount of tasks; and to decide behavior in real-time. The authors (1999) have proposed the following approach: 1) based on sensor information, tasks will be dynamically generated using task templates; 2) generation of task is tuned in quality by feeding back executed result and in the number by action suppression; 3) main part of the architecture consists of two real-time planner: priority-based task-assignment planner solved by using a linear programming method and motion planners based on short-time estimation. In this paper, we implement our task-assignment architecture on a real robot system and verify the effectiveness of the proposed method by cooperative transport experiment in unknown environment.

This paper deals with the motion generation approach developed for our omnidirectional mobile robots. Our objective, overall control architecture of the mobile robot, and its motion controller are considered. The key idea and advantages of the proposed motion generation approach are discussed. The approach is implemented and tested on the omnidirectional mobile robot and the experimental results obtained are presented. The operation of the control system is illustrated by a video on the remote control of the robot and the visually-coupled motion of the two robots.

We propose a motion planning method for cooperative transportation of a large object by multiple mobile robots in a 3 dimensional environment. This task has various kinds of problems, such as path planning, manipulation and so on. All of these problems can't be solved at once, since computational time is exploded. Accordingly, we divide a motion planner into a local manipulation planner and a global path (motion) planner, and design these two planners respectively, and we integrate two planners. Namely, we aim at integrating a gross motion planner and a fine motion planner. As to the local manipulation planner, we build a manipulation technique, which is suitable for mobile robots by position control. We compute conditions, in which the object becomes unstable during manipulation, and generate each robot's motion considering the robots' motion errors and indefinite factors from the planning stage. As to the global path planner, we reduce the dimensions of the configuration space (C-space) using the feature of transportation by mobile robots. We can find a solution with searching in this smaller dimensional C-space using the potential field defined in the C-space, and constraints of the object manipulation are considered as the potential function. We verify the effectiveness of our proposed motion planning method through simulations and experiments.

We propose a decentralized control algorithm of multiple mobile robots transporting a single object in coordination. In this algorithm, each robot is controlled as if it has a caster-like mechanism and transportation of a single object by multiple mobile robots is realized without using the geometric relations among robots. The proposed control algorithm is experimentally applied to two mobile robots. Experimental results illustrate the validity of the proposed control algorithm.

We propose a behavior learning method of a mobile robot using visual information. The proposed system consists of two networks for the state expression and the behavior generation. The state expression network, which is based on self-organizing algorithm, categorizes the robot's states. Also, the behavior generation network acquires the robot's behaviors using the output of the state expression network. Here, for training the behavior generation network, we utilize the actor-critic method as a kind of unsupervised learning scheme. We report some experimental results using the mobile robot in real environment. The mobile robot generates adaptive behaviors utilizing visual information.

In, this paper, we discuss a problem relating to the force/moment transformation for the handling of a large object by multiple mobile robots in coordination. We propose a control algorithm using geometrical constraints among the grasping points and the representative point of the object, which reduce the effect of noise amplified by force/moment transformation. We extend this algorithm to the decentralized control algorithm of multiple robots handling an object in coordination. The proposed control algorithm is experimentally applied to the mobile robots. Experimental results illustrate the validity of the proposed control algorithm.

This paper discusses the problems in teleoperation systems for mobile robots and the use of a virtual world in these systems. In order to achieve smooth operation of the mobile robot via the communication link, we should consider the delays in data transfer. To compensate the incomplete data sets, virtual images can be generated by computer graphics when the information of the working environment can be acquired beforehand. We construct a virtual world teleoperation system. The performance of the system is examined through experiments with an actual mobile robot. It is shown that the virtual robot can be operated by a human operator in almost the same manner as a teleoperated real robot.

We have proposed the collision avoidance methods in a multirobot system based on the information exchanged by the "LOCISS: Locally Communicable Infrared Sensory System", which is developed by the authors. One of the problems in the LOCISS based methods is that the number of situations which should be considered increases very much when the number of the robots and stationary obstacles in the working environment increases. In order to reduce the required computational power and memory capacity for such a large number of situations, we propose, in this paper, a multilayered reinforcement learning scheme to acquire appropriate collision avoidance behaviors. The feasibility and the performance of the proposed scheme is examined through the experiment using actual mobile robots.

This paper presents a novel concept of self-organizing collective robots with morphogenesis in a vertical plane. It is potentially applicable to autonomous mobile robots. For physical reconfiguration of a swarm of robots against gravity, new types of mechanisms and control strategies are proposed and demonstrated. Prototype robots have been fabricated in order to confirm the basic feasibility of the mechanisms. Each robot is composed of a body and a pair of arms. The body can be regarded as a cube with edge length of 90 mm, and is equipped with permanent magnets for bonding with another robot. The arms change the bonding configuration by rotating and sliding motions. As for the control strategies, we proposed the algorithms which can locally generate specific global formations of robots, with minimum interactions between neighboring robots. The overall scheme is similar to cellular automata. The control algorithms proposed have been tested by simulations.

We propose an algorithm for unforeseen obstacle avoidance by an autonomous sweeping robot. Sweeping means a motion that a robot covers a 2-dimensional area. When a mobile robot finds an unforeseen obstacle, it not only has to avoid collision but also sweep whole area including around the obstacle. In the proposed algorithm, a robot determines local area where path for sweeping should be arranged according to its limited sensing ability. By connecting the paths inside and outside the local area, the robot obtains a new total path. The local path re-planning saves heavy calculation to generate whole path again at online.

In this paper, we discuss autonomous navigation in multirobot environment based on local communication. To carry out cooperative tasks using multiple autonomous robots, navigation systems should be implemented to the robots because mutual alignment of their positions and collision avoidance to each other become important. Therefore, it seems a collision avoidance and a self-localization are indispensable as elemental techniques for the navigation. Although these techniques need to recognize situation of robots surrounding environment, conventional devices to acquire information around them can not work sufficiently in such environment. For this problem, we propose techniques based on the local communication and the navigation system by combining them for multirobot environment. It is shown that robots can arrive their goal precisely without collision among them through proposed navigation systems.

Using a model based on the omni-directional robots developed at the Institute of Physical and Chemical Research (RIKEN), we discuss the possibility of automatically generating a collision avoidance algorithm for autonomous mobile robots. To this end, we show that an effective collision avoidance algorithm for two robots can be generated by a very simple learning algorithm that simulates a naive human trial-and-error learning process, using only the robots' sensor outputs and a suitable reward function, where the exact form of the reward function is also learned autonomously by the robots. We also discuss how a robot can use its "experience" gained in a simple environment to adjust itself to a more complex environment, by automatically generating a collision avoidance algorithm for a three-robot situation utilizing a reduced state space resulting from the learning process for the case of two robots. The results of computer simulation and the experiments conducted at RIKEN using physical robots demonstrate the effectiveness of the collision avoidance algorithms generated and our learning-based approach.

In this paper, we propose a new mobile robot architecture with a body force sensor. The body force sensor is the force/torque sensor which is located between the drive mechanism and the body of the mobile robot. The use of the body force sensor almost realizes the collocation of sensors and actuators, and makes the whole body of the mobile robot sensitive to external force/moment. Decentralized motion control algorithm for handling a single object by multiple robots in coordination is implemented in each mobile robot and stable handling of an object is realized by multiple mobile robots with the body force sensor. Experimental results illustrate the validity of the proposed architecture.

We discuss adaptive behavior acquisition for collision avoidance among multiple autonomous mobile robots which are equipped with the locally communicable infrared sensory system (LOCISS). The LOCISS is a local sensing device for collision avoidance by which robots can detect other robots and obstacles and discriminate them by exchanging relevant information. We (1996) reported previously a collision avoidance method between two robots based on the predetermined rules using LOCISS. It is, however, difficult to realize collision avoidance among three or more robots by the predetermined rules only because situations around the robots become more complicated as the number of robots increases. Thus, it is desirable for the robots to have an adaptive capability for acquisition of the behaviors to avoid collision with other robots and obstacles. To acquire the adaptive behavior, the reinforcement learning is introduced in this paper. It is shown that appropriate behaviors for collision avoidance can be successfully acquired through the proposed learning process.

Collision avoidance is an essential problem for applications of multiple mobile robots. The authors have proposed new sensor system called LOCISS (Locally Communicable Infrared Sensory System) to detect robots and obstacles using infrared local communication. Robots mounting LOCISS can exchange useful information for collision avoidance such as speed and moving direction. In this paper, a set of rules for collision avoidance among multiple autonomous mobile robots using LOCISS is proposed. Each robot mutually carries out collision avoidance using the proposed rule. The validity of this set of rules is shown through a series of experiments using mobile robots.

A concept of mutual transportation by cooperative multiple mobile robots is proposed in this paper, which enables robots to process complicated or heavy-duty tasks, and overcome the large obstruction in an environment. A method of mutual transportation utilizing forklift mechanisms is introduced with discussion on the lifting process and moving process. Cooperative mobile robots including a holonomic omni-directional platform and a forklift mechanism are developed. Finally, as an example of applications of mutual transportation, step-climbing motion is presented, and the motion realized by the developed mobile robots is shown.

This paper proposes a new strategy for knowledge sharing among multiple autonomous mobile robots using newly developed intelligent data carriers (IDCs) to make the multi-robot system more flexible and robust against complex environmental conditions and, to reduce the amount of global communication for mutual exchange of information which is useful locally in a limited area. The IDC is an immobile but portable device (data carrier) for information storage and management with a unique mushroom-like structure to be easily handled by a forklift mechanism which can be mounted on actual mobile robots. A robot can place a unit at a specific location in its working environment and write locally relevant knowledge into the unit. Other robots can share the knowledge which should indicate the existence of specific instruments, objects or topographies, e.g., dead ends, drop-offs, steps, etc., by reading from the placed IDC units. The handling method of the IDC unit is examined through the experiment using an omni-directional mobile robot. It is shown that the robot succeeded in placing the unit at an appropriate location to share the information indicating that a dead end exists.

In this paper, we propose an off-line planning algorithm for cooperative tasks of multiple mobile robots. Sweeping means a motion that a robot covers a 2-dimensional area by its effector. Sweeping of a whole work area is fundamental and essential task of mobile robots. It is more effective if a robot can move an obstacle during a sweeping task as if we clean our room with relocating chairs. We can also consider that this is the simplest task which includes both point-to-point motion and sweeping. We model sweeping and relocation, and propose an algorithm to find appropriate path for robot and way of relocation of each obstacle. We apply the LT graph to solve the problem, which describes both motions of robots and blockades by obstacles in path-time space. We verify the efficiency of the algorithm through simulations and experiments.

A new driving mechanism for holonomic omnidirectional mobile robots is designed, which enables 3 DOF motion control by three correspondent actuators in a decoupled manner with no redundancy. A prototype of the omnidirectional mobile robot with the driving mechanism is developed including a parallel link suspension mechanism. The kinematics of the omnidirectional mobile robot is also analyzed, and simulation for velocity control of the robot is performed by a method for velocity modulation with interpolation to achieve the given target position and velocity.

Distributed autonomous robotic systems which are composed of multiple robotic agents have been attracted the attention of many researchers as a new strategy for flexible and robust robotic systems. For cooperative multiple mobile robots in distributed autonomous robotic systems, recognition of the dynamic environment is required. In this paper, a new sensory system with local communication functionality is developed utilizing infra-red devices. Based on the discussion on sensing and communication for cooperative multiple mobile robots, requirements for the sensory system are derived. Then, the infra-red sensory system is designed and developed taking account of the required function. This system enables not only detection of collisions against obstacles or other robots but also local communication between robots. This system can also detect interference of plural infra-red signals. Finally the developed system as introduced, and its performance obtained as a results of experiments is shown.

This paper first discusses the relation between the human operator and the decentralized autonomous robotic system. The operator relates himself to the system loosely in the decentralized autonomous system. The authors position the operator as a problem solver and a monitor of the system. The human operator is regarded as an agent in the decentralized autonomous robotic system. Then strategies of communication between the human operator and agents are discussed. The authors propose explicit and implicit communication strategies to monitor the system, and several monitoring methods to implement them: time-based and event-based monitoring for explicit communication and eavesdropping messages for implicit communication. The authors compare the monitoring methods in order to ascertain how much information the human operator can gather in each method using simulation. Finally, the characteristics of each monitoring method are analyzed.

As a flexible and robust intelligent robot system, we have been developing an autonomous and decentralized robot system called ACTRESS, which is composed of multiple robotic agents who have various kinds of functionalities. In order to achieve a given mission in multirobot environment, cooperative behaviors by autonomous agents are essential. In this paper, we focus on the case that multiple robots execute a task cooperatively with functional complement, especially for sensing function. We discuss cooperation of multiple robots using communication in a multi-agent robotic system, and propose a method of functional complement by multiple agents, including group organization and cooperative motion control. Finally, we show the experimental result to verify the proposed method.>

The paper describes the human interface system in the autonomous and decentralized multi-agent robotic system ACTRESS. The human interface system coordinates the interaction between the human operator and robotic agents, monitors system's activities and presents them do the operator. The operator can send commands to individual robotic agents or a group of agents through the human interface. He can also intervene in the system based on monitored system status if and when necessary. The human interface also acts as a window through which the robotic agents contact the operator for judgment and advice. The paper presents the architecture of the human interface system. The architecture was designed so as to allow the operator to interact with the system through consistent interface in the real environment and in the simulator environment, in which robotic agents' programs are developed.>

As a flexible and robust intelligent robot system, we have been developing an autonomous and decentralized robot system called ACTRESS, which is composed of multiple robotic agents. In this paper, introducing an idea of cooperative action with group organization and a strategy for cooperative task processing using communication, a method for team organization by efficient negotiation is presented. The efficient negotiation is realized by a newly developed communication functionality called groupcast and a learning mechanism utilizing historical records on past negotiation. The negotiation procedures for team organization are implemented, and as a result of simulation experiments, the efficiency of the learning mechanism is verified.>

The authors describe the basic design of Fun-ARM, and discuss the strategy of the fault tolerance on distributed manipulator. The fault model is described, and assumptions in the system are discussed. The methods to realize fault tolerance are proposed, including methods for fault-detection, fault-localization and fault-containment.

This paper addresses a method for synchronizing motion by multiple autonomous mobile robots using radio communication. The autonomous and decentralized robot system, ACTRESS, is a multi-agent robotic system including multiple autonomous mobile robots. The autonomous mobile robots are developed so as not only to act individually, but also to cooperate with the other robots if they cannot accomplish the task alone. The synchronized motion is sometimes required to execute such cooperative tasks. Every agent of ACTRESS is equipped with the communication system, which enables a message passing between any robotic agents. The prototype ACTRESS system is introduced, and cooperation among multiple robotic agents is discussed. A method for synchronization using the communication system is proposed. The method is applied to the prototype system, and side-by-side motion on the actual mobile robots is implemented as a example of synchronized motion.

An autonomous and decentralized system called ACTRESS (ACTor based Robot and Equipment Synthetic System) is being under development as an intelligent robot system which can execute high level tasks. ACTRESS is composed of multiple robotic agents. Each robotic agent is called robotor. A robotor may be a robot, a computing system and any other equip- ments. While researching in this field, many research issues need to have solutions, such issues may include, the communication among the agents and protocols, cooperation among a group of agents, describing the tasks to be executed, multi-agent motion planning, multi-agent task planning, task description, human interface role, solving deadlocks ... etc. The aim of this paper is to develop a simulation environment for an autonomous and decentralized multi-agent robotic system. For this purpose, it is neces- sary to simulate and analyze the main functions of each agent, design and analysis of the actions to be carried out by each agent, analyzing their real time behavior with a variety of situations considering their environ- ment and the treatment of unexpected events. Also, it is aecessary to find out an efficient strategy to realize effective means for the agents to form a cooperating manner that support task execution and problem solv- ing. In parallel with the mentioned purposes the ACTRESS structure is simulated as functions and behavior using the developed simulation environment. This simulation includes a group of mobile robots, human interface and global environment manager. Then, software integration that deal with various aspect of the overall problem is developed. Different experi- ments are performed to analyze the performance of each individual robotor and the overall system.

An autonomous and decentralized robot system, ACTRESS, is being developed as an intelligent robot system which can execute high level tasks. The concept of ACTRESS is introduced, and the communication framework between robotic agents that compose the robot system is proposed. Taking a task of pushing objects by multiple mobile robots in a certain working environment as a case study of ACTRESS, the strategy for organizing the robot system is discussed, and the required functions for the robot system are introduced. The optimal distribution of the functions is designed based on communication evaluation. An algorithm for static path planning and a strategy for dynamic path planning are proposed.>

An autonomous and decentralized robot system, ACTRESS, is being developed as an intelligent robot system which is composed of multiple robotic agents. The basic concept and strategy for path planning in a dynamically changing environment with multiple autonomous mobile robots are introduced. Collision avoidance with dynamic path planning according to the complexity of the situations is also discussed. A method for collision avoidance based on rules and negotiation by communication is presented. Finally, this method is applied to a developed prototype system consisting of two mobile robots, and the result of experiments operating on real robots is given.>

Deals with a kind of cellular robotic system (CEBOT), called self organizing manipulator system. In this system the configuration of the manipulator can be changed according to the given task. The authors use two manipulators to assemble, disassemble, and reassemble cellular universal manipulator by cooperation work. When a simple task is given, the system can analyze the task and generate the structure of the cellular manipulator. The assembling work was divided into modular subtasks, and then the system generate the macro command based on the database. The order of subtask and the collision avoidance are considered for planning the cooperation work by using the collision database which was calculated before hand. They show the algorithm for calculation the collision database and for planning the cooperation work. The results of the simulation and the experiment are shown.>

A technique is developed based on free link concept to construct the available free space between obstacles within robot's environment in terms of free convex area. Then, a new kind of vertex graph called MAKLINK is constructed to support the generation of a collision free path. This graph is constructed using the midpoints of common free links between free convex area as passing points. These points correspond to nodes in a graph and the connection between them within each convex area as arcs in this graph. A collision free path can be efficiently generated using the MAKLINK graph. The complexity of the search for a collision free path is greatly reduced by minimizing the size of the graph to be searched concerning the number of nodes and the number of arcs connecting them. The analysis of the proposed algorithm shows its efficiency in terms of computation ability, safety, optimality, and in supporting robot navigation along the generated path.>