Building LiDAR generative models holds promise as powerful data priors for restoration, scene manipulation, and scalable simulation in autonomous mobile robots. In recent years, approaches using diffusion models have emerged, significantly improving training stability and generation quality. Despite their success, diffusion models require numerous iterations of running neural networks to generate high-quality samples, making the increasing computational cost a potential barrier for robotics applications. To address this challenge, this paper presents R2Flow, a fast and high-fidelity generative model for LiDAR data. Our method is based on rectified flows that learn straight trajectories, simulating data generation with significantly fewer sampling steps compared to diffusion models. We also propose an efficient Transformer-based model architecture for processing the image representation of LiDAR range and reflectance measurements. Our experiments on unconditional LiDAR data generation using the KITTI-360 dataset demonstrate the effectiveness of our approach in terms of both efficiency and quality.

Interactive art appreciation is a method in which multiple participants engage in repeated discussions to interpret artworks. In this approach, a facilitator plays a crucial role by asking questions to the viewers. However, there are currently limited opportunities for facilitator training. To address this, this paper proposes a Mixed Reality (MR) AI system that utilizes MR headsets and a Large Language Model to train facilitators. The system features five virtual viewers, each with distinct personalities and whose facial expressions change through dialogue, allowing users to practice facilitation through interactive dialogue. Contents of what the virtual viewers say are generated by the Large Language Model, GPT-4o. Moreover, in addition to GPT-4o, conversations are conducted using the speech recognition system, Whisper, and the speech synthesis system, Azure Text-To-Speech.

As global efforts to explore the Moon intensify, the need for high-quality 3D lunar maps becomes increasingly critical—particularly for long-distance missions such as NASA’s Endurance mission concept, in which a rover aims to traverse 2,000 km across the South Pole–Aitken basin. Kaguya TC (Terrain Camera) images, though globally available at 10 m/pixel, suffer from altitude inaccuracies caused by stereo matching errors and JPEG-based compression artifacts. This paper presents a method to improve the quality of 3D maps generated from Kaguya TC images, focusing on mitigating the effects of compression-induced noise in disparity maps. We analyze the compression behavior of Kaguya TC imagery, and identify systematic disparity noise patterns, especially in darker regions. In this paper, we propose an approach to enhance 3D map quality by reducing residual noise in disparity images derived from compressed images. Our experimental results show that the proposed approach effectively reduces elevation noise, enhancing the safety and reliability of terrain data for future lunar missions.

A novel neural network called the isomorphic mesh generator (iMG) is proposed to generate isomorphic meshes from point clouds containing noise and missing parts. Isomorphic meshes of arbitrary objects exhibit a unified mesh structure, despite objects belonging to different classes. This unified representation enables various modern deep neural networks (DNNs) to easily handle surface models without requiring additional pre-processing. Additionally, the unified mesh structure of isomorphic meshes enables the application of the same process to all isomorphic meshes, unlike general mesh models, where processes need to be tailored depending on their mesh structures. Therefore, the use of isomorphic meshes can ensure efficient memory usage and reduce calculation time. Apart from the point cloud of the target object used as input for the iMG, point clouds and mesh models need not be prepared in advance as training data because the iMG is a data-free method. Furthermore, the iMG outputs an isomorphic mesh obtained by mapping a reference mesh to a given input point cloud. To stably estimate the mapping function, a step-by-step mapping strategy is introduced. This strategy enables flexible deformation while simultaneously maintaining the structure of the reference mesh. Simulations and experiments conducted using a mobile phone have confirmed that the iMG reliably generates isomorphic meshes of given objects, even when the input point cloud includes noise and missing parts.

Generative modeling of 3D LiDAR data is an emerging task with promising applications for autonomous mobile robots, such as scalable simulation, scene manipulation, and sparse-to-dense completion of LiDAR point clouds. While existing approaches have demonstrated the feasibility of image-based LiDAR data generation using deep generative models, they still struggle with fidelity and training stability. In this work, we present R2DM, a novel generative model for LiDAR data that can generate diverse and high-fidelity 3D scene point clouds based on the image representation of range and reflectance intensity. Our method is built upon denoising diffusion probabilistic models (DDPMs), which have shown impressive results among generative model frameworks in recent years. To effectively train DDPMs in the LiDAR domain, we first conduct an in-depth analysis of data representation, loss functions, and spatial inductive biases. Leveraging our R2DM model, we also introduce a flexible LiDAR completion pipeline based on the powerful capabilities of DDPMs. We demonstrate that our method surpasses existing methods in generating tasks on the KITTI-360 and KITTI-Raw datasets, as well as in the completion task on the KITTI-360 dataset. Our project page can be found at https://kazuto1011.github.io/r2dm.

In this paper, we introduce a framework for imitation learning in navigation that enables policy learning from one-shot images without a physical robot and facilitates the transfer of this policy from simulation to reality. Utilizing Neural Radiance Fields (NeRF), our approach generates a simulated environment and simultaneously models expert behavior. This removes the necessity for a physical robot during both the expert teaching phase and the agent’s learning process, allowing for the application of policies learned within the NeRF simulation to real-world robots. We validate our method by demonstrating the navigation with an actual robot using the policy learned by our approach. Moreover, we present a method for adapting to changes in the robot configuration, such as camera parameters and robot dimensions, by simulating adjustments in the robot configuration throughout the learning and assessing its generalizability.

Indoor robot localization is important for the realization of autonomous service robots. Various studies have been conducted on "indoor GPS" measurements using wireless distance sensors such as ultrasonic beacons. However, when these beacons encounter non-line-of-sight (NLoS) conditions due to obstacles, accurate distance measurements become challenging because of multipath and other effects. In this study, we propose a method for simultaneously estimating a robot’s position and distance to reflective surfaces in an environment using wireless distance sensors. The proposed method can estimate not only the robot’s position but also the reflection of the beacon signal. First, the wheel odometry of the robot is assumed to be the initial value, and the measured distance from the beacon to the robot is used as a factor to construct the factor graph. Second, the distance to the reflective surface of the beacon signal, which is parallel to the robot’s movement plane, was estimated from the robot position sequence using the GMM and used as a noise model in the factor graph. Finally, the method is evaluated by acquiring data in a real environment with obstacles. Compared with a method that does not consider reflection paths, this method demonstrated improved accuracy and effectiveness.

Mobile robot navigation in crowded environments with pedestrians is a crucial challenge in realizing service robots that can assist people in their daily lives. Navigation methods for mobile robots in environments employing deep reinforcement learning have been extensively studied. However, addressing such unexpected situations is a significant challenge. This study presents an approach that discerns whether a situation has been supposed to utilize a normalizing flow and dynamically switches between learning- and rule-based methods. Specifically, the proposed method achieves a higher success rate than employing only a learning-based approach and reaches the destination faster than employing only a rule-based approach in unexpected situations. Experiments are conducted to validate the performance enhancement achieved with the proposed switching method in both simulated and real-world settings.

The search for refining 3D LiDAR data has attracted growing interest motivated by recent techniques such as supervised learning or generative model-based methods. Existing approaches have shown the possibilities for using diffusion models to generate refined LiDAR data with high fidelity, although the performance and speed of such methods have been limited. These limitations make it difficult to execute in real-time, causing the approaches to struggle in real-world tasks such as autonomous navigation and human-robot interaction. In this work, we introduce a novel approach based on conditional diffusion models for fast and high-quality sparse-to-dense upsampling of 3D scene point clouds through an image representation. Our method employs denoising diffusion probabilistic models trained with conditional inpainting masks, which have been shown to give high performance on image completion tasks. We introduce a series of experiments, including multiple datasets, sampling steps, and conditional masks. This paper illustrates that our method outperforms the baselines in sampling speed and quality on upsampling tasks using the KITTI-360 dataset. Furthermore, we illustrate the generalization ability of our approach by simultaneously training on real-world and synthetic datasets, introducing variance in quality and environments.

In recent years, labor accidents and a shortage of skilled workers due to an aging population have become significant issues at construction sites in Japan. To address these challenges, we are developing a Cyber-Physical System (CPS) platform called ROS2-TMS for Construction, which aims to improve both the efficiency and safety of earthwork operations. In this study, we propose a task management system for construction machinery using an open platform named OPERA as an additional function of ROS2-TMS for Construction. This task management system controls construction machinery using environmental information stored in a database, which collects and stores data from sensors deployed throughout the construction site, and an extended Behavior Tree. At the end of this study, the results of the initial validation tests of autonomous earthwork operations using an OPERA-compatible backhoe ZX200 are presented.

We have developed HEARTS, a dementia care training system using augmented reality based on Humanitude. Humanitude is a multimodal comprehensive care technique for dementia, and has attracted attention as a method to reduce the burden on both caregivers and patients. However, the HEARTS developed so far could not evaluate “speaking” skills based on the content of conversations among “seeing,” “touching,” and “speaking,” all of which are fundamental skills in Humanitude. Therefore, we attempted a new quantitative evaluation of trainees' “speaking” skills by estimating the emotional value of conversational content using GPT-4 named HEARTS 5. A survey of caregivers was conducted using the developed system and was well received by the participants. We also developed a HEARTS 5 conversational version based on HEARTS 5, with the addition of GPT-4 conversation generation and Azure Text-to-Speech.

It is difficult to automate the grasping of delicate fruits such as peaches, which require sensitive work. Envelope grasping is an effective approach for grasping such objects due to its large contact area with the object. A method to realize envelope grasping is the fingered gripper, which can perform grasping that follows the shape of the object through compliant interaction. However, this method has the problem that it is structurally difficult to fill the space between fingers. Another method of achieving enveloping grasp is to envelop the object by membrane. This method can touch the object without gap between fingers, but it is difficult to grasp unexpectedly large objects because it cannot deform as large as a fingered gripper. In this study, we propose a soft gripper that realizes envelope grasping while largely deforming several fingers. The proposed method opens and closes using the turning over of the hemispherical shell due to the 3D snap through buckling mechanism. By employing this mechanism, allowing all objects touching the edge of the hemispherical shell to be moved simultaneously and significantly, fingers arranged without gaps can be deformed largely. Another advantage is that it requires no energy to maintain the grasping state due to its bistable, snap through buckling characteristics. In this paper, we present a design methodology based on specific examples and an evaluation of performance.

We proposed a teleoperation method, illusory control (IC), that provides a comfortable operation experience using a seamless transition between real and pre-prepared virtual environments. Therefore, the mobile robot with IC could function solely in familiar environments. To make IC applicable in unfamiliar environments, this study proposes a novel method, instant IC, that eliminates the requirement for a pre-prepared virtual environment. The proposed robot system can instantly generate a virtual environment using actual 360° images of the robot in motion, utilizing instant neural graphics primitives and neural radiance fields. The 360° images allow the entire surrounding environment to be virtualized without requiring specific camera orientations. In addition, by optimizing the density of neural radiance fields using depth estimation results beforehand, the reconstruction accuracy at unknown poses can be guaranteed. Furthermore, we propose a depth scaling method based on the actual measurements obtained by LiDAR to increase the consistency of virtual and real environments. With this instant virtual environment, the proposed system enables teleoperation in unknown environments via the seamless transition between real and virtual environments. The experimental results exhibit consistent and smooth back-and-forth transitions between virtual and real space in mobile robot teleoperation.

3D LiDAR sensors are indispensable for the robust vision of autonomous mobile robots. However, deploying LiDAR-based perception algorithms often fails due to a domain gap from the training environment, such as inconsistent angular resolution and missing properties. Existing studies have tackled the issue by learning inter-domain mapping, while the transferability is constrained by the training configuration and the training is susceptible to peculiar lossy noises called ray-drop. To address the issue, this paper proposes a generative model of LiDAR range images applicable to the data-level domain transfer. Motivated by the fact that LiDAR measurement is based on point-by-point range imaging, we train an implicit image representation-based generative adversarial networks along with a differentiable ray-drop effect. We demonstrate the fidelity and diversity of our model in comparison with the point-based and image-based state-of-the-art generative models. We also showcase upsampling and restoration applications. Furthermore, we introduce a Sim2Real application for LiDAR semantic segmentation. We demonstrate that our method is effective as a realistic ray-drop simulator and outperforms state-of-the-art methods.

Assisting patients with dementia is a significant social issue. Currently, to assist patients with dementia, a multimodal care technique called Humanitude is gaining popularity. In Humanitude, the patients are assisted through various techniques to stand up independently by utilizing their motor functions as much as possible. Humanitude care techniques encourage caregivers to increase the area of contact with patients during the sit-to-stand motion. However, Humanitude care techniques are not accurately performed by novice caregivers. Therefore, in this study, a smock-type wearable sensor was developed to measure the proximity between caregivers and care recipients during sit-to-stand motion assistance. A measurement experiment was conducted to evaluate the proximity differences between Humanitude care and simulated novice care. In addition, the effects of different care techniques on the center of mass (CoM) trajectory and muscle activity of the care recipients were investigated. The results showed that the caregivers tend to bring their top and middle trunk closer in Humanitude care compared with novice simulated care. Furthermore, it was observed that the CoM trajectory and muscle activity under Humanitude care were similar to those observed when the care recipient stands up independently. These results validate the effectiveness of Humanitude care and provide useful information for teaching techniques in Humanitude.

3D laser scanning by LiDAR sensors plays an important role for mobile robots to understand their surroundings. Nevertheless, not all systems have high resolution and accuracy due to hardware limitations, weather conditions, and so on. Generative modeling of LiDAR data as scene priors is one of the promising solutions to compensate for unreliable or incomplete observations. In this paper, we propose a novel generative model for learning LiDAR data based on generative adversarial networks. As in the related studies, we process LiDAR data as a compact yet lossless representation, a cylindrical depth map. However, despite the smoothness of real-world objects, many points on the depth map are dropped out through the laser measurement, which causes learning difficulty on generative models. To circumvent this issue, we introduce measurement uncertainty into the generation process, which allows the model to learn a disentangled representation of the underlying shape and the dropout noises from a collection of real LiDAR data. To simulate the lossy measurement, we adopt a differentiable sampling framework to drop points based on the learned uncertainty. We demonstrate the effectiveness of our method on synthesis and reconstruction tasks using two datasets. We further showcase potential applications by restoring LiDAR data with various types of corruption.

Shared control, in which teleoperation and autonomous control are combined to move the robot, is expected to improve the efficiency of the user teleoperation problem. However, a problem exists whereby the user acceptance decreases owing to the conflict of intention between the teleoperation and autonomous control. In this study, we address this problem by providing an illusion to humans. We propose a teleoperation method named the “Illusory Control” that can achieve both mobility efficiency and user acceptance by implementing a cyber-physical system that controls a robot in real space through robot operations in virtual space. Illusory Control has two functions: the “Illusion of Intention,” which provides the illusion that the robot is operating according to human intention, and “Illusion of Time,” which provides the illusion of time to fill the gap by changing human behavior when the robot positions in the virtual space and real space diverge. Preliminary teleoperation experiments with subjects demonstrated that the system improves the operational efficiency and acceptance of the system compared to conventional teleoperation methods, namely direct teleoperation and shared control.

This paper presents a new system for predicting the spatial change of Nasopharyngeal carcinoma(NPC) and organ-at-risks (OARs) volumes over the course of the radiation therapy (RT) treatment for facilitating the workflow of adaptive radiotherapy. The proposed system, called “Tumor Evolution Prediction (TEP-Net)”, predicts the spatial distributions of NPC and 5 OARs, separately, in response to RT in the coming week, week n. Here, TEP-Net has (n-1)-inputs that are week 1 to week n-1 of CT axial, coronal or sagittal images acquired once the patient complete the planned RT treatment of the corresponding week. As a result, three predicted results of each target region are obtained from the three-view CT images. To determine the final prediction of NPC and 5 OARs, two integration methods, weighted fully connected layers and weighted voting methods, are introduced. From the experiments using weekly CT images of 140 NPC patients, our proposed system achieves the best performance for predicting NPC and OARs compared with conventional methods.

This study develops a training system for a multimodal comprehensive care methodology for dementia patients called Humanitude. Humanitude has attracted much attention as a gentle and effective care technique. It consists of four main techniques, namely, eye contact, verbal communication, touch, and standing up, and more than 150 care elements. Learning Humanitude thus requires much time. To provide an effective training system for Humanitude, we develop a training system that realizes sensing and interaction simultaneously by combining a real entity and augmented reality technology. To imitate the interaction between a patient and a caregiver, we superimpose a three-dimensional CG model of a patient's face onto the head of a soft doll using augmented reality technology. Touch information such as position and force is sensed using the whole body wearable tactile sensor developed to quantify touch skills. This training system enables the evaluation of eye contact and touch skills simultaneously. We build a prototype of the proposed training system and evaluate the usefulness of the system in public lectures.

Detection of spatial change around a robot is indispensable in several robotic applications, such as search and rescue, security, and surveillance. The present paper proposes a fast spatial change detection technique for a mobile robot using an on-board RGB-D/stereo camera and a highly precise 3D map created by a 3D laser scanner. This technique first converts point clouds in a map and measured data to grid data (ND voxels) using normal distributions transform and classifies the ND voxels into three categories. The voxels in the map and the measured data are then compared according to the category and features of the ND voxels. Overlapping and voting techniques are also introduced in order to detect the spatial changes more robustly. We conducted experiments using a mobile robot equipped with real-time range sensors to confirm the performance of the proposed real-time localization and spatial change detection techniques in indoor and outdoor environments.

We aim to develop a supporting system which enhances the ability of human's short-term visual memory in an intelligent space where the human and a service robot coexist. Particularly, this paper focuses on how we can interpret and record diverse and complex life events on behalf of humans, from a multi-perspective viewpoint. We propose a novel method named "fourth-person captioning", which generates natural language descriptions by summarizing visual contexts complementarily from three types of cameras corresponding the first-, second-, and third-person viewpoint. We first extend the latest image captioning technique and design a new model to generate a sequence of words given the multiple images. Then we provide an effective training strategy that needs only annotations supervising images from a single viewpoint in a general caption dataset and unsupervised triplet instances in the intelligent space. As the three types of cameras, we select a wearable camera on the human, a robot-mounted camera, and an embedded camera, which can be defined as the first-, second-, and third-person viewpoint, respectively. We hope our work will accelerate a cross-modal interaction bridging the human's egocentric cognition and multi-perspective intelligence.

Mutual subspace method (MSM), which is one of image-based approaches, showed strong discrimination capability in gait recognition. In general, 2D image matrices are transformed into 1D image vectors to be used as input into MSM, and then principal component analysis (PCA) is applied to 1D vectors to generate a subspace. However, due to the high dimensionalities of 1D vectors, the evaluation accuracy of the covariance matrix in PCA is not high enough. This results in a decrease in performance, especially in case that speed difference between gallery and probe dataset is big. Thus in this paper we propose a method, which expands the MSM-based method, to recognize people with higher accuracy. The proposed method divides the human body area into multiple areas, followed by adaptive choice of areas that have high discrimination capability. Moreover, the proposed method utilizes the frieze pattern, which is one of gait features, as an additional input into MSM. The use of divided areas and the frieze pattern allows us to evaluate the covariance matrix with higher accuracy. In experiments we applied the proposed method to challenging databases with speed variations, and we show the effectiveness of the proposed method.

This paper proposes new software and hardware platforms named ROS-TMS and Big Sensor Box, respectively, for an informationally structured environment. We started the development of a management system for an informationally structured environment named Town Management System (TMS) in the Robot Town Project in 2005. Since then we have been continuing our efforts to improve performance and to enhance TMS functions. Recently, we launched a new version of TMS named ROS-TMS, which resolves some critical problems in TMS by adopting the Robot Operating System (ROS) and utilizing the high scalability and numerous resources of ROS. In this paper, we first discuss the structure of a software platform for the informationally structured environment and describe in detail our latest system, ROS-TMS version 4.0. Next, we introduce a hardware platform for the informationally structured environment named Big Sensor Box, in which a variety of sensors are embedded and service robots are operated according to the structured information under the management of ROS-TMS. Robot service experiments including a fetch-and-give task and autonomous control of a wheelchair robot are also conducted in Big Sensor Box.

This paper presents a near-future perception system named “Previewed Reality”. The system consists of an informationally structured environment (ISE), an immersive VR display, a stereo camera, an optical tracking system, and a dynamic simulator. In an ISE, a number of sensors are embedded, and information such as the position of furniture, objects, humans, and robots, is sensed and stored in a database. The position and orientation of the immersive VR display are also tracked by an optical tracking system. Therefore, we can forecast the next possible events using a dynamic simulator and synthesize virtual images of what users will see in the near future from their own viewpoint. The synthesized images, overlaid on a real scene by using augmented reality technology, are presented to the user. The proposed system can allow a human and a robot to coexist more safely by showing possible hazardous situations to the human intuitively in advance.

3D modeling of real objects by a 3D laser scanner has become popular in many applications, such as reverse engineering of petrochemical plants, civil engineering and construction, and digital preservation of cultural properties. Despite the development of lightweight and high-speed laser scanners, the complicated measurement procedure and long measurement time are still heavy burdens for widespread use of laser scanning. To solve these problems, a robotic 3D scanning system using multiple robots has been proposed. This system, named CPS-SLAM, consists of a parent robot with a 3D laser scanner and child robots with target markers. A large-scale 3D model is acquired by an on-board 3D laser scanner on the parent robot from several positions determined precisely by a localization technique, named the Cooperative Positioning System (CPS), that uses multiple robots. Therefore, this system can build a 3D model without complicated post-processing procedures such as ICP. In addition, this system is an open-loop SLAM system and a very precise 3D model can be obtained without closed loops. This paper proposes an automatic planning technique for a laser measurement by using CPS-SLAM. Planning a proper scanning strategy depending on a target structure makes it possible to perform laser scanning efficiently and accurately even for a large-scale and complex environment. The proposed technique plans an efficient scanning strategy automatically by taking account of several criteria, such as visibility between robots, error accumulation, and efficient traveling. We conducted computer simulations and outdoor experiments to verify the performance of the proposed technique.

This paper proposes a new method for mapping volume models of human organs onto a target volume with simple shapes. The proposed method is based on our modified Self-organizing Deformable Model (mSDM) which finds the one-to-one mapping with no foldovers between an arbitrary object surface model and a target surface. By extending mSDM to apply to organ volume models, the proposed method, called volumetric SDM (vSDM), establishes the one-to-one correspondence between the volume model and its target volume. At the same time, vSDM preserves geometrical properties of the original model before and after the mapping. In addition, vSDM allows to control the mapping of interior structures of the organ model onto specific regions inside the target volume. These characteristics of vSDM enables to easily find a reliable correspondence between different volume models via a common target volume.

We propose an object tracking system which recognizes everyday objects and estimates their positions by using distributed sensors in a room and mobile robots. The placement of objects is frequently changed according to human activities. Although a passive RFID tag is attached to each object for the object's recognition, the placement is often not uniquely determined due to the deficiency of measured data. We have already proposed a method for estimating the placement of objects by using the moving trajectories of objects. This estimation result is expressed as the probability distribution of the object placement. However intersections of trajectories cause the decrease of the estimation accuracy. So we propose a new method based on Bayesian inference to improve the estimation accuracy by using the size and the shape of an object measured by laser range finder. Then a mobile robot settles the placement with small workload by using the mounted sensor. The system successfully recognized and localized 10 objects in the experiment.

We present two multi-modal panoramic 3D outdoor (MPO) datasets for semantic place categorization with six categories: forest, coast, residential area, urban area and indoor/outdoor parking lot. The first dataset consists of 650 static panoramic scans of dense (9,000,000 points) 3D color and reflectance point clouds obtained using a FARO laser scanner with synchronized color images. The second dataset consists of 34,200 real-time panoramic scans of sparse (70,000 points) 3D reflectance point clouds obtained using a Velodyne laser scanner while driving a car. The datasets were obtained in the city of Fukuoka, Japan and are publicly available in [1], [2]. In addition, we compare several approaches for semantic place categorization with best results of 96.42% (dense) and 89.67% (sparse).

In this paper, for the purpose of monitoring people moving on the ground from a low-altitude aerial vehicle, we propose a method for stable image stabilization using planar information of the ground. In existing methods the homography-based method has been popularly used for image stabilization. However, in case that captured images consist of non-planar areas, the performance of the homography-based method decreases. The essential-based method works well in this scene, but the accuracy gets worse in case that images mainly include planar areas. Thus in the proposed method, we utilize the Geometric Robust Information Criterion (GRIC) model to choose a method from the homography-based method or the essential-based method. After the selection of the method, we extract a planar area in the scene by fitting a plane using RANSAC, followed by detection of people on the ground with high accuracy. Experimental results confirm that the effectiveness of the proposed method.

In recent years, several low-cost 3D laser scanners are being brought to the market and 3D laser scanning is becoming widely used in many fields. For example, 3D modeling of architectural structures or digital preservation of cultural heritages are typical applications for 3D laser scanning. Despite of the development of light-weight and high-speed laser scanners, however, the complicated measurement procedure and long measurement time are still a heavy burden for the widespread use of laser scanning. We have proposed a robotic 3D scanning system using multiple robots named CPS-SLAM, which consists of parent robots with a 3D laser scanner and child robots with target markers. In this system, a large-scale 3D model can be acquired by an on-board 3D laser scanner on a parent robot from several positions determined precisely by the localization technique using multiple robots named Cooperative Positioning System, CPS. Therefore, this system enables to build a 3D model without complicated post-processing procedures such as ICP. In addition, this system is an open-loop SLAM system and a quite precise 3D model can be obtained without closed loops. This paper proposes an automatic planning technique of a laser measurement for CPS-SLAM. By planning a proper scanning strategy depending on a target structure, it is possible to perform laser scanning efficiently and accurately even for a large-scale and complex environment. Proposed technique plans an efficient scanning strategy automatically by taking account of several criteria, such as visibility between robots, error accumulation, and efficient traveling. We conducted computer simulations and outdoor experiments to verify the performance of the proposed technique.

We propose an evaluation method of grasp stability which takes into account the elastic deformation of fingertips from the viewpoint of energy. An evaluation value of grasp stability is derived as the minimum energy which causes slippage of a fingertip on its contact surface. To formulate the evaluation value, the elastic potential energy of fingertips and the gravitational potential energy of a grasped object are considered. It is ensured that fingertips do not slip on grasped object surfaces if the external energy applied to the object is less than the evaluation value. Since our evaluation value explicitly considers the deformation values of fingertips, grasp stability is evaluated by taking into consideration the contact forces generated by the deformation. The effectiveness of our method is verified through numerical examples.

Person recognition from gait images is generally not robust to changes in appearance, such as variations of the walking direction. In general conventional methods have focused on training a model to transform gait features or gait images to those at a different viewpoint, but the performance gets worse in case the model is not trained at a viewpoint of a subject. In this paper we propose a novel gait recognition approach which differs a lot from existing approaches in that the subject's sequential 3D models and his/her motion are directly reconstructed from captured images, and arbitrary viewpoint images are synthesized from the reconstructed 3D models for the purpose of gait recognition robust to changes in the walking direction. Moreover, we propose a gait feature, named Frame Difference Frieze Pattern (FDFP), which is robust to high frequency noise. The efficiency of the proposed method is demonstrated through experiments using a database that includes 41 subjects.

We present a novel approach for outdoor place categorization using synchronized texture and depth images obtained using a laser scanner. Categorizing outdoor places according to type is useful for autonomous driving or service robots, which work adaptively according to the surrounding conditions. However, place categorization is not straight forward due to the wide variety of environments and sensor performance limitations. In the present paper, we introduce a two-dimensional local ternary pattern (2D-LTP) descriptor using a pair of synchronized texture and depth images. The proposed 2D-LTP describes the local co-occurrence of a synchronized and complementary image pair with ternary patterns. In the present study, we construct histograms of a 2D-LTP as a feature of an outdoor place and apply singular value decomposition (SVD) to deal with the high dimensionality of the place. The novel descriptor, i.e., the 2D-LTP, exhibits a higher categorization performance than conventional image descriptors with outdoor place experiments.

This paper introduces the concept of first-person animal activity recognition, the problem of recognizing activities from a view-point of an animal (e.g., a dog). Similar to first-person activity recognition scenarios where humans wear cameras, our approach estimates activities performed by an animal wearing a camera. This enables monitoring and understanding of natural animal behaviors even when there are no people around them. Its applications include automated logging of animal behaviors for medical/biology experiments, monitoring of pets, and investigation of wildlife patterns. In this paper, we construct a new dataset composed of first-person animal videos obtained by mounting a camera on each of the four pet dogs. Our new dataset consists of 10 activities containing a heavy/fair amount of ego-motion. We implemented multiple baseline approaches to recognize activities from such videos while utilizing multiple types of global/local motion features. Animal ego-actions as well as human-animal interactions are recognized with the baseline approaches, and we discuss experimental results.

This paper presents a grasp planner which allows a robot to grasp the constricted parts of objects in our daily life. Even though constricted parts can be grasped more firmly than convex parts, previous planners have not sufficiently focused on grasping this part. We develop techniques for quadric surface approximation, grasp posture generation, and stability evaluation for grasping constricted parts. By modeling an object into multiple quadric surfaces, the planner generates a grasping posture by selecting one-sheet hyperbolic surfaces or two adjacent ellipsoids as constricted parts. When a grasping posture being generated, the grasp stability is evaluated based on the distribution of the stress applied to an object by the fingers. We perform several simulations and experiments to verify the effectiveness of our proposed method.

For creating a three-dimensional (3D) model of a real object using a laser scanner and a camera, texture mapping is an effective technique to enhance the reality. However, in case that the positions of the camera and the laser scanner differ from each other, some textureless regions (holes) may exist on the object surface where the appearance information is missing due to the occlusion or out-of-sight of the camera. In this paper, we propose a new texture completion technique utilizing laser reflectivity for hole-free texture mapping. The laser reflectivity, which denotes the power of a reflected laser light/pulse, is obtained as by-product of the range information at laser scanning. Since the laser reflectivity captures the appearance property of the target as a camera image, it is reasonable that the regions with similar reflectance properties have similar color textures. Based on this idea, texture information in these holes is copied and pasted from the other texture regions according to the similarity and the order determined by the texture and laser reflectivity. To verify the performance of the proposed technique, we carried out texture completion experiments in real scenes.

In this paper, we propose a new technique for adding color to a surface of a 3D geometrical model utilizing laser reflectivity. A time-of-flight laser scanner obtains a range image from the sensor toward the target by measuring the round-trip time of a laser pulse. At the same time, for most laser scanners, the reflectance image (which is the strength of the reflected light) is available as a by-product of the range value. The proposed technique first assigns appearance information to a 3D model by colorizing a reflectance image based on the similarity of color and reflectance images. Then the color information is transferred to the corresponding range image, and the colorized 3D model is obtained. We carried out experiments using a laser scanner and showed the performance of the proposed technique in several conditions.

Conventional methods of gait analysis for person identification use features extracted from a sequence of camera images taken during one or more gait cycles. An implicit assumption is made that the walking direction does not change. However, cameras deployed in real-world environments (and often placed at corners) capture images of humans who walk on paths that, for a variety of reasons, such as turning corners or avoiding obstacles, are not straight but curved. This change of the direction of the velocity vector causes a decrease in performance for conventional methods. In this paper we address this aspect, and propose a method that offers improved identification results for people walking on curved trajectories. The large diversity of curved trajectories makes the collection of complete real world data infeasible. The proposed method utilizes a 4D gait database consisting of multiple 3D shape models of walking subjects and adaptive virtual image synthesis. Each frame, for the duration of a gait cycle, is used to estimate a walking direction for the subject, and consequently a virtual image corresponding to this estimated direction is synthesized from the 4D gait database. The identification uses affine moment invariants as gait features. Experiments using the 4D gait database of 21 subjects show that the proposed method has a higher recognition performance than conventional methods.

This paper describes an object tracking system for a robot working in an everyday environment, which tracks and recognizes everyday objects. Passive RFID (Radio Frequency IDentification) tags are attached to the objects for object recognition. The system consists of static sensors embedded in the environment and mobile sensors mounted on mobile robots. By utilizing the different characteristics and advantages of these sensors, the system achieves good performance in an everyday environment. Although the tag ID and the position of an object carried by a person is not measurable by static sensors or by mobile sensors, the system can estimate them by using an SIR (Sequential importance resampling) particle filter that integrates the data obtained by the static sensors and the mobile sensors. In the experiment, the system successfully tracked 20 objects, some of which were held by a person.

This paper proposes a new visual servoing method for object manipulation robust to considerable time-delays of visual information. There still remain several problems in visual servoing methods although they are quite useful and effective for dexterous object manipulation. For instance, time-delays to obtain necessary information for object manipulation from visual images induce unstable behavior. The time-delays are mainly caused by low sampling rate of visual sensing system, computational cost for image processing, and latency of data transmission from visual sensor to processor. The method makes it possible to avoid such unstable behavior of the systems due to considerable time-delays using virtual object frame defined by only each joint angle. Firstly, a new control scheme for object manipulation using the virtual object frame is designed. Next, numerical simulations are conducted to verify the effectiveness of the control scheme. Finally, experimental results are shown to demonstrate the practical usefulness of proposed method.

In this paper, a new iterative learning control method which uses multiple space variables for a musculoskeletal-like arm system is proposed to improve the robustness against noises being included in sensory information. In our previous works, the iterative learning control method for the redundant musculoskeletal arm to acquire a desired endpoint trajectory simultaneous with an adequate internal force was proposed. The controller was designed using only muscle space variables, such as a muscle length and contractile velocity. It is known that the movement of the musculoskeletal system can be expressed in a hierarchical three-layered space which is composed of the muscle space, the joint space and the task space. Thus, the new iterative learning control input is composed of multiple space variables to improve its performance and robustness. Numerical simulations are conducted and their result is evaluated from the viewpoint of the robustness to noises of sensory information. An experiment is performed using a prototype of musculoskeletal-like manipulator, and the practical usefulness of the proposed method is demonstrated through the result.

This paper proposes a novel vision-based grasping and manipulation scheme of a multi-fingered hand-arm system robust for a temporary lack of sensory information. Visual information is one of the fundamental components for reliable grasping and manipulation by a multi-fingered hand-arm system. However, in case that visual information such as position and attitude of an object comes to be unavailable due to the occlusion or if the object goes out-of-sight temporarily, unstable and unfavorable behavior is often induced. The proposed method, which utilizes the stable grasping control and the concept of virtual frame, enables to grasp and manipulate an object stably even if the visual information becomes suddenly and temporarily unavailable during manipulation. Firstly, a dynamical model of object grasping using a multi-fingered hand-arm system is formulated. Next, a new control scheme for robust object grasping and manipulation using the virtual frame is proposed. Finally, numerical simulations are performed to verify the usefulness of the proposed method.

Two denoising techniques using reflectivity for noisy range images are proposed: range image smoothing by trilateral filter and range image inpainting by belief propagation. The trilateral filter makes use of reflectivity as well as spatial and intensity information so that geometric features, such as jump and roof edges, are preserved while smoothing. The range image inpainting technique based on belief propagation recovers a deteriorated range image using not only the adjacent range values but also the continuity of the reflectance image. We conduct simulations and experiments using synthesized images and actual range images taken by a laser scanner and verify that the proposed techniques suppress noise while preserving jump and roof edges and repair deteriorated range images.

We introduce a novel person identification method for a surveillance system of much wider area than conventional systems using CCTV cameras. In the proposed system, we install cameras to rooftops of buildings or a low altitude airship, and identify people by gait features extracted from shadows, which are projected on the ground by the sun in the daytime or lights in the evening. Since conventional systems extract gait features from actual body area, the correct classification ratio is reduced due to the lack of information of body area, in case that images are captured by overhead cameras. On the other hand, the proposed system enables to identify people by gait features which are extracted from shadows projected on the ground, even if images are captured by overhead cameras. In the proposed system, shadow areas projected on the ground are extracted automatically from captured images, and then analyze dynamics of shadow areas by the spherical harmonics. Experiments of person identification using actual outside images revealed that the proposed method showed the best performance than conventional methods, and the results indicate the feasibility of person identification based on shadow analysis.

People identification has numerous applications, ranging from surveillance/security to robotics. Face and body movement/ gait biometrics are the most important tools for this task. Traditional biometrics use direct observation of the body, yet in some situations a projection may offer more information than the direct signal, for example the shadow of a person observed from overhead, e.g. from an unmanned aerial vehicle, may contain more detail than the top view of the head/body. We introduced the idea of shadow biometrics, exploiting biometrics information in human shadow silhouettes as derived from video imagery; this enables “overhead biometrics”, for recognition of human identity and behavior from high altitude airborne platforms using overhead video sequences. In this paper, we provide a demonstration of person identification based on gait recognition from shadow analysis. We describe compensation steps to address shadow variation with conditions of observation (sun position, etc). We define measures of shape variation, such as horizontal stripes on the silhouette, their length change in time determines frequency components (here spherical harmonics) for each gait cycle, which are used for classification by a k-nearest neighbor classifier. A correct classification rate (CCR) of 95 % was obtained. A degradation of CCR from 95 % to 75 % was observed when reduced spatial and temporal resolution from 1cm to 2cm, and from 30fps to 15fps.

For a coexisting and collaborative society that incorporates humans and robots, the detection, tracking, and recognition of human motion are indispensable techniques for a robot to safely and securely interact with humans. The present paper proposes a motion tracking system using distributed network cameras that are placed in a sizeable environment, such as a street or a town. Model-based motion tracking is adopted in this system, and an asynchronous process is invoked for updating motion estimation in each camera individually. A 2D distance map created by the Fast Marching Method is used to estimate human motion in real-time. Experiments demonstrate that human motion while walking among eight distributed cameras is tracked correctly by automatically selecting appropriate cameras.

This paper describes a tactile sensing to estimate the position and orientation of a joint-axis of a linked object. This tactile sensing is useful when a multi-jointed multi-fingered robotic hand manipulates a tool which has a joint. This estimation requires sensing of the location of a contact point and the direction of an edge of the tool as contact information measured by a robotic fingertip. A conventional hard fingertip with a force sensor can measure only the location of a contact point. In contrast, we have already developed a robotic fingertip with a force sensor and a soft skin, and it can measure not only the location of a contact point but also the direction of an edge of an object. The estimation of a joint-axis of a linked object is demonstrated by using the soft fingertip.

We propose an object tracking system for a service robot working in an everyday environment. The system is composed of an intelligent cabinet, a floor sensing system and a data management system. The position of an object can be classified into three areas: 1) in/on furniture, 2) on the floor, 3) held by a human or a robot. Being equipped with a RFID reader and loadcells, the intelligent cabinet measures the position of an object in/on itself. The floor sensing system which uses a laser range finder, measures the position of an object on the floor and the position of a human walking in a room. The data management system integrates the position data of the intelligent cabinets and the floor sensing system, and it performs position measurement of an object carried by a human. The data management system provides robots with position information to support robot activities.

Repeated patterns are useful clues to learn previously unknown events in an unsupervised way. This paper presents a novel method that detects relatively long variable-length unknown repeated patterns in a motion sequence efficiently. The major contribution of the paper is two-fold: (1) Partly Locality Sensitive Hashing (PLSH) [1] is employed to find repeated patterns efficiently and (2) the problem of finding consecutive time frames that have a large number of repeated patterns is formulated as a combinatorial optimization problem which is solved via Dynamic Programming (DP) in polynomial time O(N1+1/α) thanks to PLSH where N is the total amount of data. The proposed method was evaluated by detecting repeated interactions between objects in everyday manipulation tasks and outperformed previous methods in terms of accuracy or computational time.

Mobile platforms when negotiating steps and stairs should be able to control theirs posture in order to avoid sudden tilting or falls. In particular, when considering applications for search and rescue operations where users have a very limited time of operation, the motion on stairs should be automated as much as possible. In this way operators can concentrate on their tasks (i.e. search of survivors and/or exploration of dangerous environments) rather than having to focus on the stability of the vehicle. A simple but very effective mechanism called ldquotailrdquo is introduced. The mechanical design and its control method is presented together with several tests and experiments carried out with a simple tracked vehicle in real environments.

This paper proposes a new person identification method using physiological and behavioral biometrics. Various person recognition systems have been proposed so far, and one of the recently introduced human characteristics for the person identification is gait. Although the shape of one's body has not been considered much as a characteristic, it is closely related to gait and it is difficult to disassociate them. So, the proposed technique introduces a new hybrid biometric, combining body shape (physiological) and gait (behavioral). The new biometric is the full spatio-temporal volume carved by a person who walks. In addition to this biometric, we extract unique biometrics in individuals by the following way: creating the average image from the spatio-temporal volume and forming the new spatio-temporal volume from differential images which are created by subtracting an average image from original images. Affine moment invariants are derived from these biometrics, and classified by a support vector machine. We used the leave-one-out cross validation technique to estimate the correct classification rate of 94 %.

In order to construct three-dimensional shape models of large-scale architectural structures using a laser range finder, a number of range images are taken from various viewpoints. These images are aligned using post-processing procedures such as the ICP algorithm. However, in general, before applying the ICP algorithm, these range images must be aligned roughly by a human operator in order to converge to precise positions. The present paper proposes a new modeling system using a group of multiple robots and an on-board laser range finder. Each measurement position is identified by a highly precise positioning technique called Cooperative Positioning System (CPS), which utilizes the characteristics of the multiple-robot system. Thus, the proposed system can construct 3D shapes of large-scale architectural structures without any post-processing procedure or manual registration. ICP is applied optionally for a subsequent refinement of the model. Measurement experiments in unknown and large indoor/outdoor environments are carried out successfully using the newly developed measurement system consisting of three mobile robots named CPS-V. Generating a model of Dazaifu Tenmangu, a famous cultural heritage, for its digital archive completes the paper.

In this paper, we propose a method that detects repeated motion patterns in a long motion sequence efficiently. Repeated motion patterns are the structured information that can be obtained without knowledge of the context of motions. They can be used as a seed to find causal relationships between motions or to obtain contextual information of human activity, which is useful for intelligent systems that support human activity in everyday environment. The major contribution of the proposed method is two-fold: (1) motion density is proposed as a repeatability measure and (2) the problem of finding consecutive time frames with large motion density is formulated as a combinatorial optimization problem which is solved via Dynamic Programming (DP) in polynomial time O(N log N) where N is the total amount of data. The proposed method was evaluated by detecting repeated interactions between objects in everyday manipulation tasks and outperformed the previous method in terms of both detectability and computational time.

Fire brigades and special agencies are often demanded to operate for search and aid of human lives in extremely dangerous scenarios. It is very important to first verify the safety of the environment and to obtain remotely a clear image of the scenario inside buildings or underground spaces. Several studies have been addressing the possibility of using robotic tools to carry out safe operations. This contribution presents the development of the HELIOS team, consisting of five tracked robots for urban search and rescue. Two units are equipped with manipulators for the accomplishment of particular tasks, such as the handling of objects and opening doors; the other three units, equipped with cameras and laser range finders, are utilized to create virtual 3D maps of the explored environment. The three units can move autonomously while collecting the data by using a collaborative positioning system (CPS). After an overview on the specifications of the team of robots and with respect to previous publications, detailed information about the improvements of the robot mechanical design and control systems are introduced. Tests of the CPS system and HELIOS IX vehicle together with a typical mission experiment are presented and discussed.

This paper proposes a novel control method for stable grasping using a multi-fingered hand-arm system with soft hemispherical finger tips. The proposed method is simple but easily achieves stable grasping of an arbitrary polyhedral object using an arbitrary number of fingers. Firstly, we formulate nonholonomic constraints between a multi-fingered hand-arm system and an object constrained by rolling contact with finger tips, and derive a condition for stable grasping by stability analysis. A new index for evaluating the possibility of stable grasping is proposed and efficient initial relative positions between finger tips and the object for realizing stable grasping are analyzed. The stability of the proposed system and the validity of the index are verified through numerical simulations.

This paper proposes a segmentation method of human manipulation task based on measurement of contact force imposed by a human hand on a grasped object. We define an index measure for segmenting a human manipulation task into primitives. The indices are calculated from the set of the contact forces measured at all the contact points during a manipulation task. Then, we apply the EM algorithm to the set of the indices in order to segment the manipulation task into primitives. These primitives are mapped onto the robotic hand to impose appropriate contact forces on a grasped object. In the experiments, manipulation tasks performed in daily human life have been successfully segmented.

This paper presents a parallel algorithm of the Level Set Method named the Parallel Fast Level Set Method, and its application for real-time 3D reconstruction of human shape and motion. The Fast Level Set Method is an efficient implementation algorithm of the Level Set Method and has been applied to several applications such as object tracking in video images and 3D shape reconstruction using multiple stereo cameras. In this paper, we implement the Fast Level Set Method on a PC cluster and develop a real-time motion capture system for arbitrary viewpoint image synthesis. To obtain high performance on a PC cluster, efficient load-balancing and resource allocation algorithms are crucial problems. We develop a novel optimization technique of load distribution based on the estimation of moving direction of object boundaries. In this technique, the boundary motion is estimated in the framework of the Fast Level Set Method, and the optimum load distribution is predicted and performed according to the estimated boundary motion and the current load balance. Experiments of human shape reconstruction and arbitrary viewpoint image synthesis using the proposed system are successfully carried out.

The present paper describes a decision method for the placement of tactile elements for manipulation task recognition. Based on the mutual information of the manipulation tasks and tactile information, an effective placement of tactile elements on a sensing glove is determined. Although the effective placement consists of a small number of tactile elements, it has a recognition performance that is as high as that of a placement consisting of many tactile elements. The effective placement of tactile elements decided by the proposed method has been evaluated through experiments involving the recognition of grasp type from grasp taxonomy defined by Kamakura [1].

This paper proposes a calibration method of a distributed vision network in a unified world coordinate system. The vision network system is conceived to support a robot working in our daily human life environment: the system provides with visual observation of the dynamically changing situation surrounding the robot. Vision cameras are rather sparsely distributed to cover a wide area such as a block of a town. Position, view direction and range of view are the camera parameters of principal importance to be estimated by the proposed method. A set of calibration data for each distributed camera is provided by a group of mobile robots having a cooperative positioning function and visually distinguishable markers mounted on the body of the robot.

This paper presents a sensor network system consisting of distributed cameras and laser range finders for multiple objects tracking. Sensory information from cameras is processed by the level set method in real time and integrated with range data obtained by laser range finders in a probabilistic manner using novel SIR/MCMC combined particle filters. Though the conventional SIR particle filter is a popular technique for object tracking, it has been pointed out that the conventional particle filter has some disadvantages in practical applications such as its low tracking performance for multiple targets due to the degeneracy problem. In this paper, the new combined particle filters consisting of a low-resolution MCMC particle filter and a high-resolution SIR particle filter is proposed. Simultaneous tracking experiments for multiple moving targets are successfully carried out and it is verified that the combined particle filters has higher performance than the conventional particle filters in terms of the number of particles, the processing speed, and the tracking performance for multiple targets.

In this paper, we propose a method for estimating meaningful actions from long-term observation of everyday manipulation tasks without prior knowledge as part of an action understanding framework for life support robotic systems. The target task is defined as a sequence of interactions between objects. An interaction that appears many times is assumed to be meaningful and repetitious relative motion patterns are detected from trajectories of multiple objects. The main contribution is that the problem is formulated as a combinatorial optimization problem with two parameters, target object labels and correspondences on similar motion patterns, and is solved using local and global Dynamic Programming (DP) in polynomial time O(N logN), where N is a total amount of data. The proposed method is evaluated against manipulation tasks using everyday objects such as a cup and a tea-pot.

The Chan-Vese level set algorithm has been successfully applied to segmentation of images on Cartesian coordinate meshes, including ordinary planar images. In this paper we present a Chan-Vese model for segmentation of images on polar coordinate meshes, such as topography and remote sensing images. The image segmentation is accomplished by formulating the associated evolution equation in the polar coordinate system and then numerically solving the partial differential equation on an overset grid system called the Yin-Yang grid, which is free from the problem of singularity at the poles. We include examples of segmentations of real earth data that demonstrate the performance of our method.

In medical diagnostic imaging, an X-ray CT scanner or a MRI system have been widely used to examine 3D shapes or internal structures of living organisms or bones. However, these apparatuses are generally very expensive and of large size. A prior arrangement is also required before an examination, and thus, it is not suitable for an urgent fracture diagnosis in emergency treatment. This paper proposes a method to estimate a patient-specific 3D shape of a femur from only two fluoroscopic images using a parametric femoral model. Firstly, we develop a parametric femoral model by statistical analysis of a number of 3D femoral shapes created from CT images of 51 patients. Then, the pose and shape parameters of the parametric model are estimated from two 2D fluoroscopic images using a distance map constructed by the level set method. Experiments using synthesized images and fluoroscopic images of a phantom femur are successfully carried out and the usefulness of the proposed method is verified.

This paper introduces a new motion capture system for recovering 3D models of multiple persons separately and robustly against occlusion in real-time. Various markerless motion capture systems using video cameras have been proposed so far. However, in case that there are multiple persons in the scene at the same time, it is quite difficult to reconstruct a precise 3D model of each person separately due to the occlusion between them. To deal with this problem, the fast level set method is utilized in the proposed system for integrating stereo range data which is captured by multiple stereo cameras located around the target people. To reconstruct precise 3D models in real-time, the proposed system is implemented on a PC cluster with seven PCs and four stereo cameras. Tracking experiment of multiple persons and real-time reconstruction of 3D human models using the proposed system are successfully carried out

The inertia matching ellipsoid (IME) is proposed as a new index of dynamic performance for serial-link robotic manipulators. The IME integrates the existing dynamic manipulability and manipulating-force ellipsoids to achieve an accurate measure of the dynamic torque-force transmission efficiency between the joint torque and the force applied to a load held by an end-effector. The dynamic manipulability and manipulating-force ellipsoids can both be derived from the IME as limiting forms, with respect to the weight of the load. The effectiveness of the IME is demonstrated numerically through the selection of an optimal leg posture for jumping robots and optimal active stiffness control, and experimentally through application to a pick-up task using a commercial manipulator. The index is also extended theoretically to the case of a manipulator mounted on a free-flying satellite

This paper presents a new methodology for generating a straight legged walking pattern for a biped robot utilizing up-and-down motion of an upper body. Firstly, we define two new indexes, the knee stretch index (KSI) and the knee torque index (KTI), which indicate how efficiently the knee joints are utilized. Next, up-and-down motion of the upper body is automatically planned so that these indexes are optimized and straight legged walking is realized. The basic idea of the proposed method is: i) when a large number of DOFs of motion are required for controlling the ZMP, a robot makes its body height lower; ii) when there is a extra number of DOFs of motion, the body is lifted and the knee joint is stretched. By stretching the knee joints, human-like natural walking motion is obtained. Moreover, energy efficiency is improved since required torque and energy consumption to support the body weight become small at knee joints. The effectiveness of the proposed method is demonstrated by computer simulation and experiments using a humanoid robot, HOAP-1.

This work presents a new method of combining dynamic control of LOD and conservative occlusion culling based on a new hierarchical data structure of polygons. Our method is effective for rendering a large amount of data in complex environments.

Scanning from the air is one of the most efficient methods for obtaining 3D data of large-scale objects. For this purpose, we have been developing a flying laser range finder that is suspended under a balloon. Even though the scanning speed of the finder is quite rapid, it is difficult to eliminate the influence of the swing of a balloon. As a result the scanned data have some distortion due to the intra-scanning movement. In order to compensate this intra-scanning movement, we propose a evolutional registration algorithm which not only aligns multiple range images to determine inter-scanning movement parameters, but also rectifies distortion of range image by determining intra-scanning movement parameters. In this paper, we describe our aerial scanning system especially focusing on the design of the flying laser range finder and deformation registration algorithm. To show the effectiveness of our method, we evaluate its performance using synthesized and real data.

We propose a new index for dynamic performance analysis of serial link manipulators named impedance matching ellipsoid, or IME. Several indexes have been proposed for indicating static and dynamic performance of robot manipulators. For example, dynamic manipulability ellipsoid (DME) characterizes distributions of hand acceleration produced by normalized joint torque. manipulating-force ellipsoid (MFE) denotes static torque-force transmission efficiency from actuators at joints to a hand. On the other hand, the proposed IME characterizes dynamic torque-force transmission efficiency from actuators at joints to a load held at the hand of the manipulator. The IME includes a wide range of concepts proposed so far as measures of manipulator's performance. The DME and the MFE are both derived from the IME as limiting forms about the load mass. In this paper, we demonstrate the IME with some numerical examples including the selection of an optimal leg posture for jump robots, optimum active stiffness control, and an extension for manipulators mounted on satellites in outer space.

From 1970's, legged robots have attracted much attention of many researchers. In spite of this, it has been regarded that dynamically stable walking is very difficult to be tackled for any types of legged robots. For a trot gait for quadruped walking robots, we have proposed "the sway compensation trajectory". This method utilizes a lateral, longitudinal, and vertical motion of a robot body to keep a zero moment point (ZMP) on a diagonal line between support legs. In this paper, we develop the sway compensation trajectory for a biped robot, and show that dynamically stable walking is realized. This method makes it quite easy to design stable ZMP and COG (center of gravity) trajectories, which have been regarded as a very complicated and delicate problem. The effectiveness of the proposed method is verified through computer simulations and walking experiments by a humanoid robot, HOAP-1.

Though a legged robot has high terrain adaptability as compared with a wheeled vehicle, its moving speed is considerably low in general. For attaining a high moving speed with a legged robot, a dynamically stable gait, such as running for a biped robot and a trot gait or a bound gait for a quadruped robot, is a promising solution. However, the energy efficiency of the dynamically stable gait is generally lower than the efficiency of the stable gait such as a crawl gait. In this paper, we present an experimental study on the energy efficiency of a quadruped walking vehicle. Energy consumption of two walking patterns for a trot gait is investigated through experiments using a quadruped walking vehicle named TITAN-VIII. The obtained results show that the 3D sway compensation trajectory proposed in our previous paper [R. Kurazume et al., 2002] has advantages in view of energy efficiency as compared with the original sway compensation trajectory.

We propose a method which refines the range measurement of range finders by computing correspondences of vertices of multiple range images acquired from various viewpoints. Our method assumes that a range image acquired by a laser rangefinder has anisotropic error distribution which is parallel to the ray direction. Thus, we find the corresponding points of range images along with the ray direction. We iteratively converge range images to minimize the distance of corresponding points. We demonstrate the effectiveness of our method by presenting the experimental results of artificial and real range data. Also, we show that our method refines a 3D shape more accurately as opposed to that achieved by using the Gaussian filter.

We propose a new trajectory planning for the stable trot gait, named 3D sway compensation trajectory, and show that this trajectory has lower energy consumption than the conventional sway trajectory proposed previously by Yoneda et al. (1995). As the adaptive attitude control method was used during the 2-leg supporting phase of the trot gait, we consider four methods: 1) rotation of body along the diagonal line between supporting feet; 2) translation of body along the perpendicular line between supporting feet; 3) vertical swing motion of recovering legs; and 4) horizontal swing motion of recovering legs. The stabilization efficiency of each method was verified through computer simulation and the damping experiment using a quadruped walking robot, TITAN-VIII. Furthermore, the dynamic trot gait control that combines the feedforward control based on the proposed 3D sway compensation trajectory and the adaptive feedback control using body translation and vertical motion of swing legs was developed, and the walking experiment on rough terrain using TITAN-VIII was carried out.

Walking robots have high adaptability for terrain variation. Mobile robots that perform hazardous tasks such as mine detection or the inspection of an atomic power plant are typically controlled by operators from distant places. For a remote operation system, use of visual information from a camera mounted on a robot body is very useful. However, unlike wheeled vehicles, the camera mounted on the walking robot oscillates because of the impact of walking, and the obtained unstable images cause inferior operation performance. In this paper, we introduce an image stabilization system for remote operation of walking robots using a high speed CCD camera and gyrosensors. The image stabilization is executed in two phases, that is, the estimation of the amount of oscillation by the combination of the template matching method and gyrosensors, and change of the display region. Pentium MMX instruction is used for template matching calculation, and the estimated amount of oscillation is outputted every 12 msec. Furthermore, developed image stabilization mechanism can be used an external attitude sensor from the visual information, and the damping control of the robot body while walking is also possible. Experimental results showed stabilized images that eliminates the oscillation component are taken even when the robot moves dynamically or in long distance, and verified that the performance of attitude control using the developed image stabilization system is almost same as the case using an attitude sensor.

This paper proposes a new method called "cooperative positioning system" (CPS) for mobile robots position identification. The main concept of CPS is to divide the robots into two groups, A and B respectively Group A remains stationary and acts as a landmark while group B is moving; group B then stops and acts as a landmark for group A. This process is repeated until the target position is reached. Compared with dead reckoning, CPS has a far lower accumulation of positioning error, and can also work in three-dimensions. Furthermore, CPS employs inherent landmarks and therefore can be used in uncharted environments unlike the landmark method. This paper focuses on the the relationship between moving configurations of CPS and its positioning accuracy for the latest prototype CPS model, CPS-III, using simulation and analytical techniques. Optimum moving strategies in order to minimize the positioning error are then discussed and verified through experiments.

Several position identification methods have been used for mobile robots. We propose a new method called "cooperative positioning system (CPS)". For CPS, we divide the robots into two groups, A and B. The group A remains stationary and acts as a landmark while group B is moving. Group B then stops and acts as a landmark for group A. This "dance" is repeated until the target position is reached. By using the concept of "portable landmarks", CPS has a far lower accumulation of positioning error than dead reckoning, and can work in three-dimensions which is not possible with dead reckoning; it can therefore work in uncharted environments. In this paper we outline the second prototype CPS machine model (CPS-II) and report the results of position identification experiments. Experimental results using this model give a positioning accuracy of o.4% for position and 1.0 degree for attitude.

A number of positioning identification techniques have been used for mobile robots. Dead reckoning is a popular method, but is not reliable when a robot travels long distances or over an uneven surface because of variations in wheel diameter and wheel slippage. The landmark method, which estimates the current position relative to landmarks, cannot be used in an uncharted environment. The authors propose a new method called "cooperative positioning with multiple robots". For cooperative positioning, the authors divide the robots into two groups, A and B. One group, say A, remains stationary and acts as a landmark while group B moves. The moving group B then stops and acts as a landmark for group A. This "dance" is repeated until the target robot position are reached. Cooperative positioning has a far lower accumulated positioning error than dead reckoning, and can work in three-dimensions which is not possible with dead reckoning. Also, this method has inherent landmarks and therefore works in uncharted environments. This paper discusses the positioning accuracy of the authors' method with error variances for an example with three mobile robots.>

The authors present a basic formulation of motion dynamics of a free-floating rigid-link system to establish a basis of the collision dynamics. They propose a novel concept named extended generalized inertia tensor (Ex-GIT), which is an extended version of the GIT for ground-based arms, and discuss the virtual mass concept. By means of these concepts, they formulate the collision problem focusing on a velocity relationship just before and after the collision without sensing the impact force, but considering the momentum conservation law.>

The control problem of multiple manipulators installed on a free-flying space robot is presented. Kinematics and dynamics are studied and the generalized Jacobian matrix is formulated for the motion control of a multiarm system. Individual and coordinated control of dual manipulators is discussed. For the coordinated operation, a new method of controlling two arms simultaneously-one arm traces a given path, while the other arm works both to keep the satellite attitude and to optimize the total operation torque of the system-is developed. By means of this control method, an interesting torque optimum behavior is observed and a practical target capture operation is exhibited by computer simulation.>

Presents a coordinated control of multiple manipulators in space free-floating robots. The authors firstly develop the generalized Jacobian matrix and equation of motion for a space robot with multiple arms, then propose a new control method optimising the sum of squared control torque in the sense of local (instantaneous) minimization by means of redundancy. The method is applied to realistic models installing a mission arm and reaction wheels, and also a redundant arm. Through the simulation study, this paper shows that the installation and utilization of the redundant arm has great effectiveness in terms of reducing the burden of reaction wheels for satellite attitude control.>