In this study, we present a method for all-around depth estimation from multiple omnidirectional images for indoor environments. In particular, we focus on plane-sweeping stereo as the method for depth estimation from the images. We propose a new icosahedron-based representation and ConvNets for omnidirectional images, which we name "CrownConv" because the representation resembles a crown made of origami. CrownConv can be applied to both fisheye images and equirect- angular images to extract features. Furthermore, we propose icosahedron-based spherical sweeping for generating the cost volume on an icosahedron from the extracted features. The cost volume is regularized using the three-dimensional CrownConv, and the final depth is obtained by depth regression from the cost volume. Our proposed method is robust to camera alignments by using the extrinsic camera parameters; therefore, it can achieve precise depth estimation even when the camera alignment differs from that in the training dataset. We evaluate the proposed model on synthetic datasets and demonstrate its effectiveness. As our proposed method is computationally efficient, the depth is estimated from four fisheye images in less than a second using a laptop with a GPU. Therefore, it is suitable for real-world robotics applications. Our source code is available at https://github.com/matsuren/crownconv360depth.

In this paper, we propose a novel motion planning method for off-road unmanned ground vehicles, based on three-dimensional (3-D) terrain map information. Previous studies on the motion planning of a vehicle traveling on rough terrain dealt only with a relatively small environment. Furthermore, unique vehicle characteristics were not considered, and it was also impossible to incorporate regulations, such as maintaining driving speed and suppressing posture change. The proposed method enables vehicles to adaptively generate a path by considering vehicle characteristics and the regulations, in a large-scale environment, with rough terrain. A random sampling based scheme was applied to carrying out global path planning, based on a 3-D environmental model. Experimental results showed that the proposed off-road motion planner could generate an appropriate path, which satisfies vehicle characteristics and predefined regulations.

We propose a prediction method for lane changes in other vehicles. According to previous research, over 90 % of car crashes are caused by human mistakes, and lane changes are the main factor. Therefore, if an intelligent system can predict a lane change and alarm a driver before another vehicle crosses the center line, this can contribute to reducing the accident rate. The main contribution of this work is to propose a new feature describing the relationship of a vehicle to adjacent vehicles. We represent the new feature using a dynamic characteristic potential field that changes the distribution depending on the relative number of adjacent vehicles. The new feature addresses numerous situations in which lane changes are made. Adding the new feature can be expected to improve prediction performance. We trained the prediction model and evaluated the performance using a real traffic dataset with over 900 lane changes, and we confirmed that the proposed method outperforms previous methods in terms of both accuracy and prediction time.

In this study, we developed a human attention model for smooth human-robot interaction. The model consists of the saliency map generation module and manipulation map generation module. The manipulation map describes top-down factors, such as human face, hands and gaze in the input image. To evaluate the proposed model, we applied the model to a magic video, and measured human gaze points during watching the video. Based on the experimental results, the proposed model can better explain human attention than the saliency map.

For smooth interaction between human and robot, the robot should have an ability to manipulate human attention and behaviors. In this study, we developed a visual attention model for manipulating human attention by a robot. The model consists of two modules, such as the saliency map generation module and manipulation map generation module. The saliency map describes the bottom-up effect of visual stimuli on human attention and the manipulation map describes the top-down effect of face, hands and gaze. In order to evaluate the proposed attention model, we measured human gaze points during watching a magic video, and applied the attention model to the video. Based on the result of this experiment, the proposed attention model can better explain human visual attention than the original saliency map.

A teleoperation system for a crane is developed. The crane uses a small actuator and large actuator in each direction (x and y). The large actuator is controlled by input signals from a laser pointer interface, and the small actuator is used as a regulator and is feedback-controlled by a wire angle sensor. To avoid overshoots in the suspended object at the target point and to eliminate the effect of the trembling of the irradiated light by the laser pointer, the velocity commands sent to the large actuator are designed by accumulating cubic functions, each of which is based on the displacement of the irradiated light in a sampling period. The effectiveness of the developed control system and its control algorithm were verified through experimental results.

In order for robots to safely move in human-robot coexisting environment, they must be able to predict their surrounding people's behavior. In this study, a pedestrian behavior model that produces humanlike behavior was developed. The model takes into account the pedestrian's intention. Based on the intention, the model pedestrian sets its subgoal and moves toward the subgoal according to virtual forces affected by other pedestrian and environment. The proposed model was verified through pedestrian observation experiments.

To achieve a balanced walking for a humanoid, it is necessary to estimate the dynamic stability of the system. However, in a small size humanoid with restricted system resource, it is hard to satisfy the performance level desired by dynamics analysis. Therefore, in this paper, we propose the feasible methods to generate gait pattern and stabilize walking based on coupled oscillators which have a clear correlation between oscillator parameters and system dynamics without a real time ZMP calculation. The proposed method was tested on the open humanoid platform DARwIn-OP for the evaluation, and the result showed that a real time gait pattern generation and stabilization were realized.

Processors have become multi-core, and transactional memory has been getting attention as a new concurrency control and coordination mechanism for shared data in this type of processor. Transactional memory is a memory management technology based on transactions used in the field of databases. An atomic action provided by transactional memory is a very effective programming abstraction not only for concurrency process control but also for system recovery. However, the previous research on transactional memory in an embedded system requires dedicated processor with hardware-assistance, not a generic middle-range or lower processor, mostly for concurrency control. In order to deploy transactional memory into embedded systems more widely, for a generic middle-range or lower processor without hardware-assistance we developed a transactional memory system for embedded systems from sensor and controller nodes to consumer electronics (STEM). Moreover the purposes of STEM are not only for shared data access control but also for partial recovery of the system. We designed and implemented an STEM prototype, and we demonstrate that the design and implementation is effective in practical usage.

In order for service robots to safely coexist with humans, collision avoidance with humans is the most important issue. On the other hand, working efficiencies are also important and cannot be ignored. In this paper, we propose a method to estimate a pedestrian's behavior. Based on the estimation, we realize smooth collision avoidances between a robot and a human. A robot detects pedestrians by using a laser range finder and tracks them by a Kalman filter. We apply the social force model to the observed trajectory for a determination whether the pedestrian intends to avoid a collision with the robot or not. The robot selects an appropriate behavior based on the estimation results. We conducted experiments that a robot and a person pass each other. Through the experiments, the usefulness of the proposed method was demonstrated.

We have proposed “attentive workbench (AWB),” an assembly cell that supports the production activities of human workers. Attentive Workbench is composed of an augmented desk interface and self-moving trays with the Sawyer planar motors. In this paper, real assembly experiments using the implemented AWB system are carried out. The merit of the physical assembly support by the proposed system is evaluated quantitatively in the view of necessary time for product assembly.

We aim to realize an intelligent robotic system that delivers necessary objects to human desk-workers in home or office environments. In order to meet this goal, we propose methods to understand a request for support of a user of the system. We adopt an approach that the system measures the userpsilas hand and eye movements in reaching for an object to detect and understand the request. Based on the detection and understanding of the request, the system delivers necessary objects to the user using self-moving trays. The experimental results demonstrate the advantage of the methods proposed here.

We aim to realize a robotic system that hands over a necessary object to a user as soon as he/she attempts to reach out his/her hand for it. In order to realize such system, the following is required: 1) detection of the reaching movements, 2) prediction of the target object among multiple objects, and 3) handing-over the object to a user. In this paper, the first two of them are discussed. To detect the reaching movements, we use the speed of a hand movement, the smoothness of the hand trajectory, and the relation between a position of hand and a gaze point. To predict the target object, we use a probabilistic model based on the user's hand movements and his/her gaze direction. The proposed system can detect reaching movements and predict the target object at a very early stage. A description of the experimental results demonstrates the advantage of the method proposed here.

We have proposed "attentive workbench (AWB)," a deskwork support system. Using cameras, projectors, and self-moving trays, the system supports human user from both physical and informational aspects. In this paper, an interface between the user and AWB based on pointing gestures is developed. Considering the usage of AWB, a simple and reliable interface with quick response is implemented. Deskwork support using the present interface and real self-moving trays are demonstrated

We aim to realize a robotic system that hands over a necessary object to a user as soon as he/she attempts to reach out for it. In order to realize such system, the following is required: 1) detection of the reaching movement, 2) prediction of the target object among multiple objects, and 3) handing over the object to a user. In this paper, the first two of them are described and discussed. We apply the smoothness and the speed of the hand movement to distinguish whether the movement is for reaching or not. And in order to predict the target object, we define the certainty according to the relative movement of the hand to each object. To evaluate the performance of the proposed method, we compare the method with the minimum jerk model based approach. A description of the experimental results demonstrates the usefulness of the method proposed here

We propose “Attentive Workbench (AWB),” a new cell production system in which an intelligent system supports human workers. Using cameras, projectors, planar motor driven self-moving trays and other devices, the system supports workers from both physical and information aspects, recognizing worker’s condition and intention. This paper outlines AWB and deals with physical assembly support using self-moving parts trays. Two different schemes, centralized and decentralized, for controlling multiple parts trays are evaluated through experiments and simulations. The centralized control scheme is found to have the higher performance than the decentralized control scheme.

We have proposed "attentive workbench (AWB)", a new cell production system in which an intelligent system supports human workers. Recognizing worker's condition and intention, the system supports workers from both physical and information aspects. This paper deals with physical assembly support using self-moving parts trays. The system delivers necessary assembly parts to workers and clears finished products quickly. On transporting large products, multiple trays are subjected to form a rigid body working as a large single tray. In order to realize this arrangement of trays in real-time, a planning method based on the priority scheme and heuristic rule is proposed. The present method is evaluated through simulations. A demonstration of assembly support using real self-moving trays is shown.

We have proposed "attentive workbench (AWB)," a deskwork support system. Using cameras, projectors, and self-moving trays, the system supports human from both physical and informational aspects. This paper deals with the problem for estimating the object that a user needs. We present one of the approaches for accurate estimation of a necessary object from the user's pointing gesture. The system arranges the self-moving trays by itself based on the user's past action sequences. Using force-directed method, good placement, which decrease false estimations and is not moved very much from the initial placement, can be obtained. A description of the experimental results demonstrates the usefulness of the method proposed here.

This paper presents a robot system for handling various objects in home or office environments. A fixed manipulator utilizes marks on objects for handling and motion planning. A mark consists of two parts. One is the outer part, which indicates its pose (in this paper, "pose " denotes 3D position and orientation), and the other is a QR code, which is a kind of 2D barcode. A QR code is stored information of an object (e.g. its name). A user attaches several marks on each object. The manipulator accesses the information using a camera in its hand. The robot can decide its complex handling motion based on the information in QR codes, pose of the objects, which are estimated from pose of marks, and signals from proximity sensors. Experiments are conducted to verify the whole system.