The deployment of Reinforcement Learning to robotics applications faces the difficulty of reward engineering. Therefore, approaches have focused on creating reward functions by Learning from Observations (LfO) which is the task of learning policies from expert trajectories that only contain state sequences. We propose new methods for LfO for the important class of continuous control problems of learning to stabilize, by introducing intermediate proxy models acting as reward functions between the expert and the agent policy based on Lyapunov stability theory. Our LfO training process consists of two steps. The first step attempts to learn a Lyapunov-like landscape proxy model from expert state sequences without access to any kinematics model, and the second step uses the learned landscape model to guide in training the learner's policy. We formulate novel learning objectives for the two steps that are important for overall training success. We evaluate our methods in real automobile robot environments and other simulated stabilization control problems in model-free settings, like Quadrotor control and maintaining upright positions of Hopper in MuJoCo. We compare with state-of-the-art approaches and show the proposed methods can learn efficiently with less expert observations.

There are two major challenges for scaling up robot navigation around dynamic obstacles: the complex interaction dynamics of the obstacles can be hard to model analytically, and the complexity of planning and control grows exponentially in the number of obstacles. Data-driven and learning-based methods are thus particularly valuable in this context. However, data-driven methods are sensitive to distribution drift, making it hard to train and generalize learned models across different obstacle densities. We propose a novel method for compositional learning of Sequential Neural Control Barrier models (SN-CBFs) to achieve scalability. Our approach exploits an important observation: the spatial interaction patterns of multiple dynamic obstacles can be decomposed and predicted through temporal sequences of states for each obstacle. Through decomposition, we can generalize control policies trained only with a small number of obstacles, to environments where the obstacle density can be 100x higher. We demonstrate the benefits of the proposed methods in improving dynamic collision avoidance in comparison with existing methods including potential fields, end-to-end reinforcement learning, and model-predictive control. We also perform hardware experiments and show the practical effectiveness of the approach in the supplementary video.

In multi-agent delivery tasks, Dijkstra's Algorithm, which identifies and plans the shortest path based on costs, is very popular. Specifically, using dynamic path costs, namely updating each path cost to reflect the number of agents having visited it over time, is an efficient method for accomplishing multi-agent delivery tasks. Indeed, it can consider the following logic: “a path that is jammed is not likely to be selected in the routing process”. Suppose tasks that can train with similar delivery tendencies in advance, optimizing the update path cost rule set would lead to more efficient agent path planning. Indeed, it would consider that “the paths that will be jammed are not likely to be selected in the routing process”. Thus, we propose a dynamic path cost update method that reflects delivery tendencies. We design the update path cost if-then rule (what path costs should be added, and under what conditions?) as a current partial agent distribution around an observation point for the if-part, and the amount of added (or subtracted) path cost for the then-part. In one delivery task, each observation point is tied to partial agents' distribution. If a distribution is collated with an update rule's if-part, the corresponding path of the observation point is updated given the update path costs that the collated rule's then-part indicates. This update continues during a task at each observation point at an interval. The set of update path cost if-then rules is optimized with Learning Classifier Systems. We evaluated our method with a multi-agent delivery task that simulates a warehouse swarm robot's Pickup and Delivery task. The results showed our method accomplished the tasks faster than a previous approach. In addition, we evaluated the superiority range of the proposed method when the number of agents used changes. Finally, we compared the path cost transition tendency. It revealed our method was more likely to drastically alter path costs.

Over the last few years, tumor detection with deep learning has attracted ample attention, and various models have been proposed. Some models obtain output images directly from end-to-end models. Other models consist of a complicated network structure. However, it is difficult for doctors to understand the detection procedures of these models. This difficulty hinders their practical implementation. In this paper, therefore, we propose a model that detects tumors in a way that is easy for doctors to understand. The proposed deep neural network model was inspired by a way that doctors interpret images. It consists of two processes: tumor position estimation, and tumor/non-tumor classification. To evaluate the proposed method, we used magnetic resonance images actually collected by a hospital. The output images reveal that the proposed method's procedure of tumor detection is consistent with the way that doctors detect tumors. Likely tumor positions are first estimated. Then, these likely positions are narrowed down to the correct tumor positions. Our proposed method detected tumors with 94.14% accuracy and 93.44% recall.

Considering the ease of data collection, it is desirable to build a voice conversion system (VCS) from non-parallel voice data, with different sentences read by the source and target speakers. Previous non-parallel VCSs have used either a mel-cepstrum or spectral envelope as an input feature. However, these features have different acoustic characteristics that play important roles in speaker recognition. Thus, we propose a non-parallel VCS that efficiently uses both mel-cepstrum and spectral envelopes as input features. Our method is based on three key strategies: 1) we use generative adversarial networks for voice conversion; 2) we add noise to facilitate the training of the formant part; and 3) we integrate the acoustic features to generate high-quality converted voices. Subjective evaluations in the Voice Conversion Challenge 2016 (VCC 2016) revealed that our model outperformed the previous approaches in terms of the naturalness and similarity of the converted voice.

In recent years, significant research has been conducted on grasp planning for multifingered robot hands. These studies have focused on determining how to obtain suitable grasps from among an infinite number of candidate grasps. This domain's goal is a successful application to unknown environments through the adoption of the extracted grasps. Under difficult conditions, such as grasping a target object that is adjacent to other objects, manipulating robot hands by indicating grasping points has been insufficient. Instead, grasp strategies that construct movements using each finger's joint servo controls and robot hand movements should be used. In addition, it is necessary to automatically acquire various grasp strategies to apply to unknown environments. In this paper, we propose a method that automatically obtains grasp strategies using a real-coded genetic algorithm (RCGA), which is an evolutionary algorithm. This method derives grasp strategies by optimizing combinations and structures that consist of simple finger joint servo controls and robot hand movements. By applying our method to several objects on a simulator, we collected various grasp strategies capable of handling difficult conditions.