Model-based reinforcement learning is an effective approach to reducing sample complexity by adding more data from the model. Dyna is a well-known architecture that contains model-based reinforcement learning and integrates learning from interactions with an environment and a model of the environment. Although the model can greatly help to speed up the agent's learning, acquiring an accurate model is a hard problem in spite of the recent great success of function approximation using neural networks. A wrong model causes degradation of the agent's performance and raises another question: to which extent should an agent rely on the model to update its policy? In this paper, we propose to use the confidence of the model simulations to the integrated learning process so that the agent avoids updating its policy based on uncertain simulations by the model. To obtain confidence, we apply the Monte Carlo dropout technique to the state transition model. We show that this approach contributes to improving early-stage training, thus helping speed up the agent to reach reasonable performance. We conduct experiments on simulated robotic locomotion tasks to demonstrate the effectiveness of our approach.

Despite the recent success of deep reinforcement learning (RL), domain adaptation remains an open problem. Although the generalization ability of RL agents is critical for the real-world applicability of Deep RL, zero-shot policy transfer is still a challenging problem since even minor visual changes could make the trained agent completely fail in the new task. To address this issue, we propose a two-stage RL agent that first learns a latent unified state representation (LUSR) which is consistent across multiple domains in the first stage, and then do RL training in one source domain based on LUSR in the second stage. The cross-domain consistency of LUSR allows the policy acquired from the source domain to generalize to other target domains without extra training. We first demonstrate our approach in variants of CarRacing games with customized manipulations, and then verify it in CARLA, an autonomous driving simulator with more complex and realistic visual observations. Our results show that this approach can achieve state-of-the-art domain adaptation performance in related RL tasks and outperforms prior approaches based on latent-representation based RL and image-to-image translation.

Motivation: Multi-dimensional NMR spectra are generally used for NMR signal assignment and structure analysis. There are several programs that can achieve highly automated NMR signal assignments and structure analysis. On the other hand, NMR spectra tend to have a large number of noise peaks even for data acquired with good sample and machine conditions, and it is still difficult to eliminate these noise peaks. Results: We have developed a method to eliminate noise peaks using convolutional neural networks, implemented in the program package Filt_Robot. The filtering accuracy of Filt_Robot was around 90-95% when applied to 2D and 3D NMR spectra, and the numbers of resulting non-noise peaks were close to those in corresponding manually prepared peaks lists. The filtering can strongly enhance automated NMR spectra analysis. Availability and implementation: The full package of the program, documents and example data are available from http://bmrbdep.pdbj.org/en/nmr_tool_box/Filt_Robot.html. Supplementary information: Supplementary data are available at Bioinformatics online.

A new type of stepping motor, named electric nutation motor was developed. The aim of this research is to develop a new stepping motor with high torque. Based on the magnetic field analysis using finite element method, theoretical value of the electromagnetic force was obtained. Magnetic path was designed and optimized. The force of the electromagnet was 60 N with the current of 0.3 A. The motor was found to have the maximum torque of 0.64 N/spl middot/m, the maximum speed of 4.3 rpm, and positioning accuracy of 1,248 step/rev.

When a fabrication process is renewed, of practical importance is how to make the best use of layout resources so far accumulated for old fabrication processes. The present paper describes an automatic recycling system for layout data once used dedicatedly for functional cells of the standard-cell level. The main process of this system is i) to transform given layout data into a layout description format expressed in parameters associated with shapes, sizes, and locations of layout elements, and then ii) to resynthesize an optimal layout in accordance with a new set of design rules by means of a graph theoretic linear programming approach. A part of implementation results is also shown.