Reinforcement learning for hard-exploration tasks remains challenging due to the long-term dependence and sparse-and-delay rewards in complex environments. In these challenging tasks, intrinsic motivation has become a dominant paradigm to enable the agent to explore the environment when no external reward feedback is available. In this work, inspired by studies from the human memory mechanism, we present a mnemonic dictionary learning (MDL) model for intrinsic motivation in reinforcement learning. The MDL model leverages sparse dictionary learning to incremental abstract the exploration histories into a compact memory-like dictionary, providing an excellent intrinsic motivation model. This mnemonic dictionary model not only drives the agent to explore novel stats in the environments indicated by the memory reconstruction error but also helps the agent to remember the key states and structure of the environments using its learned bases and reconstruction coefficients. The proposed MDL model can serve as a generative module for existing exploration methods. Extensive experimental results on typical sparse-reward tasks demonstrate its effectiveness and applicability over several competing algorithms. We will release the source code and trained models to facilitate further studies in this research direction.

Recent years have witnessed an increasing application of deep reinforcement learning (DRL) on video games. While deeper and wider neural networks have played a crucial role in computer vision and natural language processing, such capacity remain under-explored in most DRL works. Under the fact that feature propagation together with large networks contributes to learning a good representation, we propose an end-to-end Large Feature Extractor Network (LFENet) that uses large neural networks with dense connections to train a high-capacity encoder. Even though the increased dimensionality of input is usually thought to result in poor performance for RL agents, we introduce the information bottleneck to alleviate the problem. Finally, we combine LFENet with Proximal Policy Optimization (PPO) algorithm. Through numerical experiments on Atari 2600 video games, we demonstrate our method matches or outperforms state-of-the-art algorithms.

Recently, model-based agents have achieved better performance than model-free ones using the same computational budget and training time in single-agent environments. However, due to the complexity of multi-agent systems, it is tough to learn the model of the environment. The significant compounding error may hinder the learning process when model-based methods are applied to multi-agent tasks. This paper proposes an implicit model-based multi-agent reinforcement learning method based on value decomposition methods. Under this method, agents can interact with the learned virtual environment and evaluate the current state value according to imagined future states in the latent space, making agents have the foresight. Our approach can be applied to any multi-agent value decomposition method. The experimental results show that our method improves the sample efficiency in different partially observable Markov decision process domains.

Search based software engineering (SBSE) is an evolving field where meta-heuristic techniques are applied to solve many software engineering problems. One area of SBSE, where considerable research is underway, is software testing. We see much application of meta-heuristics search techniques for generating input test data. But most of the work in this area is concentrated on test data generation from source code. We see very little application of such techniques to testing from other sources such as requirement and design models. Zhan and Clark applied such techniques to generate test data for Simulink models. This paper extends the work of Zhan and Clark by investigating the application of genetic algorithms (GAs) to Simulink models and then statistically compares the results to the existing work, which is mainly based on simulated annealing (SA).

Search based test-data generation has proved successful for code-level testing. In this paper we investigate the application of such approaches at the higher levels of abstraction offered by Matlab-Simulink models. The presence of persistent state has been shown to be problematic at the code level and such difficulties remain when Matlab-Simulink models are to be tested. In such cases, sequences of inputs that can put the model under test into particular states are needed to enable the underlying test goals to be achieved. Simple search guidance appears to be insufficient and results in a 'flat' cost function landscape. To address this problem, we introduce a technique called tracing and deducing, which helps provide better guidance to the search, allowing our developed tools to home in on the targeted test-data.

The efficient and effective generation of test-data from high-level models is of crucial importance in advanced modern software engineering. Empirical studies have shown that mutation testing is highly effective. This paper describes how search-based automatic test-data generation methods can be used to find mutation adequate test-sets for Matlab/Simulink models.