A major challenge in reinforcement learning is the design of exploration strategies, especially for environments with sparse reward structures and continuous state and action spaces. Intuitively, if the reinforcement signal is very scarce, the agent should rely on some form of short-term memory in order to cover its environment efficiently. We propose a new exploration method, based on two intuitions: (1) the choice of the next exploratory action should depend not only on the (Markovian) state of the environment, but also on the agent’s trajectory so far, and (2) the agent should utilize a measure of spread in the state space to avoid getting stuck in a small region. Our method leverages concepts often used in statistical physics to provide explanations for the behavior of simplified (polymer) chains in order to generate persistent (locally self-avoiding) trajectories in state space. We discuss the theoretical properties of locally self-avoiding walks and their ability to provide a kind of short-term memory through a decaying temporal correlation within the trajectory. We provide empirical evaluations of our approach in a simulated 2D navigation task, as well as higher-dimensional MuJoCo continuous control locomotion tasks with sparse rewards.

We present the first differentially private algorithms for reinforcement learning, which apply to the task of evaluating a fixed policy. We establish two approaches for achieving differential privacy, provide a theoretical analysis of the privacy and utility of the two algorithms, and show promising results on simple empirical examples.

In this paper, we propose an efficient mechanism dealing with trust assessment for agent societies, aiming to accurately assess the trustworthiness of the collaborating agents. In the proposed model, we formulate on-line assessment process, which applies a number of measurements affecting the evaluator agent's estimated value regarding to the evaluated agent's trust. We then represent the off-line trust assessment, which compares and adjusts the involved features with the actual performance of the evaluated agent. The off-line process updates the belief set of the evaluator agents in the sense that it adapts the agents with respect to the changes in the network. In this paper, the proposed framework is described, a theoretical analysis of its assessment and its implementation along with simulations comparison with other models are provided. We also show how our model is more efficient than the existing models, particularly in very dynamic environments.

Security is a substantial concept in multi-agent systems where agents dynamically enter and leave the system. Different models of trust have been proposed to assist agents in deciding whether to interact with requesters who are not known (or not very well known) by the service provider. To this end, in this paper we progress our work on security for agent-based systems, which is embedded in service provider’s trust evaluation of the counter part. Agents are autonomous software equipped with advanced communication (using public dialogue game-based protocols and private strategies on how to use these protocols) and reasoning capabilities. The service provider agent obtains reports provided by trustworthy agents (regarding to direct interaction histories) and referee agents (in the form of recommendations) and combines a number of measurements, such as number of interactions and timely relevance, to provide an overall estimation of a particular agent’s likely behavior. Requesting this agent, called the target agent, to provide the number of interactions it had with each agent, the service provider penalizes the agents who lied about having information for trust evaluation process. In addition, after a periodic time, the actual behavior of the target agent is compared against the information provided by others. This comparison leads to both adjusting the credibility of the contributing agents in trust evaluation and improving the system trust evaluation by minimizing the estimation error. Overall the proposed framework is shown to assist agents effectively perform the trust estimation of interacting agents.