Multiagent deep reinforcement learning (MDRL) is a promising research area in which agents learn complex behaviors in cooperative or competitive environments. However, MDRL comes with several challenges that hinder its usability, including sample efficiency, curse of dimensionality, and environment exploration. Recent works proposing federated reinforcement learning (FRL) to tackle these issues suffer from problems related to model restrictions and maliciousness. Other proposals using reward shaping (RS) require considerable engineering and could lead to local optima. In this article, we propose a novel Blockchain-assisted multiexpert demonstration cloning (MEDC) framework for MDRL. The proposed method utilizes expert demonstrations in guiding the learning of new MDRL agents, by suggesting exploration actions in the environment. A model sharing framework on Blockchain is designed to allow users to share their trained models, which can be allocated as expert models to requesting users to aid in training MDRL systems. A Consortium Blockchain is adopted to enable traceable and autonomous execution without the need for a single trusted entity. Smart Contracts are designed to manage users and models allocation, which are shared using IPFS. The proposed framework is tested on several applications and is benchmarked against existing methods in FRL, RS, and imitation learning-assisted RL. The results show the outperformance of the proposed framework in terms of learning speed and resiliency to faulty and malicious models.

Deep Reinforcement Learning (DRL) has emerged as a powerful paradigm for solving complex problems. However, its full potential remains inaccessible to a broader audience due to its complexity, which requires expertise in training and designing DRL solutions, high computational capabilities, and sometimes access to pre-trained models. This necessitates the need for hassle-free services that increase the availability of DRL solutions to a variety of users. To enhance the accessibility to DRL services, this paper proposes a novel blockchain-based crowdsourced DRL as a Service (DRLaaS) framework. The framework provides DRL-related services to users, covering two types of tasks: DRL training and model sharing. Through crowdsourcing, users could benefit from the expertise and computational capabilities of workers to train DRL solutions. Model sharing could help users gain access to pre-trained models, shared by workers in return for incentives, which can help train new DRL solutions using methods in knowledge transfer. The DRLaaS framework is built on top of a Consortium Blockchain to enable traceable and autonomous execution. Smart Contracts are designed to manage worker and model allocation, which are stored using the InterPlanetary File System (IPFS) to ensure tamper-proof data distribution. The framework is tested on several DRL applications, proving its efficacy.

In target localization applications, readings from multiple sensing agents are processed to identify a target location. The localization systems using stationary sensors use data fusion methods to estimate the target location, whereas other systems use mobile sensing agents (UAVs, robots) to search the area for the target. However, such methods are designed for specific environments, and hence are deemed infeasible if the environment changes. For instance, the presence of walls increases the environment’s complexity and affects the collected readings and the mobility of the agents. Recent works explored deep reinforcement learning (DRL) as an efficient and adaptable approach to tackle the target search problem. However, such methods are either designed for single-agent systems or for noncomplex environments. This work proposes two novel multiagent DRL models for target localization through search in complex environments. The first model utilizes proximal policy optimization, convolutional neural networks, Convolutional AutoEncoders to create embeddings, and a shaped reward function using breadth first search to obtain cooperative agents that achieve fast localization at low cost. The second model improves the first model in terms of computational complexity by replacing the shaped reward with a simple sparse reward, subject to the availability of Expert Demonstrations. Expert demonstrations are used in Demonstration Cloning, a novel method that utilizes demonstrations to guide the learning of new agents. The proposed models are tested on a scenario of radioactive target localization, and benchmarked with existing methods, showing efficacy in terms of localization time and cost, in addition to learning speed and stability.

Workers recruitment remains a significant issue in Mobile Crowdsourcing (MCS), where the aim is to recruit a group of workers that maximizes the expected Quality of Service (QoS). Current recruitment systems assume that a pre-defined pool of workers is available. However, this assumption is not always true, especially in cold-start situations, where a new MCS task has just been released. Additionally, studies show that up to 96% of the available candidates are usually not willing to perform the assigned tasks. To tackle these issues, recent works use Online Social Networks (OSNs) and Influence Maximization (IM) to advertise about the desired MCS tasks through influencers, aiming to build larger pools. However, these works suffer from several limitations, such as 1) the lack of group-based selection methods when choosing influencers, 2) the lack of a well-defined worker recruitment process following IM, 3) and the non-dynamicity of the recruitment process, where the workers who refuse to perform the task are not substituted. In this paper, an Influence- and Interest-based Worker Recruitment System (IIWRS), using OSNs, is proposed. The proposed system has two main components: 1) an MCS-, group-, and interest-based IM approach, using a Genetic Algorithm, to select a set of influencers from the network to advertise about the MCS tasks, and 2) a dynamic worker recruitment process which considers the social attributes of workers, and is able to substitute those who do not accept to perform the assigned tasks. Empirical studies are performed using real-life datasets, while comparingIIWRSwith existing benchmarks.