Mobile crowdsensing (MCS) is a distributed sensing architecture that utilizes existing sensors on mobile units (MUs) to perform sensing tasks. A mobile crowdsensing platform (MCSP) publishes the sensing tasks and the MUs decide if they want to participate in their execution in exchange for money. The MCS system is characterized by its dynamic nature in which the task requirements, the MUs’ availability, and their available resources change over time. The MUs aim to find an efficient task participation strategy to maximize their income while the MCSP focuses on maximizing the number of completed tasks. As optimal task participation strategies require perfect non-causal information about the MCS system, which is unavailable in realistic scenarios, the main challenge in MCS is to find an efficient task participation strategy for the MUs under incomplete information. To this aim, a novel fully decentralized federated deep reinforcement learning algorithm, termed FDRL-PPO is proposed. FDRL-PPO enables every MU to learn its own task participation strategy based on its experiences, available resources, and preferences, without relying on perfect non-causal information about the MCS system. To replenish their batteries, the MUs rely on energy harvesting. As a result, their available energy varies over time, leading to varying availability and fragmented learning experiences. To mitigate these challenges, the proposed approach leverages federated learning, enabling MUs to collaboratively improve their models without having to share private raw data like their own experiences. By exchanging only learned models, MUs collectively compensate for individual limitations, and find more scalable, robust, and efficient task participation strategies. Comprehensive evaluations on both synthetic and real-world datasets show that FDRL-PPO consistently outperforms benchmark algorithms in terms of task completion ratio, fairness in task completion, energy consumption, and number of conflicting proposals.

Status Update Systems (SUSs) are central components in applications like environmental sensing or smart cities. They consist of a sender monitoring a remote process and sending the sensed information to a receiver. The sender aims to deliver fresh information about the monitored process's state to allow the receiver to timely respond to the process's changes. In SUSs, the sender is usually battery operated. Therefore, to increase the available energy we consider Energy Harvesting (EH). Moreover, as at the receiver the information transmitted by the sender is only relevant when the process's state changes, we measure the information's freshness using Age of Incorrect Information (AoII). Finding the optimal transmission strategy at the sender that minimizes the AoII requires perfect system knowledge, i.e., the behavior of the monitored process, the channel quality, and the available energy. However, in real applications this knowledge is usually not available. To overcome this challenge, we first establish the optimality of threshold-based policies for AoII minimization in SUSs with EH capabilities by proving that there exists an AoII value depending on the observed state of the monitored process, the battery level and the receiver's estimation of the monitored process's state beyond which transmitting is preferable over idling. Next, we exploit the threshold-based policies' structure and deploy a learning algorithm based on Finite-Difference Policy Gradient (FDPG). Our proposed approach finds the AoII thresholds without requiring perfect system knowledge. Simulations show that our approach outperforms reference algorithms by at least 20% and efficiently learns near-optimal policies for AoII minimization.

In Mobile Crowdsensing (MCS) a mobile crowd-sensing platform (MCSP) collects sensing data from mobile units (MUs) in exchange for payment. The MCSP broadcasts a list of available sensing tasks. Based on this list, each MU solves a task proposal problem to decide which task it is willing to perform and sends a proposal to the MCSP. Based on the MUs' proposals, the MCSP solves a task assignment problem. There are two challenges when finding efficient task proposal strategies for the MUs and an efficient task assignment strategy for the MCSP (i) The techno-economic perspective of MCS: From the technical perspective, MCS should maximize the data quality while minimizing time and energy consumption. From the economic perspective, there are two sides, the MUs and the MCSP which act as selfish decision-makers, who aim at maximizing their own income. (ii) Incomplete information at two sides: Initially, the MCSP does not know the expected data quality and the MUs do not know the expected effort required for task completion. To overcome these challenges, we propose a novel Two-Sided Learning (TSL) approach. At the MU side, TSL is based on an innovative gradient-based multi-armed bandit solution to maximize the MUs' utility under incomplete information about the strategies of other MUs. At the MCSP side, a learning strategy is used to find the task assignment strategy that maximizes its utility. Simulation results show that TSL achieves near-optimal social welfare, which is the sum of MUs' and MCSP's utilities, and a near-optimal energy efficiency.

Mobile Crowdsensing (MCS) is a promising distributed sensing architecture that harnesses the power of sensors on mobile units (MUs) to perform sensing tasks. The MCS is a dynamic system in which the requirements of the sensing tasks, the MUs' conditions and the available resources change over time. The performance of an MCS system depends on the selection of the MUs participating in each sensing task. However, this is not a trivial problem. An optimal task participation strategy requires non-causal knowledge about the dynamic MCS system, a requirement that cannot be fulfilled in real implementations. Moreover, centralized optimization-based approaches do not scale with increasing number of participating MUs and often ignore the MUs' preferences. To overcome these challenges, in this paper we propose a novel multi-agent federated deep reinforcement learning algorithm (FDRL-PPO) which does not need this perfect non-causal knowledge, but instead, enables the MUs to learn their own task participation strategies based on their own conditions, available resources, and preferences. Through federated learning, the MUs share their learned strategies without disclosing sensitive information, enabling a robust and scalable task participation scheme. Numerical evaluations validate the effectiveness and efficiency of FDRL-PPO in comparison with reference schemes.

Mobile crowd-sensing (MCS) is an upcoming sensing architecture which provides better coverage, accuracy, and requires lower costs than traditional wireless sensor networks. It utilizes a collection of sensors, or crowd, to perform various sensing tasks. As the sensors are battery operated and require a mechanism to recharge them, we consider energy harvesting (EH) sensors to form a sustainable sensing architecture. The execution of the sensing tasks is controlled by the mobile crowd-sensing platform (MCSP) which makes task allocation decisions, i.e., it decides whether or not to perform a task depending on the available resources, and if the task is to be performed, assigns it to suitable sensors. To make optimal allocation decisions, the MCSP requires perfect non-causal knowledge regarding the channel coefficients of the wireless links to the sensors, the amounts of energy the sensors harvest and the sensing tasks to be performed. However, in practical scenarios this non-causal knowledge is not available at the MCSP. To overcome this problem, we propose a novel Deep-Q-Network solution to find the task allocation strategy that maximizes the number of completed tasks using only realistic causal knowledge of the battery statuses of the available sensors. Through numerical evaluations we show that our proposed approach performs only 7.8% lower than the optimal solution. Moreover, it outperforms the myopically optimal and the random task allocation schemes.

Mobile crowdsensing (MCS) is a novel approach to increase the coverage, lower the costs, and increase the accuracy of sensing data. Its main idea is to collect sensor data using mobile units (MUs). The sensing is controlled by a mobile crowdsensing platform (MCSP) through the assignment of delay-sensitive sensing tasks to the MUs. Although promising, research effort in MCS is still needed to find task assignment solutions that maximize the coverage while considering the cost incurred by the MCSPs, the preferences of the MUs and the limited communication resources available. Specifically, we identify two main challenges: (i) A task assignment problem which incorporates the MCSP’s utility and the preferences of the MUs. (ii) An underlying communication resource allocation problem formulating the requirement of the timely transmission of sensing results given the limited communication resources. To address these challenges, we propose a novel two-stage matching algorithm. In the first stage, potential MU-task pairs are constructed considering the preferences of the MUs and the utility of the MCSP. In the second stage, the communication resource allocation is done based on potential MU-task pairs from the first stage. Through numerical simulations, we show that our proposed approach outperforms state-of-the-art methods in terms of the MCSP’s utility, coverage and MU’s satisfaction.