Many emerging applications in edge computing require processing of huge volumes of data generated by end devices, using the freshest available information. In this paper, we address the distributed optimization of multi-user long-term average Age-of-Information (AoI) objectives in edge networks that use NOMA transmission. This poses a challenge of non-convex online optimization, which in existing work often requires either decision making in a combinatorial space or a global view of entire network states. To overcome this challenge, we propose a reinforcement learning-based framework that adopts a novel hierarchical decomposition of decision making. Specifically, we propose three different types of distributed agents to learn with respect to efficiency of AoI scheduling, fairness of AoI scheduling, as well as a high-level policy balancing these potentially conflicting design objectives. Not only does the proposed decomposition improve learning performance due to disentanglement of different design objectives/rewards, but it also enables the algorithm to learn the best policy while also learning the explanations – as actions can be directly compared in terms of the design objectives. Our evaluations show that the proposed algorithm improves the long-term average AoI by$200\%{-}300\%$and 400% compared to prior works with NOMA and the optimal solution without NOMA, respectively.

This paper addresses the problem of reliable communications for cooperative learning on Internet-of-Vehicles, where a large amount of data from users and services needs to be processed. Previous works have proposed various cooperative learning schemes, but they often assume that the communications between vehicles are reliable, without considering how to achieve this in an Internet-of-Vehicles network. This paper is the first one that implements an abstract MAC layer using a distributed deep reinforcement learning scheme, which can directly meet the reliable communication requirements of cooperative learning in previous works. Our abstract MAC layer performs two operations:acknowledgement, which makes sure that all vehicles can successfully broadcast their messages to all of their neighbors, andprogress, which ensures that each vehicle can receive at least one message from its neighbors. These operations facilitate vehicles to exchange and update their training models in a cooperative learning service. Our simulation results show the efficiency and fairness of our deep reinforcement learning abstract MAC layer.

We investigate over-the-air federated learning (OTA-FL) that exploits over-the-air computing (AirComp) to integrate communication and computation seamlessly for FL. Privacy presents a serious obstacle for OTA-FL, as it can be compromised by maliciously manipulating channel state information (CSI). Moreover, the limited band at edge hinders OTA-FL from training large-scale models. It remains open how to enable a multitude of devices with constrained resources and sensitive data to collaboratively train a global model at band-limited edge. To tackle this, we design a novel algorithmPROBEbuilding upon a lightweight over-the-air gradients aggregation rulePB-O-GAR. Specifically,PB-O-GARcombines a random sparsification-like dimension reduction with Gaussian perturbation to provide rigorous privacy and band-adapted communication. It elaborately calibrates the transmission signal according to devices’ perceived CSI for heterogeneous power constraints accommodation and CSI attack resilience. We show that by utilizing the common randomness, which deviates from the conventional FL, random sparsification-like dimension reduction can augment privacy in addition to the intrinsic privacy amplification effect of AirComp. We establish near-optimal convergence rates and explicit trade-offs among privacy, communication and utility forPROBE. Finally, extensive experiments on benchmark datasets are conducted to validate our theoretical findings and showcase the superiority ofPROBEin realistic settings.