Knowledge Distillation (KD) is considered as an efficient way to replace the parameter averaging in federated learning, aiming to handle the clients with heterogeneous model architectures. Relying on the prepared distillation datasets across clients and the server, KD may encounter impractical difficulties in real-world implementations. Existing works explore the data-free KD in federated learning, which generates the distillation datasets on-site. However, the distillation datasets with global data distribution generated by these state-of-the-art schemes cannot be adapted to local non-IID data. In this article, we propose a new Personalized Data-Free Knowledge Distillation, namely PDKD, for federated learning under heterogeneous models and data. PDKD solves the problem of model drift caused by the inconsistent distribution of distillation datasets and the local data by generating personalized distillation datasets for each client while protecting client data privacy. In addition, we design a distillation dataset update scheme that maximizes the difference between teacher and client outputs on distillation datasets to accomplish deeper knowledge transfer. Furthermore, in order to accomplish the co-evolution of the teacher model and the clients’ model, PDKD incorporates a mutual distillation scheme. Numerous experiments show that PDKD significantly outperforms several state-of-the-art algorithms, with an 18% improvement in prediction accuracy and has a much lower communication cost than the compared algorithms.

Edge AI applications enable edge devices to collaboratively learn a model via repeated model aggregations, aiming to utilize the distributed data on the devices for achieving high model accuracy. Existing methods either leverage a centralized server to directly aggregate the model updates from edge devices or need a central coordinator to group the edge devices for localized model aggregations. The centralized server (or coordinator) has a performance bottleneck and a high cost of collecting the global state needed for making the grouping decision in large-scale networks. In this paper, we propose an Autonomous Model Aggregation (AMA) method for large-scale decentralized learning on edge devices. Instead of needing a central coordinator to group the edge devices, AMA allows the edge devices to autonomously form groups using a highly efficient protocol, according to model functional similarity and historical grouping information. Moreover, AMA adopts a reinforcement learning approach to optimize the size of each group. Evaluation results on our self-developed edge computing testbed demonstrate that AMA outperforms the benchmark approaches by up to 20.71% in accuracy and reduced the convergence time by 75.58%.

Federated learning enables multiple clients to collaboratively learn machine learning models in a privacy-preserving manner. However, in real-world scenarios, a key challenge encountered in federated learning is the statistical heterogeneity among clients. Existing work mainly focused on a single global model shared across the clients, making it hard to generalize well to all clients due to the large discrepancy in the data distributions. To address this challenge, we propose pFedLT , a novel approach that can adapt the single global model to different data distributions. Specifically, we propose to perform a pluggable layer-wise transformation during the local update phase based on scaling and shifting operations. In particular, these operations are learned with a meta-learning strategy. By doing so, pFedLT can capture the diversity of data distribution among clients, therefore, can generalize well even when the data distributions among clients exhibit high statistical heterogeneity. We conduct extensive experiments on synthetic and real-world datasets (MNIST, Fashion_MNIST, CIFAR-10, and Office+Caltech10) under different Non-IID settings. Experimental results demonstrate that pFedLT significantly improves the model accuracy by up to 11.67% and reduces the communication costs compared with state-of-the-art approaches.

In smart cities, data monitoring for basic infrastructures, such as an urban water supply pipeline system or an oil/gas supply pipeline system, has become one of the most important tasks. An urban pipeline system that can be called a large-scale acyclic directed pipeline network (LSADPN), which has some characteristics, such as wide geographical distribution, complex connection, and deep underground. Therefore, it is difficult to be monitored comprehensively and accurately in real time. In recent years, some studies have proposed methods that use mobile sensors that are put into a pipeline network to obtain accurate monitoring results from the interior of the network. However, the mobile sensors have no motion devices and can only flow with the liquid in the pipelines. When a pipeline connection is encountered, it is uncertain whether all branches of the connection can be covered. Therefore, the maximum monitoring capacity, i.e., liquid capacity in the network monitored by the mobile sensors, is difficult to be maximized. In this article, the problem of maximizing the monitoring capacity of LSADPN is first proved to be NP-hard. Then, two new mobile sensors deployment algorithms based on the submodular function optimization method are proposed. Theoretical analyses and experimental results show that the two algorithms can monitor a whole network with high probability and achieve near maximum monitoring capacity with a specified number of mobile sensors and a given time.