Knowledge Distillation (KD) is considered as an efficient way to replace the parameter averaging in federated learning, aiming to handle the clients with heterogeneous model architectures. Relying on the prepared distillation datasets across clients and the server, KD may encounter impractical difficulties in real-world implementations. Existing works explore the data-free KD in federated learning, which generates the distillation datasets on-site. However, the distillation datasets with global data distribution generated by these state-of-the-art schemes cannot be adapted to local non-IID data. In this article, we propose a new Personalized Data-Free Knowledge Distillation, namely PDKD, for federated learning under heterogeneous models and data. PDKD solves the problem of model drift caused by the inconsistent distribution of distillation datasets and the local data by generating personalized distillation datasets for each client while protecting client data privacy. In addition, we design a distillation dataset update scheme that maximizes the difference between teacher and client outputs on distillation datasets to accomplish deeper knowledge transfer. Furthermore, in order to accomplish the co-evolution of the teacher model and the clients’ model, PDKD incorporates a mutual distillation scheme. Numerous experiments show that PDKD significantly outperforms several state-of-the-art algorithms, with an 18% improvement in prediction accuracy and has a much lower communication cost than the compared algorithms.

In an IMANET, mobile users access both text and media web contents on the Internet through gateway nodes, with web access patterns, i.e., the Zipf-like distribution or the Stretched Exponential distribution. To reduce data access delay from the Internet, we consider the cache placement problem in cooperative caching, which is that each mobile node selects a subset of web contents to cache cooperatively in its limited cache so that total access cost is minimized. It has been proved NP-hard. We propose a solution named Adaptive Allocation Cooperative Caching (AACC), which adaptively divides the cache space of each node into two components: altruistic and selfish, according to detected data access patterns. AACC aims to find the optimal cache allocation solution to allocate appropriate cache spaces for two components in order to minimize total access cost. Given the Zipf-like access distribution, we find a near-optimal allocation solution to the cache placement problem. Simulation results show that AACC achieves much better performance than the existing best cooperative caching strategy in IMANETs in terms of average query delay, caching overheads, and query success ratio. In particular, AACC reduces caching overheads by 40% in average.