In this work, a novel federated distillation weight aggregation method is proposed. Specifically, an algorithm designed for effective learning in distributed environments is introduced. This algorithm includes an innovative federated distillation scheme, proposing a sophisticated aggregation of model outputs, employing a global server model to manage process. On the server side, feature mixing is employed to aggregate client representations before they are transmitted back to the client side for knowledge distillation. During feature mixing a weight factor is assigned to each client’s logits, penalizing bad quality clients and preserving system’s credibility. Thorough experimentation has been conducted to comprehensively study the issue at hand. Key findings reveal the significant potential of the proposed solution, which achieves robust performance in a federated setting while reducing communication costs.

In this work, a novel federated distillation weight aggregation method is proposed. Specifically, an algorithm designed for effective learning in distributed environments is introduced. This algorithm includes an innovative federated distillation scheme, proposing a sophisticated aggregation of model outputs, employing a global server model to manage process. On the server side, feature mixing is employed to aggregate client representations before they are transmitted back to the client side for knowledge distillation. During feature mixing a weight factor is assigned to each client’s logits, penalizing bad quality clients and preserving system’s credibility. Thorough experimentation has been conducted to comprehensively study the issue at hand. Key findings reveal the significant potential of the proposed solution, which achieves robust performance in a federated setting while reducing communication costs.

This paper addresses the challenge of reducing annotation costs in distributed learning environments, particularly in systems with limited data and computational resources, such as those found in edge devices. We propose Federated Reinforced Active Learning, a framework that integrates Federated Learning with Reinforced Active Learning to optimize data labeling under strict cost constraints. The method is designed for small-scale networks where data is sparse, and minimal training epochs are available. By utilizing reinforcement learning within active learning, the system selects the most informative data samples, allowing for efficient training while significantly reducing the need for extensive annotations. This approach is particularly suited for environments where minimizing both annotation and computational costs is critical, such as in applications where cost efficiency and resource limitations are top priorities. The proposed method is evaluated on the CIFAR-10 and CIFAR-100 datasets using ResNet18, across 5 and 10 clients. Results demonstrate that the method significantly reduces annotation costs and improves learning outcomes, making it an ideal solution for cost-sensitive distributed systems.