The rise of 6G networks brings ultra-fast communication and broad connectivity, enabling intelligent applications such as IoT, smart cities, and autonomous vehicles. However, training AI models in these environments often faces privacy concerns and limited labeled data. Federated Learning (FL) provides a privacy-preserving solution by allowing clients to collaboratively train models without sharing raw data. Yet, FL still struggles with the scarcity of labeled data due to privacy constraints. Semi-Supervised Learning (SSL) can alleviate this issue by leveraging both labeled and unlabeled data. While combining SSL and FL (FSSL) offers promise, it also introduces new challenges, such as confirmation bias and degraded performance under non-IID data distributions. Most existing FSSL methods assume uniform labeling capabilities across clients, which is rarely the case in practice. This leads to a new challenge only occur in FSSL called annotation heterogeneity, where some clients have many labeled samples while others have few or none. When combined with data heterogeneity, this dual-data heterogeneity (data and annotation heterogeneity) severely affects global model performance. Therefore, FSSL must learn under label scarcity while preserving privacy, remain stable when non-IID data is compounded by annotation heterogeneity, and stay communication-efficient for mobile deployment. In this work, we propose Federated Fisher Focus Filtering (FedF3), an enhanced framework built upon our previous SynFMPL method, to address the combination effect of dual-data heterogeneity. FedF3 introduces a two-stage strategy, Adaptive Loss Filtering stabilizes early training by suppressing unreliable contributions under dual heterogeneity, and Fisher Focus Selection preserves accuracy with Fisher-guided sparsity to meet communication budgets. Our method demonstrates robust performance improvements across diverse and heterogeneous FL settings, mitigating the negative effects of dual-data heterogeneity while preserving personalization and privacy.

Federated Learning (FL) is a promising framework for enabling edge intelligence while preserving privacy and enhancing communication efficiency. In real-world scenarios, limited and heterogeneous hardware resources among users threaten the overall training efficiency in FL. To address this issue, several model heterogeneity frameworks have emerged in recent years. However, existing studies primarily focus on mitigating idle times on powerful devices, overlooking the fact that strong devices may not always have a positive impact and can potentially become a burden instead. Furthermore, as the resource gap among clients widens or the number of weak devices increases, the model may be hindered by the influence of strong devices. Directly applying existing model heterogeneity frameworks leads to model performance degradation due to the neglect of contributions from weak devices. To make a performance breakthrough, we proposed a novel FL framework called Progressive Width Mixing (PWM), which extracts diverse features and balances the contribution of strong and weak devices and mitigates straggler effects under scenarios with various device distributions. To our best knowledge, we are the first model heterogeneity work that considered straggler effects from the weak devices' point of view. The experiments show that PWM maintains stability over different datasets and outperforms all baselines across various device distributions and non-IID scenarios. PWM makes a leap forward and surpasses all baselines by 7%, and mitigates the negative impact from the strong devices.

Recently, Google proposed a privacy-preserving framework, Federated Learning (FL), to realize the era of edge intelligence. However, the limited labeled data that can only be labeled by the data owner with domain know-how threatens the feasibility of FL. In fact, Semi-supervised Learning (SSL) is a key to leveraging the unlabeled data on the client side while the confirmation bias issue triggered by incorrect predictions in the old-fashion Teacher-Student structure drops the model accuracy significantly. In this paper, we introduce the state-of-the-art SSL technique called Meta Pseudo Labels (MPL), which advocates a feedback strategy in the Teacher-Student architecture to mitigate the performance degradation. Unfortunately, in FL the heterogeneous data without sharing among clients are non-Independent and Identically Distributed (non-IID). The knowledge that shuttles between the Teacher and Student model may be partial and undermine the effectiveness of MPL. To make a performance breakthrough, we proposed a novel FL framework called Federated Meta Learning with Synergetic Strategy (SynFMPL), which combines the advantages of both Teacher and Student models. To our best knowledge, we are the first work that integrates MPL technique in FL. Moreover, we specifically focus on the client side, which fully leverages the unlabeled data for model generalization. The experiment shows that SynFPML outperforms all baselines in various non-IID scenarios. The Student model in SynFMPL even exceeds the upper bound in all baselines by 4% and maintains a virtuous cycle in the learning process.