With the proliferation of mobile sensing technology, substantial time series data have been generated and accumulated in various distributed domains, providing the basis for practical applications. Federated Learning (FL) has emerged as an essential framework for machine learning on decentralized data, especially with its potential for privacy-preserving. However, deployed with distributed and various edge devices, existing FL frameworks struggle to address the statistical heterogeneity of spatial feature distribution shifts from heterogeneous sensors. Due to the complex temporal dynamics of real-world time series data, temporal feature heterogeneity may result in suboptimal model adaptation performance. To this end, we propose a novelFederated learning approach withOrthogonal regularization forSpatial-Temporal heterogeneity (FedOST) on time series classification. It is featured in three aspects: (1) In the local training phase, we utilize an orthogonal projection to disentangle and align the shared and personalized features, as well as complementary information from different views of the time series data to formulate a robust multi-view training. (2) In the global aggregation phase, we adopt trainable global prototypes to improve feature space separation through orthogonal constraint, to serve as refined global knowledge in the local training. (3) In the testing phase, we leverage an uncertainty-aware test-time adaptation scheme to tackle the temporal feature shifts of unlabeled test data. We conduct extensive evaluations on real-world datasets, where FedOST outperforms existing state-of-the-art baselines with significant advantages.

In recent years, the distributed training of foundation models (FMs) has seen a surge in popularity. In particular, federated learning enables collaborative model training among edge clients while safeguarding the privacy of their data. However, federated training of FMs across resource-constrained and highly heterogeneous edge devices encounter several challenges. These include the difficulty of deploying FMs on clients with limited computational resources and the high computation and communication costs associated with fine-tuning and collaborative training. To address these challenges, we propose FedCKMS, a Cluster-Aware Framework with Knowledge-Aware Model Search. Specifically, FedCKMS incorporates three key components. The first component is multi-factor heterogeneity-aware clustering, which groups clients based on both data distribution and resource limitations and selects an appropriate model for each cluster. The second component is knowledge-aware model architecture search, which enables each client to identify the optimal sub-model from the cluster model, facilitating adaptive deployment that accommodates highly heterogeneous computational resources across clients. The final component is cluster-aware knowledge transfer, which facilitates knowledge sharing between clusters and the server, addressing model heterogeneity, and reducing communication overhead. Extensive experiments demonstrate that FedCKMS outperforms state-of-the-art baselines by 3-10% in accuracy.

Time series analysis is crucial across various fields like energy, environment, transportation, finance and health. Deep learning has significantly advanced this field, particularly, the Time Series Foundation Model (TSFM) excels in multiple domains due to extensive pre-training. In this work, we focus on TSFM’s challenges in medical practice: limited computing resources and medical data privacy. TSFM variants include fine-tuned models and those pre-trained for rapid deployment on diverse data. There may not be enough computing resources to train physiological signals locally in hospitals, and generalized TSFM is still inferior to task-specific methods on private, imbalanced local data. To address this, we propose PhysioPFM, a framework for efficiently personalizing TSFM. Our approach involves low-rank pre-training on public datasets, generator training by trained LoRA weights, and efficient weight generation via local data. Experimental results demonstrate that integrating generated models with TSFM enhances performance, and transferability, and reduces the need for additional sensitive data training.

Personalized federated learning (PFL) enhances the performance of customized client models through collaborative training without compromising data privacy and ownership. Some previous PFL methods rely on rich prior knowledge about the types of data heterogeneity (such as class imbalance or feature skew), which greatly limits their application ranges. In this paper, we study theUniversal Personalization in Federated Learning (UniPFL), the problem that has no prior knowledge about the types of data heterogeneity. In real-world PFL scenarios, UniPFL is potential because the data distributions of clients are usually heterogeneous and unknown to the server, where quantity imbalance, class imbalance, feature skew, or hybrid heterogeneity are possible contingencies. To address UniPFL, we proposeFedFD, a novel framework with local data augmentation and global concept fusion, which is based on the recent advances inthe foundation generative models(e.g., diffusion models, BLIP-2). On the client side, FedFD utilizes a diffusion model to assist local training by generating augmented data samples, and is then efficiently fine-tuned to be personalized. On the server side, we customize the aggregation strategies based on model similarities to learn both personalized models and diverse feature concepts. Extensive experiments show that FedFD reaches the state-of-the-art on (1) CIFAR-10 and CIFAR-100 for class imbalance; (2) DomainNet and Office-10 for feature skew, and (3) hybrid heterogeneity with both class and feature shifts.

Federated learning (FL) is undergoing significant traction due to its ability to perform privacy-preserving training on decentralized data. In this work, we focus on sensitive time series data collected by distributed sensors in real-world applications. However, time series data introduce the challenge of dual spatial-temporal feature skew due to their dynamic changes across domains and time, differing from computer vision. This key challenge includes inter-client spatial feature skew caused by heterogeneous sensor collection and intra-client temporal feature skew caused by dynamics in time series distribution. We follow the framework of Personalized Federated Learning (pFL) to handle dual feature drifts to enhance the capabilities of customized local models. Therefore, in this paper, we propose a method FedST to solve key challenges through orthogonal feature decoupling and regularization in both training and testing stages. During training, we collaborate time view and frequency view of time series data to enrich the mutual information and adopt orthogonal projection to disentangle and align the shared and personalized features between views, and between clients. During testing, we apply prototype-based predictions and model-based predictions to achieve model consistency based on shared features. Extensive experiments on multiple real-world classification datasets and multimodal time series datasets show our method consistently outperforms state-of-the-art baselines with clear advantages.

Federated learning (FL) offers a privacy-centric distributed learning framework, enabling model training on individual clients and central aggregation without necessitating data exchange. Nonetheless, FL implementations often suffer from non-i.i.d. and long-tailed class distributions across mobile applications, e.g., autonomous vehicles, which leads models to overfitting as local training may converge to sub-optimal. In our study, we explore the impact of data heterogeneity on model bias and introduce an innovative personalized FL framework, Multi-level Personalized Federated Learning (MuPFL), which leverages the hierarchical architecture of FL to fully harness computational resources at various levels. This framework integrates three pivotal modules: Biased Activation Value Dropout (BAVD) to mitigate overfitting and accelerate training; Adaptive Cluster-based Model Update (ACMU) to refine local models ensuring coherent global aggregation; and Prior Knowledge-assisted Classifier Fine-tuning (PKCF) to bolster classification and personalize models in accord with skewed local data with shared knowledge. Extensive experiments on diverse real-world datasets for image classification and semantic segmentation validate thatMuPFLconsistently outperforms state-of-the-art baselines, even under extreme non-i.i.d. and long-tail conditions, which enhances accuracy by as much as 7.39% and accelerates training by up to 80% at most, marking significant advancements in both efficiency and effectiveness.

Federated learning (FL) is a distributed framework for collaborative training with privacy guarantees. In real-world scenarios, clients may have Non-IID data (local class imbalance) with poor annotation quality (label noise). The co-existence of label noise and class imbalance in FL’s small local datasets renders conventional FL methods and noisy-label learning methods both ineffective. To address the challenges, we propose FEDCNI without using an additional clean proxy dataset. It includes a noise-resilient local solver and a robust global aggregator. For the local solver, we design a more robust prototypical noise detector to distinguish noisy samples. Further to reduce the negative impact brought by the noisy samples, we devise a curriculum pseudo labeling method and a denoise Mixup training strategy. For the global aggregator, we propose a switching re-weighted aggregation method tailored to different learning periods. Extensive experiments demonstrate our method can substantially outperform state-of-the-art solutions in mix-heterogeneous FL environments.

Due to the remarkable ability to model the high-order links within user-item relations, the graph neural network (GNN) is gradually applied to personalized recommendations in many online services. Besides, federated learning (FL) recently emerged as a powerful framework that enables collaborative training while protecting user data privacy. However, the integration of GNN and FL still exists with vital challenges unsolved, e.g., learning from non-IID local sub-graphs with only low-order user-item interactions jointly and overcoming the over-fitting problems with high training efficiency. In this paper, we propose CdFed, a Cluster-driven GNN-based Federated Learning framework, to address the GNN+FL challenges. CdFedhas two major components. First, to learn from non-IID sub-graphs, we design an Adaptive Model Clustering (AMC) strategy that takes advantage of the similarity across the uploaded model weights and updates clusters adaptively in each communication round. Second, we develop a Biased Message Dropout (BMD) strategy to combat the overfitting problem and accelerate the training process of federated learning. Together with AMC and BMD, CdFedcan implicitly complete the missing links between sub-graphs more efficiently and greatly improve the model’s generalization ability in non-IID scenarios. We have conducted extensive evaluations and the results reveal that our proposed approaches can outperform the SOTA solution by 24% in model performance and 8x in training speed.

Federated learning (FL) emerges as a new distributed machine learning (ML) paradigm that enables thousands of mobile devices to collaboratively train ML models using local data without compromising user privacy. However, the FL learning quality highly relies on the data contribution from the distributed mobile devices. Therefore, a well-designed incentive mechanism with effectiveness, fairness, and reciprocity is in urgent need to guarantee the stable participation of users. In this article, we propose federated auction bandit (FedAB), an incentive and client selection strategy based on a novel multiattribute reverse auction mechanism and a combinatorial multi-armed bandit (CMAB) algorithm. First, we develop a local contribution evaluation method based on importance sampling in the FL context. We then design a novel payment mechanism that is able to preserve individual rationality and incentive compatibility (truthfulness). At last, we design a UCB-based winner selection algorithm that is proven to achieve the server’s utility maximization with fairness and reciprocity. We have conducted extensive experiments on real data sets. The results demonstrate the superiority ofFedAB, with a 10%–50% improvement in total reward, final accuracy, and convergence speed compared to state-of-the-art solutions.