Edge computing, an emerging paradigm, utilizes decentralized edge nodes to offer low-latency, high-quality network services. Quality of Service (QoS) is a crucial metric to measure network service quality, and network resource scheduling relies on QoS measurement results. However, current QoS measurement methods often measure all QoS data among edge nodes, these dense measurement approaches introduce significant costs. Besides, edge nodes may adopt varied network access manners, these factors cause fluctuations in QoS between edge nodes. But existing QoS measurement works often focus on measuring exact QoS values while merely considering QoS fluctuations, resulting in unreliable measured QoS data. In addition, existing QoS measurement methods often can not support online QoS measurement, leading to stale offline QoS data affecting network resource scheduling. To tackle the two issues, we propose a novel Continual Sparse QoS range measurement method (CSQoS) with four innovative designs: (1) To reduce measurement costs, we propose to measure QoS by only sampling partial QoS data and using them to impute unmeasured QoS data. To achieve sparse QoS imputation, we propose a novel variational Bayesian model (BayGNN) with an edge-enhanced Graph Neural Network (GNN) as the encoder for feature extraction and a Multilayer Perceptron (MLP) as the decoder to predict unmeasured QoS data. (2) To assess QoS data ranges, we design the proposed BayGNN model to produce uncertainty simultaneously. (3) To fulfill reliable online QoS predictions, we incorporate continual learning and residual connections in BayGNN. Experimental results on 2 real-world datasets demonstrate that CSQoS has minimal QoS imputation error with the lowest measurement costs, reducing 17.6% RMSE and 20% sampling costs.

Model-heterogeneous federated learning (MHFL), supporting FL collaboration across clients with heterogeneous models, has become a more practical FL paradigm. Existing MHFL methods enable knowledge fusion over heterogeneous client models by sharing partial homogeneous parameters or extracted label-wise average representations, suffering from model performance bottlenecks and privacy leakage risks. To bridge this gap, we propose a novel model-heterogeneous Federated learning method with homogeneous Representation Subspace Learning (FedRSL) instead of sharing model parameters or representations. In FedRSL, each client’s local heterogeneous model comprises a feature extractor and a prediction header. (1) We construct a homogeneous representation subspace for each client to learn local representation knowledge, and the server aggregates them to generate the global representation subspace for representation knowledge fusion. (2) To facilitate representation learning capability while maintaining efficient communication and computation, we design a lightweight linear model as the homogeneous low-rank linear representation subspace. For each local data sample, its personalized representation extracted by the feature extractor is processed by the global representation subspace to produce the corresponding generalized representation. (3) To effectively bi-transfer global generalized and local personalized knowledge, we reduce the distance between the local personalized representation and the corresponding generalized representation. Experiments on 3 computer vision and 1 natural language processing benchmark datasets over 6 baselines demonstrate that FedRSL obtains state-of-the-art model accuracy (up to 5.51% accuracy improvement) while consuming low communication and computation overheads.

With growing client diversity, model-heterogeneous personalized federated learning (MHPFL) supports collaboration over structure-heterogeneous client models. However, existing MHPFL methods only achieve client-level personalization but ignore inherent discrepancies within each client's different data samples, leading to limited model performance. To this end, we propose a novel model-heterogeneouspersonalizedFederated learning withMixtureofExperts (pFedMoE) to achieve a fine-grained data-level personalization. As the first work that incorporates MoE in MHPFL, it introduces three innovations: (1) Different clients hold heterogeneous local models, we add a small proxy global homogeneous feature extractor shared by clients for knowledge exchange. (2) To achieve a fine-grained data-level personalization, we construct a personalized local MoE for each client: a local expert (local heterogeneous client model's feature extractor), a global expert (global proxy homogeneous feature extractor), and a local personalized gating network, which dynamically balances the generalization and personalization of the local model at the data sample level. (3) We customize a lightweight linear gating network to capture the generalized and personalized data characteristics of each local data sample. We theoretically prove its$\mathcal {O}(1/T)$convergence rate. Experiments on 3 benchmark image datasets, 1 real-world image dataset and 1 real-world text dataset against 9 baselines demonstrate its state-of-the-art model accuracy with up to 2.79% accuracy improvement while saving up to 43.12% computational overheads and keeping satisfactory communication costs.

Federated learning (FL) is an emerging machine learning paradigm in which a central server coordinates multiple participants (FL clients) collaboratively to train on decentralized data. In practice, FL often faces data, system, and model heterogeneity, which inspires the field of Model-Heterogeneous Personalized Federated Learning (MHPFL). However, existing MHPFL methods rely on extra public data or ignore the relationship between local private heterogeneous models and shared homogeneous models across clients. This leads to unsatisfactory model performance, computational overheads, and communication costs. To bridge this gap, we propose a novel and efficient model-heterogeneouspersonalizedFederated learning framework (pFedLoRA) based on sharing homogeneous Low-Rank Adapter (LoRA) which is popular for fine-tuning pre-trained models. Specifically, we devise a lightweight homogeneous adapter, rather than apply the typical LoRA, to facilitate each client's heterogeneous local model training with our proposed iterative training for global-local bidirectional knowledge exchange. The homogeneous small local adapters are aggregated on the FL server to generate a global adapter. We theoretically prove its$\mathcal {O}(1/T)$non-convex convergence rate. Experiments on 5 datasets demonstratepFedLoRAoutperforms 9 state-of-the-art baselines in model accuracy with$11.81 \times$computation and$7.41\times$communication cost saving.

Learning-based compression shows competitive compression ratios for genomics data. It often includes three types of compressors: static, adaptive and semi-adaptive. However, these existing compressors suffer from inferior compression ratios or throughput, and adaptive compressors also faces model cold-start problems. To address these issues, we propose DeepGeCo, a novel genomics data lossless adaptive compression framework with (s,k)-mer encoding and deep neural networks, involving three compression modes (MINI for static, PLUS for adaptive, ULTRA for semi-adaptive) for flexible requirements of compression ratios or throughput. In DeepGeCo, (1) we develop BiGRU and Transformer as the backbone to build Warm-Start and Supporter models in terms of cold-start problems. (2) We introduce (s,k)-mer encoding to pre-process genomics data before feeding it into the DNN model for improve model throughput, and we propose a new metric - Ranking of Throughput and Compression Ratio (RTCR) for effective encoding parameters selection. (3) We design a threshold controller and a probabilistic mixer within the backbone to balance compression ratios and model throughput. Experiments on 10 real-world datasets show that DeepGeCo's three compression modes improve up to a 22.949X average throughput and up to a 31.095% average compression ratio improvement while occupying low CPU or GPU memory.

Federated learning (FL), as a privacy-preserving collaborative machine learning paradigm, has attracted significant interest from industry and academia. To allow each data owner (FL client) to train a heterogeneous and personalized local model based on its local data distribution, system resources and requirements on model structure, the field of model-heterogeneous personalized federated learning (MHPFL) has emerged. Existing MHPFL approaches either rely on the availability of a public dataset with special characteristics to facilitate knowledge transfer, incur high computational and communication costs, or face potential model leakage risks. To address these limitations, we propose a model-heterogeneous personalized Federated learning approach based on generalized proxy feature Extractor Sharing (pFedES) for supervised image classification tasks. (1) We devise a shared small proxy homogeneous feature extractor before each client's heterogeneous local model. (2) Clients train them via the proposed iterative learning to enable the exchange of global generalized knowledge and local personalized knowledge. (3) The small proxy local homogeneous extractors produced after local training are uploaded to the server for aggregation to facilitate knowledge fusion across clients. We theoretically prove pFedES converges with a non-convex convergence rate O(1/T). Experiments on 3 benchmark datasets against 9 baselines demonstrate that pFedES performs state-of-the-art model accuracy while maintaining efficient communication and computation.

Explosive growth of multi-source data (MSD) poses challenges in data transmitting and storing. Neural Network (NN)-based lossless compressors are an important type of compression approaches to alleviate these problems. However, existing NN-based lossless compressors suffer from poor compression ratio and high time cost at the same time. To address these issues, we propose a novel MSD Lossless Compressor (MSDLC) with two compression stages: 1) We propose a Parallel Expansion Mapper (PEM) to map redundant pieces in MSD into unused alphabet values, which not only compresses MSD but also saves time for the next stage’s NN-based lossless compression. 2) With the mapped MSD as input, we design a NN-based lossless compressor to further improve compression ratio, where we introduce the state-of-the-art xLSTM model and design a Deep Spatial Gating Module (DSGM) as the backbone of NN. We compare MSDLC with 11 baselines on 6 real-world datasets and the results validate that MSDLC obtains the best average compression ratio and time cost. Compared with baselines, compression ratios are improved by 1.103%~113.897%, and the time costs are improved by 41.367%~73.891%. The codes can be available at https://github.com/mhuidong/MSDLC.

Learning-based lossless compressors have been validated to have competitive advantages in genomics data (GD) compression. However, learning-based GD-dedicated compressors typically need to be pre-trained on multi-source data and then are directly used to compress another target data, we denote them as static compressors, and they often face two challenges: limited compression ratios and bad-performed generalization due to data distribution variations. To solve these problems, we propose AGDLC, a novel Adaptive Genomics Data Lossless Compressor. It includes two critical designs: 1) We design a multiple (s, k)-mer mixer for extracting GD redundancy from multiple dimensions to improve compression ratios. 2) We introduce a recently popular XLSTM model as the backbone, which adaptively compresses GD while updating parameters, without pre-training, improving compression ratios and compression generalization at the same time. We compare AGDLC with 13 baselines on 7 real-world datasets, and the experimental results demonstrate that it achieves the best compression ratio with an average improvement of 2.162%-69.436%. The codes can be found at https://github.com/dingyanfeng/AGDLC.

Federated learning (FL) is a distributed machine learning technique enabling multiple clients to jointly train a global model while preserving the privacy of their non-IID (non-independent and identically) data. However, traditional FL approaches require clients to use the same model structure as the global model, which is not suitable for scenarios where clients need to train heterogeneous local models with different architectures, known as Heterogeneous Federated Learning (HFL). Current HFL approaches often exchange all features of local data through the interaction of the client model and the server model. The local data from different clients may present similar generic features, and sharing them over clients may hinder the effective learning of personalized features which truly leads to non-IID distributions. To facilitate the effective information exchange between the server and client while maintaining efficient communication and computation, we propose a novel Heterogeneous Federated learning method (HFedPFS) based on Personalized data Feature Sharing. We designed two significant patterns for this algorithm: (1) A Feature Perception Network (FPN) separates generic and personalized features at each client. (2) A global homogeneous adapter processes the personalized features, enabling effective bidirectional feature exchange. HFedPFS outperforms six state-of-the-art HFL methods on two datasets, improving accuracy by up to 3.54 and 4.17 percentage points in homogeneous and heterogeneous scenarios, respectively while reducing training time by 29.7%.

Federated learning (FL), an emerging distributed machine learning paradigm, utilizes edge decentralized data from multiple edge nodes (clients) to train a shared model under preserved data privacy. Furthermore, model-heterogeneous personalized federated learning (MHPFL) enables FL clients to train structurally different personalized models on non-independent and identically distributed (non-lID) local data. Existing MHPFL methods focus on data distribution differences among clients, and they propose various client-level personalization approaches to alleviate non-lID issues. However, different data samples in one client may also have different features, which are often ignored, resulting in constrained model performances. To bridge this gap, we propose a novel model-heterogeneous personalized Federated learning approach with Adaptive Feature Mixture (pFedAFM) to achieve data-level personalization while maintaining efficient communication and computation. It consists of three innovative designs: 1) We add a homogeneous small feature extractor alongside each client's local heterogeneous model, and the server aggregates these homogeneous small feature extractors for cross-client knowledge fusion. 2) We design an iterative training strategy to alternately train the global homogeneous small feature extractor and the local heterogeneous client model, for effective bidirectional exchange between global generalized knowledge and local personalized knowledge. 3) During model training, we devise a trainable weight vector to adaptively mix the features (representation) extracted by the global homogeneous and local heterogeneous models for different data samples, i.e., fulfilling data-level personalized feature mixture. Theoretical analysis proves that pFedAFM converges over time. Extensive experiments on 3 computer vision (CV) and 1 natural lan-guage processing (NLP) benchmark datasets demonstrate that pFedAFM significantly outperforms 8 state-of-the-art MHPFL methods, achieving up to 7.93% accuracy improvement while incurring low communication and computation costs.

Learning-based lossless compressors play a crucial role in large-scale genomic database backup, storage, transmission, and management. However, their 1) inadequate compression ratio, 2) low compression & decompression throughput, and 3) poor compression robustness limit their widespread adoption and application in both industry and academia. To solve those challenges, we propose a novel Parallel Multi-Knowledge Learning-based Compressor (PMKLC) with four crucial designs: 1) We propose an automated multi-knowledge learning-based compression framework as compressors' backbone to enhance compression ratio and robustness; 2) we design a GPU-accelerated (s,k)-mer encoder to optimize compression throughput and computing resource usage; 3) we introduce data block partitioning and Step-wise Model Passing (SMP) mechanisms for parallel acceleration; 4) We design two compression modes PMKLC-S and PMKLC-M to meet the complex application scenarios, where the former runs on a resource-constrained single GPU and the latter is multi-GPU accelerated. We benchmark PMKLC-S/M and 14 baselines (7 traditional and 7 leaning-based) on 15 real-world datasets with different species and data sizes. Compared to baselines on the testing datasets, PMKLC-S/M achieve the average compression ratio improvement up to 73.609% and 73.480%, the average throughput improvement up to 3.036X and 10.710X, respectively. Besides, PMKLC-S/M also achieve the best robustness and competitive memory cost, indicating its greater stability against datasets with different probability distribution perturbations, and its strong ability to run on memory-constrained devices. Overall, PMKLC is a balanced compression solution that optimizes compression ratio, throughput, robustness, and resource consumption. PMKLC and linkages of datasets are available at https://github.com/dingyanfeng/PMKLC.

With the rapid development of the Internet, the huge amount of Multi-Source Data (MSD) brings challenges in data sharing and storing. Lossless data compression is the major way to solve those problems. Nowadays, neural-network technologies bring significant advantage in data modeling, making learning-based lossless compressors (LLCs) for multi-source data have emerged continuously. Compared with traditional compressors, the LLCs are more useful to catch complex redundancy patterns in MSD, and thus have great potential in enhancing compression ratio. However, existing LLCs still suffer from unsatisfactory compression ratios and lower throughput. To solve those problems, we propose a novel universal MSD lossless compressor called MSDZip via Stepwise-parallel and learning-based prediction technologies, it introduces two major designs: 1) We propose a Local-Global-Deep Mixing block in the learning-based prediction module to establish dependencies for MSD symbols, where designed Deep Mixing block solves the problem of unstable weights in the perceptual layers caused by cold-start problem to enhance the compression ratio significantly. 2) We design a Stepwise-parallel multi-GPU-accelerated compression strategy to address the compression speed and graphics memory constraints of single GPU in the face of large-scale data. The Stepwise-parallel module passes the source MSD to learning-based prediction model through the data chunking strategy, where the model of the previous chunk is used to guide the compression of the next chunk in parallel. We compare MSDZip with 5 classical learning-based and 6 traditional compressors on 12 well-studied real-world datasets. The experimental results demonstrate that MSDZip optimizes 3.418%-69.874% in terms of compression ratio and 31.171%-495.649% in terms of throughput compared to advanced LLCs. The source code of MSDZip and the linkages of the experimental datasets are available at https://github.com/mhuidong/MSDZip.

Abstract As various types of data grow explosively, large-scale data storage, backup, and transmission become challenging, which motivates many researchers to propose efficient universal compression algorithms for multi-source data. In recent years, due to the emergence of hardware acceleration devices such as GPUs, TPUs, DPUs, and FPGAs, the performance bottleneck of neural networks (NN) has been overcome, making NN-based compression algorithms increasingly practical and popular. However, the research survey for the NN-based universal lossless compressors has not been conducted yet, and there is also a lack of unified evaluation metrics. To address the above problems, in this paper, we present a holistic survey as well as benchmark evaluations. Specifically, i) we thoroughly investigate NN-based lossless universal compression algorithms toward multi-source data and classify them into 3 types: static pre-training, adaptive, and semi-adaptive. ii) We unify 19 evaluation metrics to comprehensively assess the compression effect, resource consumption, and model performance of compressors. iii) We conduct experiments more than 4600 CPU/GPU hours to evaluate 17 state-of-the-art compressors on 28 real-world datasets across data types of text, images, videos, audio, etc. iv) We also summarize the strengths and drawbacks of NN-based lossless data compressors and discuss promising research directions. We summarize the results as the NN-based Lossless Compressors Benchmark (NNLCB, See fahaihi.github.io/NNLCB website), which will be updated and maintained continuously in the future.

Data deduplication is an effective technique for reducing redundant data storage space in various storage systems. Generally, deduplication consists of four steps: chunking, fingerprinting, fingerprint lookup, and data management. Recently, Non-volatile Memory (NVM) as an emerging storage device has received widespread attention. Directly applying the deduplication technique on NVM for storage cost savings faces many challenges: (a) deduplication on NVM devices suffers from computation bottleneck instead of the I/O bottleneck faced by deduplication on traditional storage devices (such as HDD and SSD); (b) new fingerprint indexes and metadata are required to be re-designed to adapt to NVM characteristics; (c) inline deduplication on NVM is more sensitive to the latency. To solve these challenges, we propose a novel Samp ling prediction-based inline data Dedup lication method ( SampDedup ) on NVM devices. It aims to ensure high deduplication ratios while reducing computation costs and latency by optimizing data chunking , fingerprinting , and fingerprint lookup . (a) For data chunking , a sampling prediction-based chunking method ( SampChunk ) is proposed to leverage chunk similarity to distinguish duplicate chunks and skip them for chunking. This method can be easily integrated into most sliding-window based and non-window based CDC chunking algorithms. (b) For fingerprinting , the commonly used SHA-1 algorithm is further optimized to reduce the extra computational overhead introduced by SampChunk, and an asynchronous fingerprinting method is proposed to reduce the fingerprinting latency of unique chunks. (c) For fingerprint lookup , we design a header fingerprint index and metadata table for each data chunk constructed by SampChunk on NVM, and we use a fast-read buffer to replace the traditional slow LRU cache to improve search efficiency. Experiments on four real-world datasets demonstrate that SampDedup consistently presents high inline data deduplication ratios on NVM with different workloads and data partitioning algorithms while saving more than 90% chunking time compared with state-of-the-art deduplication baselines.

Model heterogeneous federated learning (MHeteroFL) enables FL clients to collaboratively train models with heterogeneous structures in a distributed fashion. However, existing MHeteroFL methods rely on training loss to transfer knowledge between the client model and the server model, resulting in limited knowledge exchange. To address this limitation, we propose the **Fed**erated model heterogeneous **M**atryoshka **R**epresentation **L**earning (**FedMRL**) approach for supervised learning tasks. It adds an auxiliary small homogeneous model shared by clients with heterogeneous local models. (1) The generalized and personalized representations extracted by the two models' feature extractors are fused by a personalized lightweight representation projector. This step enables representation fusion to adapt to local data distribution. (2) The fused representation is then used to construct Matryoshka representations with multi-dimensional and multi-granular embedded representations learned by the global homogeneous model header and the local heterogeneous model header. This step facilitates multi-perspective representation learning and improves model learning capability. Theoretical analysis shows that FedMRL achieves a $O(1/T)$ non-convex convergence rate. Extensive experiments on benchmark datasets demonstrate its superior model accuracy with low communication and computational costs compared to seven state-of-the-art baselines. It achieves up to 8.48% and 24.94% accuracy improvement compared with the state-of-the-art and the best same-category baseline, respectively.

Recently, federated learning (FL) as a new learning paradigm allows multi-party to collaboratively train a shared global model with privacy protection. However, vanilla FL running on heterogeneous mobile edge devices still faces three crucial challenges: communication efficiency, statistical heterogeneity, and system heterogeneity.To tackle them simultaneously, we deviseFedPE, a communication-efficient and personalized federated learning framework, which allows each client to search for personalized optimal local subnets adaptive to system capacity in each round of FL.It consists of three core components: a)adaptive pruning-expandingcontrols model pruning or expanding according to the accuracy variations of local models, b)error compensation strategypromotes the pruned or expanded subnets to be Lottery Ticket Networks (LTNs), c) thefair aggregation ruleaggregates local models with their real-time contributions as coefficients to boost the performance of the aggregated global model. The integration of the three components facilitates that onlypersonalized optimal subnets with different footprintsinteract between the server and clients, which effectively reduces communication costs and enhances the robustness of FL to statistical and system heterogeneity. We also prove the convergence ofFedPEand design an optimal hyperparameter searching (OHS) algorithm based onPareto optimizationto search for optimal hyperparameters forFedPE. Extensive experiments evaluated on five real-world datasets with IID or Non-IID distributions demonstrate thatFedPEconfigured with found optimal hyperparameters achieves$1.86\times -121\times$communication efficiency improvement with almost no accuracy degradation, presenting the best trade-off between model accuracy and communication cost.

In cross-device horizontal federated learning (FL), the communication cost of transmitting complete models between edge devices and a central server is a significant bottleneck, due to expensive, unreliable, and low-bandwidth wireless connections. As a solution, we propose a novelFLframework namedQSFL, towardsoptimizing FL uplink (client-to-server) communication at both client and model levels. At the client level, we design aQualification Judgment (QJ)algorithm to sample high-qualification clients to upload models. At the model level, we design aSparse Cyclic Sliding Segmentation (SCSS)algorithm to further compress the local model transmitted from the client to the server in the uplink communication. We prove that QSFL can converge over wall-to-wall time, and develop an optimal hyperparameter searching algorithm based on theoretical analysis to enable QSFL to make the best trade-off between model accuracy and communication cost. Experimental results show that QSFL achieves state-of-the-art compression ratios with marginal model accuracy degradation. Since mobile edge devices as FL clients often have heterogeneous system resources, such as communication bandwidth, we propose two noveldynamic segmentation strategies with varied counts or sizesbased on QSFL to enhance the robustness of QSFL to FL system heterogeneity. For some mobile edge devices joining as FL clients with both limited uplink and downlink communication bandwidths, they can not pull up the global model from the server. To tackle it, we propose a novelsymmetric downlink compressionscheme on top of QSFL to further reduce the downlink (server-to-client) communication costs, hence enabling a bidirectional communication-efficient FL. Theory analysis and experiments demonstrate that QSFL with dynamic segmentation or symmetric downlink compression still keeps convergence and takes a better trade-off between model accuracy and communication efficiency than without them.

Federated Learning (FL) is a new paradigm of distributed machine learning, which can effectively solve the problem of data islands with privacy protection. Typical aggregation in FedAvg causes the global model bias to local models of clients with more local data. When clients’ data is Non-IID, the global model can not fully learn the data distribution of clients with less data, which incurs unfairness to these clients and then uninspired them to participate in FL. To this end, we propose a real-time fairness adjustment algorithm for the global model based on model-level contrastive learning, called FFedCL. We calculate our special-designed contrastive learning loss and use Stochastic Gradient Descent (SGD) to adjust the global model. It aims to improve the similarity of the global model to the model of clients with less data, thereby improving the fairness of FL. We evaluate FFedCL on three Non-IID datasets. The experimental results present that FFedCL improves up to 7% accuracy while maintaining almost the lowest variance compared with the state-of-the-art baseline, demonstrating its effectiveness on FL’s fairness enhancement.

Federated crowdsourcing, as a dynamic privacy-preserving distributed machine learning approach, has attracted significant research attention recently. Compared to federated learning (FL), clients can dynamically collect and label fresh data as required, and train model on the updated data. Existing research has mainly focused on incentivizing clients to spend more effort on data collection and labelling in order to improve FL model performance. However, as data collection and labeling require human effort, they need to balance work and rest. This need has been overlooked by existing federated crowdsourcing research. In this paper, we propose the Federated Workforce Management (FedWM) approach to bridge this important gap. It first measures the contribution of each client to the FL model, and estimates the urgency collecting new labelled data based on the rate of change of the contribution. Then, FedWM computes the working time taking into account of the client’s maximum productivity and self-reported mood. Finally, it takes both the urgency level of obtaining new data and clients’ productivity into consideration to provide scheduling services that advise the clients on work-rest balance in a given time slot based on Lyapunov optimization. Through theoretical analysis, we provide the performance bounds of FedWM. Through extensive experiments based on real-world datasets, we demonstrate that FedWM achieves significantly more advantageous tradeoffs between client rest and FL model performance compared to existing approaches. To the best of our knowledge, it is the first federated crowdsourcing framework designed to achieve productive laziness.

Medical image forgery has become an urgent issue in academia and medicine. Unlike natural images, images in the medical field are so sensitive that even minor manipulation can produce severe consequences. In view of the specificities of medical images, natural image forgery localization methods are difficult to generalize. While the unsatisfactory performance of existing medical image forgery localization methods, we propose MemAU-Net: a copy-move and splicing resistant medical image forgery localization network. We propose a novel attention gate named Memory-Enhanced Attention Gate (MAG), which effectively fuses shallow-deep features and improves feature representation to make attention better suit for medical image forgery localization tasks. To minimize holes and edge serrations caused by the delicate and blurred textures in medical images, we use dense CRF to smooth the boundaries, reduce false alarms and missed detection. Since there is no medical image forgery dataset publicly available, by using the copy-move and splicing forgery operations, we manually tamper and annotate two forged medical image datasets: OIAT (eye) and COVIDLT (lung) to verify the generality of the proposed model. The dataset includes the images from funds and different views of the lungs, corresponding to the specificity of medical images with flat grayscale changes and complex textures. The results show that by using MemAU-Net alone, we can improve the F-measure by 2.33% and 4.61% over the state-of-the-art baseline on these two datasets, respectively. Moreover, the raised precisions and enhanced visualizations with the addition of dense CRF indicate that it effectively removes false positives, fills holes and smooths edges. A good proof of the superiority of MemAU-Net's medical image forgery localization function is provided by these results. In addition, the proposed method achieves promising robustness to forgery post-processing such as rotation, scaling and anti-forensic attacks like noise.

As an emerging machine learning paradigm, federated learning (FL) allows multiple participants to train a shared global model collaboratively on decentralized data while protecting data privacy. But traditional FL is susceptible to adversarial poisoning attacks, the global model in an FL system poisoned by adversaries may fail to converge or present accuracy degradation. To defend against data poisoning and model poisoning attacks simultaneously, we propose a Federated learning framework with a Reputation-aware Robust Aggregation (FedRRA) rule. It involves a two-step adversary detection: 1) a DBSCAN algorithm excludes models with obviously biased parameters and 2) an accuracy evaluation process punishes models with low accuracy. The reputation calculated in the two-step detection determines that clients with low reputations are removed before the aggregation, which alleviates the negative influence of models corrupted by adversaries. Extensive experiments demonstrate that FedRRA is superior to the state-of-the-art robust FL baseline in defending against model poisoning and data poisoning attacks.

Federated learning (FL) is an emerging machine learning paradigm that allows multiple parties to train a shared model collaboratively in a privacy-preserving manner. Existing horizontal FL methods generally assume that the FL server and clients hold the same model structure. However, due to system heterogeneity and the need for personalization, enabling clients to hold models with diverse structures has become an important direction. Existing model-heterogeneous FL approaches often require publicly available datasets and incur high communication and/or computational costs, which limit their performances. To address these limitations, we propose a simple but effective Federated Global prediction Header (FedGH) approach. It is a communication and computation-efficient model-heterogeneous FL framework which trains a shared generalized global prediction header with representations extracted by heterogeneous extractors for clients' models at the FL server. The trained generalized global prediction header learns from different clients. The acquired global knowledge is then transferred to clients to substitute each client's local prediction header. We derive the non-convex convergence rate of FedGH. Extensive experiments on two real-world datasets demonstrate that FedGH achieves significantly more advantageous performance in both model-homogeneous and -heterogeneous FL scenarios compared to seven state-of-the-art personalized FL models, beating the best-performing baseline by up to 8.87% (for model-homogeneous FL) and 1.83% (for model-heterogeneous FL) in terms of average test accuracy, while saving up to 85.53% of communication overhead.

In cross-device Federated Learning (FL), the communication cost of transmitting full-precision models between edge devices and a central server is a significant bottleneck, due to expensive, unreliable, and low-bandwidth wireless connections. As a solution, we propose a novel FL framework named QSFL, towards optimizing FL uplink (client-to-server) communication at both client and model levels. At the client level, we design a Qualification Judgment (QJ) algorithm to sample high-qualification clients to upload models. At the model level, we explore a Sparse Cyclic Sliding Segment (SCSS) algorithm to further compress transmitted models. We prove that QSFL can converge over wall-to-wall time, and develop an optimal hyperparameter searching algorithm based on theoretical analysis to enable QSFL to make the best trade-off between model accuracy and communication cost. Experimental results show that QSFL achieves state-of-the-art compression ratios with marginal model accuracy degradation.