The current paradigm of training large language models (LLMs) on public available Web data is becoming unsustainable as high-quality data sources in specialized domains near exhaustion. Federated Learning (FL) emerges as a practical solution for the next generation of AI on a decentralized Web, enabling privacy-preserving collaborative fine-tuning on decentralized private data. While Low-Rank Adaptation (LoRA) is standard for efficient fine-tuning, its federated application faces a critical bottleneck: communication overhead under heterogeneous network conditions. Structural redundancy in LoRA parameters increases communication costs and causes aggregation conflicts. To address this, we propose FedSRD, a Sparsify-Reconstruct-Decompose framework for communication-efficient federated LLM fine-tuning. We introduce importance-aware sparsification to reduce the upload parameter count while preserving the structural integrity of LoRA updates. The server aggregates updates in full-rank space to mitigate conflicts, then decomposes the global update into a sparse low-rank format for broadcast, ensuring a symmetrically efficient cycle. We also propose an efficient variant, FedSRD-e, to reduce computational overhead. Experiments on 10 benchmarks show our framework significantly reduces communication costs by up to 90% while improving performance on heterogeneous client data.

Federated learning has become a promising solution for collaboration among medical institutions. However, data owned by each institution would be highly heterogeneous and the distribution is always non-independent and identical distribution (non-IID), resulting in client drift and unsatisfactory performance. Despite existing federated learning methods attempting to solve the non-IID problems, they still show marginal advantages but rely on frequent communication which would incur high costs and privacy concerns. In this paper, we propose a novel federated learning method: Federated learning via Valuable Condensed Knowledge (FedVCK). We enhance the quality of condensed knowledge and select the most necessary knowledge guided by models, to tackle the non-IID problem within limited communication budgets effectively. Specifically, on the client side, we condense the knowledge of each client into a small dataset and further enhance the condensation procedure with latent distribution constraints, facilitating the effective capture of high-quality knowledge. During each round, we specifically target and condense knowledge that has not been assimilated by the current model, thereby preventing unnecessary repetition of homogeneous knowledge and minimizing the frequency of communications required. On the server side, we propose relational supervised contrastive learning to provide more supervision signals to aid the global model updating. Comprehensive experiments across various medical tasks show that FedVCK can outperform state-of-the-art methods, demonstrating that it's non-IID robust and communication-efficient.

Federated learning has wide applications in the medical field. It enables knowledge sharing among different healthcare institutes while protecting patients’ privacy. However, existing federated learning systems are typically centralized, requiring clients to upload client-specific knowledge to a central server for aggregation. This centralized approach would integrate the knowledge from each client into a centralized server, and the knowledge would be already undermined during the centralized integration before it reaches back to each client. Besides, the centralized approach also creates a dependency on the central server, which may affect training stability if the server malfunctions or connections are unstable. To address these issues, we propose a decentralized federated learning framework named dFLMoE. In our framework, clients directly exchange lightweight head models with each other. After exchanging, each client treats both local and received head models as individual experts, and utilizes a client-specific Mixture of Experts (MoE) approach to make collective decisions. This design not only reduces the knowledge damage with client-specific aggregations but also removes the dependency on the central server to enhance the robustness of the framework. We validate our framework on multiple medical tasks, demonstrating that our method evidently outperforms state-of-the-art approaches under both model homogeneity and heterogeneity settings.

Breast cancer remains a leading cause of death among women, with early detection significantly improving prognosis. Non-contrast computed tomography (NCCT) scans of the chest, routinely acquired for thoracic assessments, often capture the breast region incidentally, presenting an underexplored opportunity for opportunistic breast lesion detection without additional imaging cost or radiation. However, the subtle appearance of lesions in NCCT and the difficulty of jointly modeling lesion detection and malignancy classification pose unique challenges. In this work, we propose Dual-Res Tandem Mamba-3D (DRT-M3D), a novel multitask framework for opportunistic breast cancer analysis on NCCT scans. DRT-M3D introduces a dual-resolution architecture, which captures fine-grained spatial details for segmentation-based lesion detection and global contextual features for breast-level cancer classification. It further incorporates a tandem input mechanism that models bilateral breast regions jointly through Mamba-3D blocks, enabling cross-breast feature interaction by leveraging subtle asymmetries between the two sides. Our approach achieves state-of-the-art performance in both tasks across multi-institutional NCCT datasets spanning four medical centers. Extensive experiments and ablation studies validate the effectiveness of each key component.

Audio is an important medium in people’s daily life, secret information can be embedded into audio for covert communication. However, traditional audio information hiding techniques cannot achieve large hiding capacity and good imperceptibility at the same time, and rely on complex encryption, which limits their applicability in resource-constrained Internet of Things (IoT) environments. In this article, we propose a new audio information hiding method, named AdvAudio, which can achieve large high capacity, as well as good imperceptibility, without reliance on cryptographic encryption. Specifically, AdvAudio leverages adversarial example technique to train a well-designed perturbation for cover audio and the secret information can only be extracted by the private automatic speech recognition (ASR) model. To achieve this, we implement two adversarial example algorithms tailored for both online transmission and physical-world transmission scenarios. In particular, our embedding algorithm dynamically adjusts the addition of simulated environmental noise depending on whether the audio is intended to propagate in the physical world. The iterative optimization process is guided by targeted adversarial attack objectives, ensuring that the private ASR model decodes the embedded secret information accurately. Taking DeepSpeech as the private model, we implement a prototype of AdvAudio, which achieves a high embedding capacity of 383.8 bps with excellent imperceptibility, yielding a Perceptual Evaluation of Speech Quality (PESQ) score of 2.351. Furthermore, it offers robust security, achieving a 100% defense success rate against both internal and external attacks. In the physical world, AdvAudio still maintains effectiveness across six different types of noise and retaining 82% accuracy even under sudden loud noises. Additionally, the secret information can only be extracted in the target environment, with a success rate of 26%, and 0% in non-target environments. In the future, we aim at enhancing the steganalysis resistance of AdvAudio and explore its potential applications in various environments or with alternative ASR models.

Representation learning frameworks in unlabeled time series have been proposed for medical signal processing. Despite the numerous excellent progresses have been made in previous works, we observe the representation extracted for the time series still does not generalize well. In this paper, we present a Time series (medical signal) Representation Learning framework via Spectrogram (TRLS) to get more informative representations. We transform the input time-domain medical signals into spectrograms and design a time-frequency encoder named Time Frequency RNN (TFRNN) to capture more robust multi-scale representations from the augmented spectrograms. Our TRLS takes spectrogram as input with two types of different data augmentations and maximizes the similarity between positive ones, which effectively circumvents the problem of designing negative samples. Our evaluation of four real-world medical signal datasets focusing on medical signal classification shows that TRLS is superior to the existing frameworks. We will open-source our code when the paper is accepted.

Federated learning is widely used in medical applications for training global models without needing local data access, but varying computational capabilities and network architectures (system heterogeneity) across clients pose significant challenges in effectively aggregating information from non-independently and identically distributed (non-IID) data (statistic heterogeneity). Current federated learning methods using knowledge distillation require public datasets, raising privacy and data collection issues. Additionally, these datasets require additional local computing and storage resources, which is a burden for medical institutions with limited hardware conditions. In this paper, we introduce a novel federated learning paradigm, named Model Heterogeneous personalized Federated Learning via Injection and Distillation (MH-pFLID). Our framework leverages a lightweight messenger model, eliminating the need for public datasets and reducing the training cost for each client. We also develops receiver and transmitter modules for each client to separate local biases from generalizable information, reducing biased data collection and mitigating client drift. Our experiments on various medical tasks including image classification, image segmentation, and time-series classification, show MH-pFLID outperforms state-of-the-art methods in all these areas and has good generalizability.

Computer-aided diagnosis plays an increasingly important role in modern medical activities, relying largely on the deployment of medical machine learning models. Protecting the security of model parameters is crucial for model providers. However, the current schemes for protecting model parameters are mostly interactive. This interactive nature makes it difficult to support offline deployment of models and flexible authorization of prediction results, thus hindering the widespread application of computer-aided diagnosis. To address these limitations, we propose a new computer-aided medical diagnosis framework by designing a new identity-based inner product functional proxy re-encryption (IB-IPFPRE) scheme. Our framework supports private deployment of medical diagnostic models without compromising model parameters. It also enables access control of prediction results based on user identity. Compared to existing privacy-preserving prediction techniques, our framework significantly reduces communication overhead and does not require the model owner to be online in real-time. Furthermore, our scheme enables flexible delegation of prediction results, allowing users to authorize the sharing of prediction results with other entities as needed. We conducted extensive experiments for logistic regression on three medical datasets. The experiments demonstrate that our scheme achieved 40% to 7× performance improvement in LAN environment and 13× to 15× improvement in WAN environment, and did not require any communication overhead during the privacy preserving prediction phase.