Federated Learning (FL) has become a prominent framework for maintaining privacy in recommender systems by enabling decentralized model training. Despite its benefits, traditional Federated Recommender Systems (FRSs)—often relying on collaborative filtering or generative models such as Variational Autoencoders (VAEs)—face limitations in capturing complex user-item interactions, resulting in suboptimal performance. Recent advancements in diffusion-based models, exemplified by L-DiffRec, have demonstrated superior capability in modeling intricate patterns. However, these models encounter significant challenges in federated settings, including data heterogeneity and slow convergence. To address these limitations, this paper introduces FedDiffRec, a novel federated framework that employs a module-wise training strategy and utilize a pseudo-interaction pretraining. Specifically, the VAE module is first trained locally, followed by fine-tuning with a diffusion module. Additionally, a pseudo-interaction pretraining mechanism is proposed to address challenges related to model initialization and convergence. Experimental results show that FedDiffRec enhances the stability and performance of diffusion-based models in federated environments, effectively bridging the performance gap between advanced diffusion-based approaches and traditional FRSs.

The bottleneck associated with the key-value(KV) cache presents a significant challenge during the inference processes of large language models. While depth pruning accelerates inference, it requires extensive recovery training, which can take up to two weeks. On the other hand, width pruning retains much of the performance but offers slight speed gains. To tackle these challenges, we propose KVPruner to improve model efficiency while maintaining performance. Our method uses global perplexity-based analysis to determine the importance ratio for each block and provides multiple strategies to prune non-essential KV channels within blocks. Compared to the original model, KVPruner reduces runtime memory usage by 50% and boosts throughput by over 35%. Additionally, our method requires only two hours of LoRA fine-tuning techniques on small datasets to recover most of the performance.

Federated recommender systems are used to address privacy issues in recommendations. Among them, FedVAE extends the representative non-linear recommendation method MultVAE. However, the bottleneck of FedVAE lies in its communication load during training, as the parameter volume of its first and last layers is correlated with the number of items. This leads to significant communication cost during the model's transmission phases (distribution and upload), making FedVAE's implementation extremely challenging. To address these challenges, we propose an Efficient Federated Variational AutoEncoder for collaborative filtering, EFVAE, which core is the Federated Collaborative Importance Sampling (FCIS) method. FCIS reduces communication costs through a client-to-server collaborative sampling mechanism and provides satisfactory recommendation performance through dynamic multi-stage approximation of the decoding distribution. Extensive experiments and analyses on real-world datasets confirm that EFVAE significantly reduces communication costs by up to 94.51% while maintaining the recommendation performance. Moreover, its recommendation performance is better on sparse datasets, with improvements reaching up to 13.79%.

Federated multimedia recommendation remains unexplored due to the high dimensionality of multimedia context, which limits the federated optimization on resource-constrained user devices. To address this issue, we propose a resource-efficient and secure federated learning framework for multimedia recommendation. Instead of training the entire model, we split the multimodal learning model to the powerful server, and the client trains the lightweight collaborative filtering model. Only the local model and item representations are transferred between the server and clients. We also propose an inter-client convolution strategy that utilizes secure multi-party computation to guarantee user privacy while alleviating heterogeneity among clients. We conduct evaluations on three datasets and demonstrate that our proposed method effectively exploits the modality features of items to improve performance while significantly reducing the communication and computation cost for clients.