Retrieval-Augmented Generation (RAG) grounds large language models (LLMs) in external knowledge, yet it is faces a fundamental trade-off between knowledge confidentiality and retrieval efficiency. Existing approaches fail to reconcile this trade-off: i) Cryptography-based solutions (e.g., homomorphic encryption and secure multi-party computation) provide strong privacy guarantees but incur prohibitive computation and communication overheads. ii) Lightweight perturbation-based methods (e.g., differential privacy) offer higher efficiency at the cost of degraded retrieval accuracy or weakened security guarantees. In this paper, we propose CipheRAG, an efficient vector-multiplicative privacy-preserving RAG framework that achieves a principled balance between robust privacy and high-performance retrieval. Technically, we first propose an efficient searchable inner product functional encryption (IPFE) mechanism enhanced with asymmetric locality-sensitive hashing (ALSH), enabling the retrieval of sensitive knowledge while effectively preserving data confidentiality. Secondly, we propose a decryption-enabled attention mechanism that uses linear weights of the attention layer to decrypt knowledge. This mechanism seamlessly integrates decrypted knowledge into the LLM's generation process, achieving efficiency and accuracy. Extensive experiments demonstrate that CipheRAG achieves up to 35x faster generation and 15x faster QKV computation compared to FHE- and OT-based baselines. By avoiding linear retrieval and full-parameter encryption, CipheRAG enables privacy-preserving RAG with bounded computational and communication overheads, making it well-suited for deployment in privacy-sensitive environments.

Differential private learning is widely used in machine learning (ML) to protect continuous and scalar-valued data. The demand for discrete and matrix-valued computations is increasing, particularly in quantized neural networks and graph learning, which require discrete-valued parameters and large-scale matrix operations for efficient data processing. However, privacy protection for discrete and matrix-valued data is less explored. Traditional differential private mechanisms fail to maintain the discrete nature of data after perturbation and often overlook data correlations, struggling to balance privacy and utility. In this paper, we propose a Discrete Matrixing Differentially Private Learning (DM-DPL) framework, which protects the privacy of discrete and matrix-valued data during ML training by adding discrete matrix-variate Gaussian noise. First, we propose a novel Discrete Matrix-Variate Gaussian (DMVG) mechanism with rigorous conditions necessary to guarantee (ϵ, δ)-differential privacy. Additionally, we present an eigenvalue-weighted analysis-based precision budget allocation strategy, designed to maintain the utility of significant dimensions while providing consistent privacy guarantees. Finally, the results illustrate that our approach significantly surpasses existing state-of-the-art methods when applied to quantized federated learning. To the best of our knowledge, this is the first work to specifically protect discrete and matrix-valued data during ML training.

Federated graph attention networks (FGATs) are gaining prominence for enabling collaborative and privacy-preserving graph model training. The attention mechanisms in FGATs enhance the focus on crucial graph features for improved graph representation learning while maintaining data decentralization. However, these mechanisms inherently process sensitive information, which is vulnerable to privacy threats like graph reconstruction and attribute inference. Additionally, their role in assigning varying and changing importance to nodes challenges traditional privacy methods to balance privacy and utility across varied node sensitivities effectively. Our study fills this gap by proposing an efficient privacy-preserving FGAT (PFGAT). We present an attention-based dynamic differential privacy (DP) approach via an improved multiplication triplet (IMT). Specifically, we first propose an IMT mechanism that leverages a reusable triplet generation method to efficiently and securely compute the attention mechanism. Second, we employ an attention-based privacy budget that dynamically adjusts privacy levels according to node data significance, optimizing the privacy-utility trade-off. Third, the proposed hybrid neighbor aggregation algorithm tailors DP mechanisms according to the unique characteristics of neighbor nodes, thereby mitigating the adverse impact of DP on graph attention network (GAT) utility. Extensive experiments on benchmarking datasets confirm that PFGAT maintains high efficiency and ensures robust privacy protection against potential threats.

It has become a great challenge to balance accuracy and diversity in recommendation systems. Graph Neural Networks (GNNs), while powerful, can lead to node representation homogeneity and information redundancy due to the indiscriminate aggregation of neighbor information. Therefore we propose a novel method that integrates maximum entropy neighbor selection with graph contrast learning to enhance the diversity of recommendations. The method introduces a strategy for neighbor selection based on maximum entropy to ensure a diverse subset of neighbors is chosen during the aggregation phase. A layer attention mechanism is implemented to address the over-smoothing issue, directing greater focus on higher-order neighbors. Furthermore, a loss re-weighting technique is applied to emphasize the learning of long-tail items. The overarching objective is to significantly improve recommendation diversity while maintaining system accuracy, underpinned by graph contrast learning method. Experimental results on the Beauty and MIND-small datasets demonstrate significant enhancements in the accuracy and diversity metrics of the proposed method. In particular, regarding the Recall@300 metric, a substantial improvement of up to 30.56% is observed. Conversely, the method experiences a mere 4.63% reduction in accuracy compared to the optimal baseline. This indicates that the proposed method markedly amplifies the diversity of recommendations without significantly compromising recommendation accuracy.

The emerging federated learning (FL) offers a feasible solution for the privacy preservation of users' sensitive data in training artificial intelligence (AI) models. Meanwhile, differential privacy (DP) is widely used in FL to ensure that data privacy is not disclosed during model training. However, in the practical deployment of DP in FL, a prominent challenge is that most existing FL solutions set the same privacy level for different users, resulting in over-protection for some users while insufficient protection for others. In this paper, we propose a novel federated learning framework with user-level personalized privacy protection (named FLUP) to meet the personalized privacy requirements of different users while maintaining high data utility. In this framework, we propose a user-level personalized DP mechanism that combines a personalized sampling algorithm and Gaussian perturbation to meet each user's personalized differential privacy corresponding to their privacy parameters. Then, we qualitatively analyze the impact of the sampling threshold on model performance. Furthermore, to balance user privacy requirements and AI model performance, we design a utility-aware game model to distributively determine the optimized sampling threshold and the users' differential privacy parameters. Finally, by conducting validation experiments, we demonstrate the feasibility and effectiveness of our proposed framework in terms of model performance as well as user privacy preservation.