Federated Learning (FL) has emerged as a promising technique for collaboratively training machine learning models among multiple participants while preserving privacy-sensitive data. However, the conventional parameter server architecture presents challenges in terms of communication overhead when employing iterative optimization methods such as Stochastic Gradient Descent (SGD). Although communication compression techniques can reduce the traffic cost of FL during each training round, they often lead to degraded convergence rates, mainly due to compression errors and data heterogeneity. To address these issues, this paper presents FedQClip, an innovative approach that combines quantization and Clipped SGD. FedQClip leverages an adaptive step size inversely proportional to the$\ell_{2}$norm of the gradient, effectively mitigating the negative impacts of quantized errors. Additionally, clipped operations can be applied locally and globally to further expedite training. Theoretical analyses provide evidence that, even under the settings of Non-IID (non-independent and identically distributed) data, FedQClip achieves a convergence rate of$\mathcal{O}(\frac{1}{\sqrt{T}})$, effectively addressing the convergence degradation caused by compression errors. Furthermore, our theoretical analysis highlights the importance of selecting an appropriate number of local updates to enhance the convergence of FL training. Through extensive experiments, we demonstrate that FedQClip outperforms state-of-the-art methods in terms of communication efficiency and convergence rate.

Graph Neural Networks (GNNs) have emerged as a powerful technique across various applications, due to their effective message-passing mechanism. However, their deployment is constrained by limited computational resources, energy concerns, and low-latency processing requirements. While existing works employ logit-based knowledge distillation from GNNs to guide Multilayer Perceptrons (MLPs) training, these methods may lead to reduced accuracy and compromised robustness. These drawbacks arise from two primary factors: the insufficient exploitation of the rich information embedded within the graph structures and the inherent susceptibility of MLPs to noisy data. To tackle these issues, we propose a Mixed Multi-order Knowledge Distillation (MMKD) method, which combines the GNN's logits with hidden layer information through the multi-order distillation to improve the accuracy of the MLP. Moreover, we employ both raw data and perturbed data as input, enhancing the density of knowledge extraction as well as the MLPs' generalization. Extensive experiments across seven benchmark datasets verify the superior performance of our approach in terms of effectiveness and robustness. In comparison with the baseline, our approach achieves an accuracy improvement of up to 8.68% in typical GNN tasks.

Trusted Execution Environment(TEE) offers a secure data processing zone for model inference. Due to its limited resources, existing solutions like MirrorNet usually deploy a lightweight model within the TEE for sensitive data, and a backbone model outside for the rest. However, these approaches do not inherently limit the learning ability of the backbone model, which could acquire inference capabilities similar to the lightweight model, inevitably weakening the security. To counter this, we propose FakeNN, an innovative mechanism with a dual-network architecture, that intentionally guides the backbone model towards low predictive performance, thereby reducing its ability to infer sensitive information. We further improve the accuracy of the entire model by integrating a channel attention mechanism which reduces the transmission of redundant information. We conduct extensive experiments and the results demonstrate that FakeNN substantially expands the performance gap between the non-secure and secure TEE models, with improvements ranging from 3.16% to 66.42% compared to MirrorNet. This enhancement strengthens the system's security without negatively impacting the accuracy of the model.