Federated learning (FL) is a popular distributed machine learning framework in which clients aggregate models' parameters instead of sharing individual data.In FL, clients frequently communicate with the server under limited network bandwidth, raising the communication challenge.Multiple compression methods have been proposed to reduce the transmitted parameters.However, these techniques show that the federated performance degrades significantly with Non-IID (non-identically independently distributed) datasets.To address this issue, we propose an effective method called FedPSE, which solves the efficiency challenge of FL with heterogeneous data.FedPSE compresses the local updates on clients using Top-K sparsification and aggregates these updates on the server by element-wise aggregation.Then clients download the personalized sparse updates from the server to update their individual local models.We then theoretically analyze the convergence of FedPSE under the non-convex setting.Moreover, extensive experiments on four benchmark tasks demonstrate that our FedPSE outperforms the state-of-the-art methods on Non-IID datasets in terms of efficiency and accuracy.

Most existing federated learning algorithms are based on the vanilla FedAvg scheme. However, with the increase of data complexity and the number of model parameters, the amount of communication traffic and the number of iteration rounds for training such algorithms increases significantly, especially in non-independently and homogeneously distributed scenarios, where they do not achieve satisfactory performance. In this work, we propose FedND: federated learning with noise distillation. The main idea is to use knowledge distillation to optimize the model training process. In the client, we propose a self-distillation method to train the local model. In the server, we generate noisy samples for each client and use them to distill other clients. Finally, the global model is obtained by the aggregation of local models. Experimental results show that the algorithm achieves the best performance and is more communication-efficient than state-of-the-art methods.

With the development of modern internet techniques, Cross-Domain Recommendation (CDR) systems have been widely exploited for tackling the data-sparsity problem. Meanwhile most current CDR models assume that user-item interactions are accessible across different domains. However, such knowledge sharing process will break the privacy protection policy. In this paper, we focus on the Privacy-Preserving Multi-Domain Recommendation problem (PPMDR). The problem is challenging since different domains are sparse and heterogeneous with the privacy protection. To tackle the above issues, we propose Federated Probabilistic Preference Distribution Modelling (FPPDM). FPPDM includes two main components, i.e., local domain modelling component and global server aggregation component with federated learning strategy. The local domain modelling component aims to exploit user/item preference distributions using the rating information in the corresponding domain. The global server aggregation component is set to combine user characteristics across domains. To better extract semantic neighbors information among the users, we further provide compactness co-clustering strategy in FPPDM ++ to cluster the users with similar characteristics. Our empirical studies on benchmark datasets demonstrate that FPPDM/ FPPDM ++ significantly outperforms the state-of-the-art models.

Federated learning (FL) naturally faces the problem of data heterogeneity in real-world scenarios, but this is often overlooked by studies on FL security and privacy. On the one hand, the effectiveness of backdoor attacks on FL may drop significantly under non-IID scenarios. On the other hand, malicious clients may steal private data through privacy inference attacks. Therefore, it is necessary to have a comprehensive perspective of data heterogeneity, backdoor, and privacy inference. In this paper, we propose a novel privacy inference-empowered stealthy backdoor attack (PI-SBA) scheme for FL under non-IID scenarios. Firstly, a diverse data reconstruction mechanism based on generative adversarial networks (GANs) is proposed to produce a supplementary dataset, which can improve the attacker's local data distribution and support more sophisticated strategies for backdoor attacks. Based on this, we design a source-specified backdoor learning (SSBL) strategy as a demonstration, allowing the adversary to arbitrarily specify which classes are susceptible to the backdoor trigger. Since the PI-SBA has an independent poisoned data synthesis process, it can be integrated into existing backdoor attacks to improve their effectiveness and stealthiness in non-IID scenarios. Extensive experiments based on MNIST, CIFAR10 and Youtube Aligned Face datasets demonstrate that the proposed PI-SBA scheme is effective in non-IID FL and stealthy against state-of-the-art defense methods.

Cross-Domain Recommendation has been popularly studied to utilize different domain knowledge to solve the cold-start problem in recommender systems. In this paper, we focus on theCross-Domain Cold-Start Recommendation(CDCSR) problem. That is, how to leverage the information from a source domain, where items are ’warm’, to improve the recommendation performance of a target domain, where items are ’cold’. It has two main challenges, i.e., (1) how to efficiently reduce the discrepancy between the latent embedding distribution across domains and (2) how to generate more robust and stable cold item embeddings. To address these two challenges, we proposeCPKSPA, a cross-domain recommendation framework for the CDCSR problem.CPKSPAcontains three modules, i.e., rating prediction module, embedding distribution alignment module, and contrastive augmentation module. To start with, we first utilize the rating prediction module to model user-item interactions. To solve the first challenge, we propose proxy Stein path alignment with typical-subgroup discovering algorithm in the embedding distribution alignment module. To tackle the second challenge, we propose the contrastive augmentation module which adopts contrastive augmentation learning to generate more stable and robust cold item embeddings. Our empirical study on Douban and Amazon datasets demonstrates thatCPKSPAsignificantly outperforms the state-of-the-art models.

Recently, Graph Neural Network (GNN) has achieved remarkable progresses in various real-world tasks on graph data, consisting of node features and the adjacent information between different nodes. High-performance GNN models always depend on both rich features and complete edge information in graph. However, such information could possibly be isolated by different data holders in practice, which is the so-called data isolation problem. To solve this problem, in this paper, we propose VFGNN, a federated GNN learning paradigm for privacy-preserving node classification task under data vertically partitioned setting, which can be generalized to existing GNN models. Specifically, we split the computation graph into two parts. We leave the private data (i.e., features, edges, and labels) related computations on data holders, and delegate the rest of computations to a semi-honest server. We also propose to apply differential privacy to prevent potential information leakage from the server. We conduct experiments on three benchmarks and the results demonstrate the effectiveness of VFGNN.

Deep Neural Networks (DNNs) have achieved remarkable progress in various real-world applications, especially when abundant training data are provided. However, data isolation has become a serious problem currently. Existing works build privacy-preserving DNN models from either algorithmic perspective or cryptographic perspective. The former mainly splits the DNN computation graph between data holders or between data holders and server, which demonstrates good scalability but suffers from accuracy loss and potential privacy risks. In contrast, the latter leverages time-consuming cryptographic techniques, which has strong privacy guarantee but poor scalability. In this article, we propose SPNN—a Scalable and Privacy-preserving deep Neural Network learning framework, from an algorithmic-cryptographic co-perspective. From algorithmic perspective, we split the computation graph of DNN models into two parts, i.e., the private-data-related computations that are performed by data holders and the rest heavy computations that are delegated to a semi-honest server with high computation ability. From cryptographic perspective, we propose using two types of cryptographic techniques, i.e., secret sharing and homomorphic encryption, for the isolated data holders to conduct private-data-related computations privately and cooperatively. Furthermore, we implement SPNN in a decentralized setting and introduce user-friendly APIs. Experimental results conducted on real-world datasets demonstrate the superiority of our proposed SPNN.

Gestures are a common form of human communication and important for Human-Computer Interaction (HCI). In this paper, we propose a new approach for skeleton-based hand gesture recognition based on the Independently Recurrent Neural Network (IndRNN). First, a bidirectional IndRNN (Bi-IndRNN) is developed to extend the IndRNN with the capability of bidirectional processing. Then, a deep Bi-IndRNN network is constructed for gesture recognition, where, in addition to the joint coordinates, the temporal displacement of each joint is also used to enhance the input features. Experimental results demonstrate that the proposed method achieves the state-of-the-art performance on the widely used DHG dataset with an accuracy of 93.15% for the 14 gesture classes case and 91.13% for the 28 gesture classes case.

This article is concerned with the periodic tracking control problem for nonlinear systems. First, the Takagi-Sugeno (T-S) fuzzy model is employed to describe the nonlinear control systems. Second, considering the partly unmeasurable states of the system, a novel fuzzy observer-based repetitive controller, which is the mixed controller of the fuzzy observer-based controller and the fuzzy repetitive controller, is designed to deal with the periodic tracking control problem. To reduce the conservatism and increase the feasible solution space of the stabilization conditions, a new fuzzy relaxed matrix technique is developed by introducing some relaxed matrices in the derivative of the fuzzy normalized membership function. Then, the fuzzy Lyapunov functional with an additional separation parameter and the augmented fuzzy matrix technique (the interactions of fuzzy observer subsystems) are proposed such that the delay-dependent stability condition of the closed-loop system in the form of linear matrix inequality is obtained with less conservatism. It is worth noting that due to introducing an additional parameter in the fuzzy Lyapunov functional, the fuzzy controller and fuzzy observer can be separately designed, which largely enhances the flexibility of design with low computational complexity. Finally, three examples are provided to illustrate the effectiveness and less conservatism of the proposed method.