Graph contrastive learning (GCL) has emerged as a representative paradigm in graph self-supervised learning, where negative samples are commonly regarded as the key to preventing model collapse and producing distinguishable representations. Recent studies have shown that GCL without negative samples can achieve state-of-the-art performance as well as scalability improvement, with bootstrapped graph latent (BGRL) as a prominent step forward. However, BGRL relies on a complex architecture to maintain the ability to scatter representations, and the underlying mechanisms enabling the success remain largely unexplored. In this paper, we introduce an instance-level decorrelation perspective to tackle the aforementioned issue and leverage it as a springboard to reveal the potential unnecessary model complexity within BGRL. Based on our findings, we present SGCL, a simple yet effective GCL framework that utilizes the outputs from two consecutive iterations as positive pairs, eliminating the negative samples. SGCL only requires a single graph augmentation and a single graph encoder without additional parameters. Extensive experiments conducted on various graph benchmarks demonstrate that SGCL can achieve competitive performance with fewer parameters, lower time and space costs, and significant convergence speedup.

Multivariate Time Series Anomaly Detection (MTS-AD) is crucial for the effective management and maintenance of devices in complex systems, such as server clusters, spacecrafts, and financial systems, and so on. However, upgrade or cross-platform deployment of these devices will introduce the issue of cross-domain distribution shift, which leads to the prototypical problem of domain adaptation for MTS-AD. Compared with general domain adaptation problems, MTS-AD domain adaptation presents two peculiar challenges: (1) the dimensions of data from the source domain and the target domain are usually different, so alignment without losing any information is necessary; and (2) the association between different variates plays a vital role in the MTS-AD task, which is overlooked by traditional domain adaptation approaches. Aiming at addressing the above issues, we propose a Variate Associated Domain Adaptation Method Combined with a Graph Deviation Network (VANDA) for MTS-AD, which includes two major contributions. First, we characterize the intra-domain variate associations of the source domain by a graph deviation network (GDN), which can share parameters across domains without dimension alignment. Second, we propose a sliding similarity to measure the inter-domain variate associations and perform joint training by minimizing the optimal transport distance between source and target data for transferring variate associations across domains. VANDA achieves domain adaptation by transferring both variate associations and GDN parameters from the source domain to the target domain. We construct two pairs of MTS-AD datasets from existing MTS-AD data and combine three domain adaptation strategies with six MTS-AD backbones as the benchmark methods for experimental evaluation and comparison. Extensive experiments demonstrate the effectiveness of our approach, which outperforms the benchmark methods, and significantly improves the AD performance of the target domain by effectively utilizing the source domain knowledge.

Graph neural networks (GNNs) have achieved state-of-the-art performance in representation learning for graphs recently. However, the effectiveness of GNNs, which capitalize on the key operation of message propagation, highly depends on the quality of the topology structure. Most of the graphs in real-world scenarios follow a long-tailed distribution on their node degrees, that is, a vast majority of the nodes in the graph are tail nodes with only a few connected edges. GNNs produce inferior node representations for tail nodes due to the lack of sufficient structural information. In the pursuit of promoting the performance of GNNs for tail nodes, we explore how the deficiency of structural information deteriorates the performance of tail nodes and propose a general structural augmentation based tailno de representation learning framework, dubbed as øurs, which can jointly learn to augment the graph structure and extract more informative representations for tail nodes. Extensive experiments on six public benchmark datasets demonstrate that øurs outperforms the state-of-the-art methods for tail node representation learning.

Recent awareness of privacy protection and compliance requirement resulted in a controversial view of recommendation system due to personal data usage. Therefore, privacy-protected recommendation emerges as a novel research direction. In this paper, we first formulate this problem as a vertical federated learning problem, i.e., features are vertically distributed over different departments. We study a contextual bandit learning problem for recommendation in the vertical federated setting. To this end, we carefully design a customized encryption scheme named orthogonal matrix-based mask mechanism (O3M). O3M mechanism, a tailored component for contextual bandits by carefully exploiting their shared structure, can ensure privacy protection while avoiding expensive conventional cryptographic techniques. We further apply the mechanism to two commonly-used bandit algorithms, LinUCB and LinTS, and instantiate two practical protocols for online recommendation. The proposed protocols can perfectly recover the service quality of centralized bandit algorithms while achieving a satisfactory runtime efficiency, which is theoretically proved and analysed in this paper. By conducting extensive experiments on both synthetic and real-world datasets, we show the superiority of the proposed method in terms of privacy protection and recommendation performance.

Deep graph learning (DGL) has achieved remarkable progress in both business and scientific areas ranging from finance and e-commerce, to drug and advanced material discovery. Despite the progress, how to ensure various DGL algorithms behave in a socially responsible manner and meet regulatory compliance requirements becomes an emerging problem, especially in risk-sensitive domains. Trustworthy graph learning (TwGL) aims to solve the above problems from a technical viewpoint. In contrast to conventional graph learning which mainly cares about model performance, TwGL considers various reliability and safety aspects of DGL, including but not limited to adversarial robustness, explainability, and privacy protection. Whilst several previous tutorials have been made for the introduction of DGL in KDD, seldom is there a special focus on its safety aspects, including reliability, explainability, and privacy protection capability. This tutorial mainly covers the key achievements of trustworthy graph learning in recent years. Specifically, we will discuss three essential topics, that is, the reliability of DGL against inherent noise, distribution shift and adversarial attack, explainability methods, and privacy protection for DGL. Meanwhile, we will introduce some guidelines for applying DGL to risk-sensitive applications (e.g., AI drug discovery) to ensure GNN models behave in a trustworthy way. We hope our tutorial can offer a comprehensive review of recent advances in this area and also provide some useful suggestions to guide the developers to choose appropriate techniques for their applications.

Deep Neural Networks (DNNs) have achieved remarkable progress in various real-world applications, especially when abundant training data are provided. However, data isolation has become a serious problem currently. Existing works build privacy-preserving DNN models from either algorithmic perspective or cryptographic perspective. The former mainly splits the DNN computation graph between data holders or between data holders and server, which demonstrates good scalability but suffers from accuracy loss and potential privacy risks. In contrast, the latter leverages time-consuming cryptographic techniques, which has strong privacy guarantee but poor scalability. In this article, we propose SPNN—a Scalable and Privacy-preserving deep Neural Network learning framework, from an algorithmic-cryptographic co-perspective. From algorithmic perspective, we split the computation graph of DNN models into two parts, i.e., the private-data-related computations that are performed by data holders and the rest heavy computations that are delegated to a semi-honest server with high computation ability. From cryptographic perspective, we propose using two types of cryptographic techniques, i.e., secret sharing and homomorphic encryption, for the isolated data holders to conduct private-data-related computations privately and cooperatively. Furthermore, we implement SPNN in a decentralized setting and introduce user-friendly APIs. Experimental results conducted on real-world datasets demonstrate the superiority of our proposed SPNN.

With the rapid growth of data volume, data-driven machine learning models have become a necessary part of many industrial applications. Intuitively, the more high-quality data used for training leads to better model performance. However, in reality, data are usually scattered and isolated in different organizations or companies. Such a "data isolation" problem stimulates both academia and industry to explore the collaborative learning paradigm to build better models jointly with multiple data sources. Despite the potential performance gains, this learning paradigm inevitably faces privacy issues, especially for the Fintech domain where data are sensitive by nature. In this paper, we present a privacy-preserving collaborative learning system in Ant Financial, named Nebula. Our system aims to facilitate privacy-preserving collaborative model training for industrial-scale applications. Our system is built upon a ring-allreduce MPI based distributed framework. On top of that, with some optimization strategies and novel sharing scheme, our system is able to scale up to tens of millions of data samples with hundreds of thousands of features and achieve more than 100x speedup compared with the existing state-of-the-art implementations.

Point-of-Interest (POI) recommendation has been extensively studied and successfully applied in industry recently. However, most existing approaches build centralized models on the basis of collecting users’ data. Both private data and models are held by the recommender, which causes serious privacy concerns. In this article, we propose a novel Privacy preserving POI Recommendation (PriRec) framework. First, to protect data privacy, users’ private data (features and actions) are kept on their own side, e.g., Cellphone or Pad. Meanwhile, the public data that need to be accessed by all the users are kept by the recommender to reduce the storage costs of users’ devices. Those public data include: (1) static data only related to the status of POI, such as POI categories, and (2) dynamic data dependent on user-POI actions such as visited counts. The dynamic data could be sensitive, and we develop local differential privacy techniques to release such data to the public with privacy guarantees. Second, PriRec follows the representations of Factorization Machine (FM) that consists of a linear model and the feature interaction model. To protect the model privacy, the linear models are saved on the users’ side, and we propose a secure decentralized gradient descent protocol for users to learn it collaboratively. The feature interaction model is kept by the recommender since there is no privacy risk, and we adopt a secure aggregation strategy in a federated learning paradigm to learn it. To this end, PriRec keeps users’ private raw data and models in users’ own hands, and protects user privacy to a large extent. We apply PriRec in real-world datasets, and comprehensive experiments demonstrate that, compared with FM, PriRec achieves comparable or even better recommendation accuracy.