As a dominant pre-training paradigm, molecular contrastive learning (MCL) has been proven effective in learning molecular representations with unlabeled data. However, high data dependency and scarce domain knowledge caused by data augmentation in MCL limit the model’s generalization and performance. To address these issues, we propose an adaptive pre-training framework with knowledge-enhanced for molecular representation learning, named APK-MRL. It seamlessly integrates diverse prior information on hierarchical skeletons and the chemical semantics of molecules, aiming to obtain stronger stability and generalization. Extensive computational experiments demonstrate that APK-MRL can achieve competitive performances over state-of-the-art baselines on both drug-target interaction and molecular properties prediction tasks. All code is released at https://github.com/lukcats/APK-MRL.

Due to the great successes of Graph Neural Networks (GNN) in numerous fields, growing research interests have been devoted to applying GNN to molecular learning tasks. The molecule structure can be naturally represented as graphs where atoms and bonds refer to nodes and edges respectively. However, the atoms are not haphazardly stacked together but combined into various spatial geometries. Meanwhile, since chemical reactions mainly occur in substructures such as functional groups, the substructure plays a decisive role in the molecule's properties. Therefore, directly applying GNN to molecular representation learning could ignore the molecular spatial structure and the substructure properties which in turn degrades the performance of downstream tasks. In this paper, we propose Knowledge-Driven Self-Supervised Model for Molecular Representation Learning (KSMRL) to address above problems. The KSMRL consists of two major pathways: (1) the Spatial Information (SI) based pathway which preserves the spatial information of molecular structure, (2) the Subgraph Constraint (SC) based pathway which retains the properties of substructures into the molecular representation. In this manner, both the atomic level and substructure level information can be included in modeling. According to the experimental results on multiple datasets, the proposed KSMRL can generate discriminative molecular representations. In molecular generation tasks, KSMRL combined with Autoregressive Flow (AF) models or Discrete Flow (DF) models outperforms the state-of-the-art baselines over all datasets. In addition, we demonstrate the effectiveness of KSMRL with property optimization experiments. To indicate the ability of predicting specified potential Drug-Target Interactions (DTIs), a case study for discriminating the interactions between molecule generated by KSMRL and targets is also given.

Aligning users across networks using graph representation learning has been found effective where the alignment is accomplished in a low-dimensional embedding space. Yet, highly precise alignment remains challenging, especially for nodes with long-range connectivity to labeled anchors. To alleviate this limitation, we propose WL-Align which employs a regularized representation learning framework to learn distinctive node representations. It extends the Weisfeiler-Lehman Isormorphism Test and learns the alignment in alternating phases of “across-network Weisfeiler-Lehman relabeling” and “proximity-preserving representation learning”. The across-network Weisfeiler-Lehman relabeling is achieved through iterating the anchor-based label propagation and a similarity-based hashing to exploit the known anchors’ connectivity to different nodes in an efficient and robust manner. The representation learning module preserves the second-order proximity within individual networks and is regularized by the across-network Weisfeiler-Lehman hash labels. Extensive experiments on real-world and synthetic datasets have demonstrated that our proposed WL-Align outperforms the state-of-the-art methods, achieving significant performance improvements in the “exact matching” scenario.

Social network alignment aims at aligning person identities across social networks. Embedding based models have been shown effective for the alignment where the structural proximity preserving objective is typically adopted for the model training. With the observation that “overly-close” user embeddings are unavoidable for such models causing alignment inaccuracy, we propose a novel learning framework which tries to enforce the resulting embeddings to be more widely apart among the users via the introduction of carefully implanted pseudo anchors. We further proposed a meta-learning algorithm to guide the updating of the pseudo anchor embeddings during the learning process. The proposed intervention via the use of pseudo anchors and meta-learning allows the learning framework to be applicable to a wide spectrum of network alignment methods. We have incorporated the proposed learning framework into several state-of-the-art models. Our experimental results demonstrate its efficacy where the methods with the pseudo anchors implanted can outperform their counterparts without pseudo anchors by a fairly large margin, especially when there only exist very few labeled anchors.

Deep graph neural networks have recently demonstrated powerful representation learning capabilities in bioinformatics. It is still crucial and challenging to design a representation model fusing the fundamental chemistry and biology knowledge. However, most existing representation models not only ignore domain knowledge but are also built on labeled datasets. To address this issue, this paper proposed a hierarchical structure-aware pre-training model that used contrastive learning to improve molecular representations with unlabeled datasets. We conducted comprehensive experiments on 13 molecular benchmark datasets from different application domains. The results demonstrate that our hierarchical structure-aware pre-trained model achieves superior performance against state-of-the-art baselines.

Aligning users across different social networks has become increasingly studied as an important task to social network analysis. In this paper, we propose a novel representation learning method that mainly exploits social structures for the network alignment. In particular, the proposed network embedding framework models the follower-ship and followee-ship of each user explicitly as input and output context vectors, while preserving the proximity of users with “similar” followers and followees in the embedded space. We incorporate both known and predicted user anchors across the networks as constraints to facilitate the transfer of context information to achieve accurate user alignment. Both network embedding and user alignment are inferred under a unified optimization framework with negative sampling adopted to ensure scalability. Also, variants of the proposed framework, including the incorporation of higher-order structural features, are also explored for further boosting the alignment accuracy. Extensive experiments on large-scale social and academia network datasets demonstrate the efficacy of our proposed model compared with state-of-the-art methods.