Due to the great successes of Graph Neural Networks (GNN) in numerous fields, growing research interests have been devoted to applying GNN to molecular learning tasks. The molecule structure can be naturally represented as graphs where atoms and bonds refer to nodes and edges respectively. However, the atoms are not haphazardly stacked together but combined into various spatial geometries. Meanwhile, since chemical reactions mainly occur in substructures such as functional groups, the substructure plays a decisive role in the molecule's properties. Therefore, directly applying GNN to molecular representation learning could ignore the molecular spatial structure and the substructure properties which in turn degrades the performance of downstream tasks. In this paper, we propose Knowledge-Driven Self-Supervised Model for Molecular Representation Learning (KSMRL) to address above problems. The KSMRL consists of two major pathways: (1) the Spatial Information (SI) based pathway which preserves the spatial information of molecular structure, (2) the Subgraph Constraint (SC) based pathway which retains the properties of substructures into the molecular representation. In this manner, both the atomic level and substructure level information can be included in modeling. According to the experimental results on multiple datasets, the proposed KSMRL can generate discriminative molecular representations. In molecular generation tasks, KSMRL combined with Autoregressive Flow (AF) models or Discrete Flow (DF) models outperforms the state-of-the-art baselines over all datasets. In addition, we demonstrate the effectiveness of KSMRL with property optimization experiments. To indicate the ability of predicting specified potential Drug-Target Interactions (DTIs), a case study for discriminating the interactions between molecule generated by KSMRL and targets is also given.

Dialogue policy learning is the core decision-making module of a task-oriented dialogue system. Its primary objective is to assist users to achieve their goals effectively in as few turns as possible. A practical dialogue-policy agent must be able to expand its knowledge to handle new scenarios efficiently without affecting its performance. Nevertheless, when adapting to new tasks, existing dialogue-policy agents often fail to retain their existing (old) knowledge. To overcome this predicament, we propose a novel continual dialogue-policy model which tackles the issues of “not forgetting the old” and “acquiring the new” from three different aspects: (1) For effective old-task preservation, we introduce the forgetting preventor which uses a behavior cloning technique to force the agent to take actions consistent with the replayed experience to retain the policy trained on historic tasks. (2) For new-task acquisition, we introduce the adaption accelerator which employs an invariant risk minimization mechanism to produce a stable policy predictor to avoid spurious corrections in training data. (3) For reducing the storage cost of the replayed experience, we introduce a replay manager which helps regularly clean up the old data. The effectiveness of the proposed model is evaluated both theoretically and experimentally and demonstrated favorable results.

Deep graph neural networks have recently demonstrated powerful representation learning capabilities in bioinformatics. It is still crucial and challenging to design a representation model fusing the fundamental chemistry and biology knowledge. However, most existing representation models not only ignore domain knowledge but are also built on labeled datasets. To address this issue, this paper proposed a hierarchical structure-aware pre-training model that used contrastive learning to improve molecular representations with unlabeled datasets. We conducted comprehensive experiments on 13 molecular benchmark datasets from different application domains. The results demonstrate that our hierarchical structure-aware pre-trained model achieves superior performance against state-of-the-art baselines.

Predicting drug combinations according to patients' electronic health records is an essential task in intelligent healthcare systems, which can assist clinicians in ordering safe and effective prescriptions. However, existing work either missed/underutilized the important information lying in the drug molecule structure in drug encoding or has insufficient control over Drug-Drug Interactions (DDIs) rates within the predictions. To address these limitations, we propose CSEDrug, which enhances the drug encoding and DDIs controlling by leveraging multi-faceted drug knowledge, including molecule structures of drugs, Synergistic DDIs (SDDIs), and Antagonistic DDIs (ADDIs). We integrate these types of knowledge into CSEDrug by a graph-based drug encoder and multiple loss functions, including a novel triplet learning loss and a comprehensive DDI controllable loss. We evaluate the performance of CSEDrug in terms of accuracy, effectiveness, and safety on the public MIMIC-III dataset. The experimental results demonstrate that CSEDrug outperforms several state-of-the-art methods and achieves a 2.93% and a 2.77% increase in the Jaccard similarity scores and F1 scores, meanwhile, a 0.68% reduction of the ADDI rate (safer drug combinations), and 0.69% improvement of the SDDI rate (more effective drug combinations).

Event detection is challenging in real-world application since new events continually occur and old events still exist which may result in repeated labeling for old events. Therefore, incremental event detection is essential where a model continuously learns new events and meanwhile prevents performance from degrading on old events. Although existing incremental event detection models achieve impressive performance, they face the data imbalance problem between old classes and new classes, and have the knowledge transfer problem which cannot adequately utilize the knowledge provided by the previous model and data. To this end, we propose a Balance-Normalization-Uncertainty (BNU) model to address above problems. Specifically, in order to mitigate the adverse effects of data imbalance, we incorporate a balanced fine-tuning stage and a cosine normalization module. Meanwhile, we consider aleatoric uncertainty to preserve previous knowledge while training for new events. Experimental results show that our proposed method resolves the above challenges effectively and achieves consistent and significant performance on ACE and TAC KBP datasets.

Hengyi Zheng, Rui Wen, Xi Chen, Yifan Yang, Yunyan Zhang, Ziheng Zhang, Ningyu Zhang, Bin Qin, Xu Ming, Yefeng Zheng. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.