Multi-objective molecular optimization is a fundamental yet inherently challenging task in drug discovery, as it requires simultaneously optimizing multiple, often conflicting, molecular properties. Although recent deep learning methods have shown promise, they often lack objective-specific specialization and dynamic coordination, making them ineffective in handling competing objectives and difficult to scale in complex, high-dimensional molecular design tasks. Inspired by the division of labor among domain experts in medicinal chemistry, we propose MAMO, a multi-agent framework for molecular design that simulates expert collaboration. Each agent specializes in optimizing a single objective, and their interactions are orchestrated by a central scheduling module that dynamically reallocates tasks based on evaluation feedback. This coordination mechanism enables interpretable and goal-conditioned optimization while adaptively balancing conflicting objectives. Extensive experiments on benchmark datasets demonstrate that MAMO consistently achieves superior performance in both objective quality and Pareto diversity, particularly in scenarios with strong inter-objective conflict. Our results highlight the potential of multi-agent coordination strategies for scalable and conflict-aware molecular design.

LLM-based Multi-agent systems (MAS) have shown strong capabilities across a wide range of domains.Their success largely hinges on the collaboration topology design, which has emerged as a central research focus in the automated MAS design.However, existing approaches are fundamentally constrained by their reliance on homogeneous LLMs, which significantly limits overall system intelligence.In response to this limitation, we for the first time propose the concept of Automated Design of Heterogeneous-LLMs-based MAS (ADHM).ADHM sheds light on a promising avenue for advancing collective intelligence, which focuses on the automated design of costeffective MAS composed of diverse LLMs and roles to suit various queries.Toward this challenging goal, we propose Hetero-Designer, a novel pipeline that efficiently encodes intricate dependencies among queries, LLMs and roles through a novel Binary-Star Transformer and constructs Hetero-MAS in an autoregressive graph generation process.Extensive experiments demonstrate that Hetero-Designer is: (i) high-performing on various benchmarks, (ii) economical in reducing overhead, (iii) extensible to unseen LLMs and roles.

Chemical reaction and retrosynthesis prediction are fundamental tasks in drug discovery. Recently, large language models (LLMs) have shown potential in many domains. However, directly applying LLMs to these tasks faces two major challenges: (i) lacking a large-scale chemical synthesis-related instruction dataset; (ii) ignoring the close correlation between reaction and retrosynthesis prediction for the existing fine-tuning strategies. To address these challenges, we propose ChemDual, a novel LLM framework for accurate chemical synthesis. Specifically, considering the high cost of data acquisition for reaction and retrosynthesis, ChemDual regards the reaction-and-retrosynthesis of molecules as a related recombination-and-fragmentation process and constructs a large-scale of 4.4 million instruction dataset. Furthermore, ChemDual introduces an enhanced LLaMA, equipped with a multi-scale tokenizer and dual-task learning strategy, to jointly optimize the process of recombination and fragmentation as well as the tasks between reaction and retrosynthesis prediction. Extensive experiments on Mol-Instruction and USPTO-50K datasets demonstrate that ChemDual achieves state-of-the-art performance in both predictions of reaction and retrosynthesis, outperforming the existing conventional single-task approaches and the general open-source LLMs. Through molecular docking analysis, ChemDual generates compounds with diverse and strong protein binding affinity, further highlighting its strong potential in drug design.

Jiachun Li, Pengfei Cao, Chenhao Wang, Zhuoran Jin, Yubo Chen, Daojian Zeng, Kang Liu, Jun Zhao. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Chenhao Wang, Pengfei Cao, Zhuoran Jin, Yubo Chen, Daojian Zeng, Kang Liu, Jun Zhao. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

In this paper, we focus on the challenging yet practical problem of Continual Few-shot Relation Extraction (CFRE), which involves extracting relations in the continuous and iterative arrival of new data with only a few labeled examples. The main challenges in CFRE are overfitting due to few-shot learning and catastrophic forgetting caused by continual learning. To address these problems, we propose a novel framework called RK2DA, which seamlessly integrates prototype-based data augmentation and relational knowledge distillation. Specifically, RK2DA generates pseudo data by introducing Gaussian noise to the prototype embeddings and utilizes a novel two-phase multi-teacher relational knowledge distillation method to transfer various knowledge from different embedding spaces. Experimental results on the FewRel and TACRED datasets demonstrate that our method outperforms the state-of-the-art baselines.

Large Language Models (LLMs) have shown impressive capabilities but still suffer from the issue of hallucinations.A significant type of this issue is the false premise hallucination, which we define as the phenomenon when LLMs generate hallucinated text when confronted with false premise questions.In this paper, we perform a comprehensive analysis of the false premise hallucination and elucidate its internal working mechanism: a small subset of attention heads (which we designate as false premise heads) disturb the knowledge extraction process, leading to the occurrence of false premise hallucination.Based on our analysis, we propose FAITH (False premise Attention head constraIining for miTigating Hallucinations), a novel and effective method to mitigate false premise hallucinations.It constrains the false premise attention heads during the model inference process.Impressively, extensive experiments demonstrate that constraining only approximately 1% of the attention heads in the model yields a notable increase of nearly 20% of model performance.

To enhance the embedding capacity, the existing linguistic steganography methods predominantly focus on the word or phrase level, with limited emphasis on the sentence level. Nevertheless, these approaches exhibit a deficiency in achieving an optimal balance between embedding capacity and semantic coherence. Moreover, compromised semantic coherence can potentially increase security risks. In this paper, we propose a novel sentence-levelSteganography framework toHideInformation inSyntaxSpace (HISS-Stega) that enables larger embedding capacity while preserving better semantic coherence. Specifically, HISS-Stega builds a syntax-controlled paraphrase generation model to automatically modify the expression forms of the covertext, thereby augmenting the diversity of transformations. This enhancement contributes to the overall improvement in embedding capacity. Subsequently, a syntactic bins coding strategy is employed for successfully embedding secret information in the generated syntax space. Furthermore, HISS-Stega incorporates a semantic distortion function aimed at identifying the optimal syntactic structure for concealing secret information, thereby ensuring enhanced semantic coherence and mitigating potential security risks. The experimental results demonstrate that, in comparison to existing methods, HISS-Stega not only enhances the embedding capacity of each sentence but also maintains a high level of semantic coherence and anti-steganalysis capability.

Document-level Relation Extraction (DRE) aims to recognize the relations between two entities. The entity may correspond to multiple mentions that span beyond the sentence boundary. Few previous studies have investigated the mention integration, which may be problematic because coreferential mentions do not equally contribute to a specific relation. Moreover, prior efforts mainly focus on reasoning at entity-level rather than capturing the global interactions between the entity pairs. In this paper, we propose two novel techniques,ContextGuidedMentionIntegration andInter-pairReasoning (CGM2IR), to improve DRE. Instead of simply applying average pooling, the contexts are utilized to guide the integration of coreferential mentions in a weighted sum manner. Additionally, inter-pair reasoning executes an iterative algorithm on the entity pair graph, so as to model the interdependency of relations. We evaluate our CGM2IR model on three widely used benchmark datasets, namely DocRED, CDR, and GDA. Experimental results show that our model outperforms previous strong baselines and achieves remarkable effectiveness.

Medical Dialogue Information Extraction (MDIE) is a promising task for modern medical care systems, which greatly facilitates the development of many real-world applications such as electronic medical record generation, automatic disease diagnosis, etc. Recent methods have firstly achieved considerable performance in Chinese MDIE but still suffer from some inherent limitations, such as poor exploitation of the inter-dependencies in multiple utterances, weak discrimination of the hard samples. In this paper, we propose a contrastive multi-utterance inference (CMUI) method to address these issues. Specifically, we first use a type-aware encoder to provide an efficient encode mechanism toward different categories. Subsequently, we introduce a selective attention mechanism to explicitly capture the dependencies among utterances, which thus constructs a multi-utterance inference. Finally, a supervised contrastive learning approach is integrated into our framework to improve the recognition ability for the hard samples. Extensive experiments show that our model achieves state-of-the-art performance on a public benchmark Chinese-based dataset and delivers significant performance gain on MDIE as compared with baselines. Specifically, we outperform the state-of-the-art results in F1-score by 2.27%, 0.55% in Recall and 3.61% in Precision (The codes that support the findings of this study are openly available in CMUI at https://github.com/jc4357/CMUI.).

Joint extraction of entities and relations has received significant attention due to its potential of providing higher performance for both tasks. Among existing methods, CopyRE is effective and novel, which uses a sequence-to-sequence framework and copy mechanism to directly generate the relation triplets. However, it suffers from two fatal problems. The model is extremely weak at differing the head and tail entity, resulting in inaccurate entity extraction. It also cannot predict multi-token entities (e.g. Steven Jobs). To address these problems, we give a detailed analysis of the reasons behind the inaccurate entity extraction problem, and then propose a simple but extremely effective model structure to solve this problem. In addition, we propose a multi-task learning framework equipped with copy mechanism, called CopyMTL, to allow the model to predict multi-token entities. Experiments reveal the problems of CopyRE and show that our model achieves significant improvement over the current state-of-the-art method by 9% in NYT and 16% in WebNLG (F1 score). Our code is available at https://github.com/WindChimeRan/CopyMTL

Xiangrong Zeng, Shizhu He, Daojian Zeng, Kang Liu, Shengping Liu, Jun Zhao. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

The relational facts in sentences are often complicated.Different relational triplets may have overlaps in a sentence.We divided the sentences into three types according to triplet overlap degree, including Normal, EntityPairOverlap and SingleEn-tiyOverlap. Existing methods mainly focus on Normal class and fail to extract relational triplets precisely.In this paper, we propose an end-to-end model based on sequence-to-sequence learning with copy mechanism, which can jointly extract relational facts from sentences of any of these classes.We adopt two different strategies in decoding process: employing only one united decoder or applying multiple separated decoders.We test our models in two public datasets and our model outperform the baseline method significantly.

Yubo Chen, Liheng Xu, Kang Liu, Daojian Zeng, Jun Zhao. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2015.

Two problems arise when using distant supervision for relation extraction.First, in this method, an already existing knowledge base is heuristically aligned to texts, and the alignment results are treated as labeled data.However, the heuristic alignment can fail, resulting in wrong label problem.In addition, in previous approaches, statistical models have typically been applied to ad hoc features.The noise that originates from the feature extraction process can cause poor performance.In this paper, we propose a novel model dubbed the Piecewise Convolutional Neural Networks (PCNNs) with multi-instance learning to address these two problems.To solve the first problem, distant supervised relation extraction is treated as a multi-instance problem in which the uncertainty of instance labels is taken into account.To address the latter problem, we avoid feature engineering and instead adopt convolutional architecture with piecewise max pooling to automatically learn relevant features.Experiments show that our method is effective and outperforms several competitive baseline methods.

Community question answering (cQA) has become an important service due to the popularity of cQA archives on the web. This paper is concerned with the problem of question search. Question search in cQA aims to find the historical questions that are semantically equivalent or similar to the queried questions. In this paper, we propose a faster and better retrieval model for question search by leveraging user chosen category. After introducing the question category, we can filter certain amount of irrelevant historical questions under a wide range of leaf categories. Experimental results conducted on real cQA data set demonstrate that the proposed techniques are more effective and efficient than a variety of baseline methods.