EDBT 2026 Demo / reviewers in the wild / expert
Sendong Zhao
dblp:119/6283
· DBLP profile ↗
38ranked-venue papers
10as first author
27since 2021 · last 2026
0000-0002-4676-1812ORCID · corroborated
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 25 · 6 first-author · 18 since 2021Graphics, computer vision, multimedia, augmented reality and games · 10 · 1 first-author · 7 since 2021Databases, data management, data science and information retrieval · 6 · 5 first-author · 2 since 2021Applied, interdisciplinary, general and emerging computing · 6 · 2 first-author · 5 since 2021Systems, architecture and hardware · 1Security and privacy · 1 · 1 first-author
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2026 | Toward Secure Tuning: Mitigating Security Risks from Instruction Fine-TuningabstractYanrui Du, Fenglei Fan, Sendong Zhao, Jiawei Cao, Ming Ma, Danyang Zhao, Shuren Qi, Ting Liu, Bing Qin. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026. Yanrui Du, Fenglei Fan, Sendong Zhao, Danyang Zhao, Ting Liu 0001, Bing Qin 0001 |
ACL (1) | 3 |
| 2026 | Collaborative Chain-of-Agents for Parametric-Retrieved Knowledge SynergyabstractRetrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs), especially for knowledge-intensive tasks.Despite its advantages, current RAG methods often struggle to fully exploit knowledge during generation.In particular, the synergy between the model's internal parametric knowledge and external retrieved knowledge remains limited.Retrieved contents may sometimes mislead generation, while certain generated content can guide the model toward more accurate outputs.In this work, we propose Collaborative Chainof-Agents, a framework designed to enhance explicitly synergy over both parametric and retrieved knowledge.Specifically, we first introduce CoCoA-zero, a multi-agent RAG framework that first performs conditional knowledge induction and then reasons answers.Building on this, we develop CoCoA, a long-chain training strategy that synthesizes extended multiagent reasoning trajectories from CoCoA-zero to fine-tune the LLM.This strategy enhances the model's capability to explicitly integrate and jointly leverage parametric and retrieved knowledge.Experimental results demonstrate the superiority of CoCoA in open-domain QA and multi-hop QA.Code is public 1 . Sendong Zhao, Haochun Wang, Lizhe Zhang, Bing Qin 0001 |
ACL (1) | 2 |
| 2026 | When Correct Beliefs Collapse: Epistemic Resilience of LLMs under Clinical PressureabstractBoyu Xiao, Xiuqi Tian, Xuwen Song, Haochun Wang, Guanchun Song, Sendong Zhao, Bing Qin. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026. Boyu Xiao, Xiuqi Tian, Xuwen Song, Haochun Wang, Guanchun Song, Sendong Zhao, Bing Qin 0001 |
ACL (1) | 6 |
| 2025 | Investigating the Security Threat Arising from "Yes-No" Implicit Bias in Large Language ModelsabstractLarge Language Models (LLMs) have gained significant attention for their exceptional performance across various domains. Despite their advancements, concerns persist regarding their implicit bias, which often leads to negative social impacts. Therefore, it is essential to identify the implicit bias in LLMs and investigate the potential threat posed by it. Our study focused on a specific type of implicit bias, termed the ''Yes-No'' implicit bias, which refers to LLMs' inherent tendency to favor ''Yes'' or ''No'' responses to a single instruction. By comparing the probability of LLMs generating a series of ''Yes'' versus ''No'' responses, we observed different inherent response tendencies exhibited by LLMs when faced with different instructions. To further investigate the impact of such bias, we developed an attack method called Implicit Bias In-Context Manipulation, attempting to manipulate LLMs' behavior. Specifically, we explored whether the ''Yes'' implicit bias could manipulate ''No'' responses into ''Yes'' in LLMs' responses to malicious instructions, leading to harmful outputs. Our findings revealed that the ''Yes'' implicit bias brings a significant security threat, comparable to that of carefully designed attack methods. Moreover, we offered a comprehensive analysis from multiple perspectives to deepen the understanding of this security threat, emphasizing the need for ongoing improvement in LLMs' security. Yanrui Du, Sendong Zhao, Yuhan Chen 0002, Bing Qin 0001 |
AAAI | 2 |
| 2025 | GainRAG: Preference Alignment in Retrieval-Augmented Generation through Gain Signal SynthesisabstractThe Retrieval-Augmented Generation (RAG) framework introduces a retrieval module to dynamically inject retrieved information into the input context of large language models (LLMs), and has demonstrated significant success in various NLP tasks.However, the current study points out that there is a preference gap between retrievers and LLMs in the RAG framework, which limit the further improvement of system performance.Some highly relevant passages may interfere with LLM reasoning because they contain complex or contradictory information; while some indirectly related or even inaccurate content may help LLM generate more accurate answers by providing suggestive information or logical clues.To solve this, we propose GainRAG, a novel approach that aligns the retriever's and LLM's preferences by defining a new metric, "gain", which measure how well an input passage contributes to correct outputs.Specifically, we propose a method to estimate these gain signals and train a middleware that aligns the preferences of the retriever and the LLM using only limited data.In addition, we introduce a pseudo-passage strategy to mitigate degradation.The experimental results on 6 datasets verify the effectiveness of GainRAG 1 . Sendong Zhao, Haochun Wang, Bing Qin 0001 |
ACL (1) | 2 |
| 2025 | Beyond Frameworks: Unpacking Collaboration Strategies in Multi-Agent SystemsabstractMulti-agent collaboration has emerged as a pivotal paradigm for addressing complex, distributed tasks in large language model (LLM)-driven applications. While prior research has focused on high-level architectural frameworks, the granular mechanisms governing agents—critical to performance and scalability—remain underexplored. This study systematically investigates four dimensions of collaboration strategies: (1) agent governance, (2) participation control, (3) interaction dynamics, and (4) dialogue history management. Through rigorous experimentation under two context-dependent scenarios—Distributed Evidence Integration (DEI) and Structured Evidence Synthesis (SES)—we quantify the impact of these strategies on both task accuracy and computational efficiency. Our findings reveal that centralized governance, instructor-led participation, ordered interaction patterns, and instructor-curated context summarization collectively optimize the trade-off between decision quality and resource utilization with the support of the proposed Token-Accuracy Ratio (TAR). This work establishes a foundation for designing adaptive, scalable multi-agent systems, shifting the focus from structural novelty to strategic interaction mechanics. Haochun Wang, Sendong Zhao, Zewen Qiang, Bing Qin 0001, Ting Liu 0001 |
ACL (1) | 2 |
| 2025 | ReLearn: Unlearning via Learning for Large Language ModelsabstractCurrent unlearning methods for large language models usually rely on reverse optimization to reduce target token probabilities. However, this paradigm disrupts the subsequent tokens prediction, degrading model performance and linguistic coherence. Moreover, existing evaluation metrics overemphasize contextual forgetting while inadequately assessing response fluency and relevance. To address these challenges, we propose ReLearn, a data augmentation and fine-tuning pipeline for effective unlearning, along with a comprehensive evaluation framework. This framework introduces Knowledge Forgetting Ratio (KFR) and Knowledge Retention Ratio (KRR) to measure knowledge-level preservation, and Linguistic Score (LS) to evaluate generation quality. Our experiments show that ReLearn successfully achieves targeted forgetting while preserving high-quality outputs. Through mechanistic analysis, we further demonstrate how reverse optimization disrupts coherent text generation, while ReLearn preserves this essential capability. Ningyuan Zhao, Sendong Zhao, Shumin Deng, Bryan Hooi, Nay Oo, Huajun Chen, Ningyu Zhang 0001 |
ACL (1) | 4 |
| 2025 | LLMs May Perform MCQA by Selecting the Least Incorrect OptionabstractIn the field of NLP, Large Language Models (LLMs) have markedly enhanced performance across a variety of tasks. However, the comprehensive evaluation of LLMs remains an inevitable challenge for the community. Recently, the adoption of Multiple Choice Question Answering (MCQA) as a benchmark for assessing LLMs has gained considerable traction. However, concerns regarding the robustness of this evaluative method persist. Building upon previous discussions on the issue of variability, we reveal an additional dimension of concern: LLMs may perform MCQA by selecting the least incorrect option rather than distinctly correct. This observation suggests that LLMs might regard multiple options as correct, which could undermine the reliability of MCQA as a metric for evaluating LLMs. To address this challenge, we introduce an enhanced dataset augmentation method for MCQA, termed MCQA+, to provide a more accurate reflection of the performance, thereby highlighting the necessity for more sophisticated evaluation mechanisms in the assessment of LLM capabilities. Haochun Wang, Sendong Zhao, Zewen Qiang, Nuwa Xi, Bing Qin 0001, Ting Liu 0001 |
COLING | 2 |
| 2025 | MolFusion: Multimodal Fusion Learning for Molecular Representations via Multi-granularity ViewsabstractArtificial intelligence advances drug design by predicting drug properties through the encoding of drug molecules. Since different molecular representations contain complementary information, a large amount of research is currently dedicated to the fusion of these representations. However, current multimodal molecular studies rely more on single-granularity methods, resulting in the loss of atomic-level information between representations. Inspired by the success of multi-granularity approaches, we introduce MolFusion, an innovative method for multi-granularity molecular representation fusion. MolFusion comprises two core components: MolSim for molecular-level fusion and AtomAlign for atomic-level fusion. Comprehensive experiments show that our method outperforms all powerful baselines in terms of average performance across both classification and regression tasks, with a particular enhancement in regression tasks. Our code and dataset will be available on https://github.com/Mengqi97/MolFusion. Muzhen Cai, Sendong Zhao, Haochun Wang, Haoqiang Guo, Yanrui Du, Zewen Qiang, Bing Qin 0001, Ting Liu 0001 |
IJCNN | 2 |
| 2025 | Knowledge-tuning Large Language Models with Structured Medical Knowledge Bases for Trustworthy Response Generation in ChineseabstractLarge Language Models (LLMs) have demonstrated remarkable success in diverse natural language processing (NLP) tasks in general domains. However, LLMs sometimes generate responses with the hallucination about medical facts due to limited domain knowledge. Such shortcomings pose potential risks in the utilization of LLMs within medical contexts. To address this challenge, we propose knowledge-tuning, which leverages structured medical knowledge bases for the LLMs to grasp domain knowledge efficiently and facilitate trustworthy response generation. We also release cMedKnowQA, a Chinese medical knowledge question-answering dataset constructed from medical knowledge bases to assess the medical knowledge proficiency of LLMs. Experimental results show that the LLMs which are knowledge-tuned with cMedKnowQA can exhibit higher levels of accuracy in response generation compared with vanilla instruction-tuning and offer a new trustworthy way for the domain adaptation of LLMs. We release our code and data at https://github.com/SCIR-HI/Huatuo-Llama-Med-Chinese . Haochun Wang, Sendong Zhao, Zewen Qiang, Zijian Li 0020, Chi Liu 0003, Nuwa Xi, Yanrui Du, Bing Qin 0001, Ting Liu 0001 |
ACM Trans. Knowl. Discov. Data | 2 |
| 2024 | From Artificially Real to Real: Leveraging Pseudo Data from Large Language Models for Low-Resource Molecule DiscoveryabstractMolecule discovery serves as a cornerstone in numerous scientific domains, fueling the development of new materials and innovative drug designs. Recent developments of in-silico molecule discovery have highlighted the promising results of cross-modal techniques, which bridge molecular structures with their descriptive annotations. However, these cross-modal methods frequently encounter the issue of data scarcity, hampering their performance and application. In this paper, we address the low-resource challenge by utilizing artificially-real data generated by Large Language Models (LLMs). We first introduce a retrieval-based prompting strategy to construct high-quality pseudo data, then explore the optimal method to effectively leverage this pseudo data. Experiments show that using pseudo data for domain adaptation outperforms all existing methods, while also requiring a smaller model scale, reduced data size and lower training cost, highlighting its efficiency. Furthermore, our method shows a sustained improvement as the volume of pseudo data increases, revealing the great potential of pseudo data in advancing low-resource cross-modal molecule discovery. Yuhan Chen 0002, Nuwa Xi, Yanrui Du, Haochun Wang, Sendong Zhao, Bing Qin 0001 |
AAAI | 6 |
| 2024 | MolTailor: Tailoring Chemical Molecular Representation to Specific Tasks via Text PromptsabstractDeep learning is now widely used in drug discovery, providing significant acceleration and cost reduction. As the most fundamental building block, molecular representation is essential for predicting molecular properties to enable various downstream applications. Most existing methods attempt to incorporate more information to learn better representations. However, not all features are equally important for a specific task. Ignoring this would potentially compromise the training efficiency and predictive accuracy. To address this issue, we propose a novel approach, which treats language models as an agent and molecular pretraining models as a knowledge base. The agent accentuates task-relevant features in the molecular representation by understanding the natural language description of the task, just as a tailor customizes clothes for clients. Thus, we call this approach MolTailor. Evaluations demonstrate MolTailor's superior performance over baselines, validating the efficacy of enhancing relevance for molecular representation learning. This illustrates the potential of language model guided optimization to better exploit and unleash the capabilities of existing powerful molecular representation methods. Our code and appendix are available at https://github.com/SCIR-HI/MolTailor. Haoqiang Guo, Sendong Zhao, Haochun Wang, Yanrui Du, Bing Qin 0001 |
AAAI | 2 |
| 2024 | Manifold-Based Verbalizer Space Re-embedding for Tuning-Free Prompt-Based ClassificationabstractPrompt-based classification adapts tasks to a cloze question format utilizing the [MASK] token and the filled tokens are then mapped to labels through pre-defined verbalizers. Recent studies have explored the use of verbalizer embeddings to reduce labor in this process. However, all existing studies require a tuning process for either the pre-trained models or additional trainable embeddings. Meanwhile, the distance between high-dimensional verbalizer embeddings should not be measured by Euclidean distance due to the potential for non-linear manifolds in the representation space. In this study, we propose a tuning-free manifold-based space re-embedding method called Locally Linear Embedding with Intra-class Neighborhood Constraint (LLE-INC) for verbalizer embeddings, which preserves local properties within the same class as guidance for classification. Experimental results indicate that even without tuning any parameters, our LLE-INC is on par with automated verbalizers with parameter tuning. And with the parameter updating, our approach further enhances prompt-based tuning by up to 3.2%. Furthermore, experiments with the LLaMA-7B&13B indicate that LLE-INC is an efficient tuning-free classification approach for the hyper-scale language models. Haochun Wang, Sendong Zhao, Chi Liu 0003, Nuwa Xi, Muzhen Cai, Bing Qin 0001, Ting Liu 0001 |
AAAI | 2 |
| 2024 | CMCOQA: A Chinese Medical Complex Open-Question Answering BenchmarkabstractWith the development of Large Language Models (LLMs), many Chinese medical benchmarks have emerged. These benchmarks have primarily used multiple-choice questions and open-ended questions as test items. However, our experimental results indicate that using multiple-choice questions to test the capabilities of LLMs is not very reasonable. Additionally, relatively simple open-ended questions do not effectively assess LLMs’ actual grasp of medical knowledge. Therefore, we propose the Chinese Medical Complex Open-Question Answering Benchmark (CMCOQA), designed to more accurately and efficiently evaluate the true medical proficiency of LLMs by constructing complex open-ended questions within medical scenarios. Our proposed benchmark involves three evaluation dimensions: Completeness, Depth, and Professionalism. Starting with 100 manually generated complex questions as seeds, we expand the set to 1,200 using the Self-Instruct method with GPT-4o. We then have GPT-4o self-check the questions, followed by a manual screening process to ensure a broad coverage and a certain level of depth. We have both humans and GPT-4o score from these three dimensions, while also employing automated metrics. We also calculate correlations between these metrics and human scores to validate the results. Through this work, CMCOQA can further promote the development of Chinese medical LLMs in terms of medical professionalism. Zijian Li 0020, Sendong Zhao, Haochun Wang, Bing Qin 0001, Ting Liu 0001 |
BIBM | 2 |
| 2024 | Probing the Dual Logic Ability of Privatized Medical-Domain LLMsabstractLarge Language Models (LLMs) are gaining widespread attention for their potential across various fields, particularly within the medical domain. Recent efforts have aimed to privatize general-domain LLMs into specialized medical-domain LLMs by feeding high-quality, medical-domain training data. However, an overlooked aspect in these privatization efforts is the Dual Logic Ability of LLMs. This ability enables LLMs to comprehend pairs of logically opposed questions, ensuring stance consistency in their responses. In our study, we investigate two primary questions: Q1. How does privatization affect the dual logic ability of LLMs? Q2. How can we maintain the robustness of LLMs’ dual logic ability after privatization? To explore these questions, we first constructed a medical-domain dual logic ability evaluation dataset comprising logically opposed question pairs, created manually by NLP experts. By examining the stance consistency in responses to logically opposed question pairs, our analysis demonstrates a significant decline in the dual logic ability of LLMs after privatization. Furthermore, we construct privatization data to investigate the effects of the pre-training and instruction fine-tuning stages on the dual logic ability of LLMs. Interestingly, our findings reveal that the instruction fine-tuning stage often inadvertently compromises the LLMs’ dual logic ability although it is not the trainers’ intention. To counteract this, we incorporated general-domain dual logic data derived from basic science during the instruction fine-tuning stage, which are automatically constructed by our designed pipeline. Experiment results show that privatized LLMs can generalize dual logic ability from general-domain dual logic data, leading to their enhanced performance in the medical domain. Our study underscores the importance of prioritizing LLMs’ dual logic ability during the privatization process and establishes a benchmark for future research. Our data and code are available at GitHub1. Yanrui Du, Sendong Zhao, Muzhen Cai, Danyang Zhao, Bing Qin 0001 |
BIBM | 2 |
| 2024 | MoGU: A Framework for Enhancing Safety of LLMs While Preserving Their UsabilityabstractLarge Language Models (LLMs) are increasingly deployed in various applications. As their usage grows, concerns regarding their safety are rising, especially in maintaining harmless responses when faced with malicious instructions. Many defense strategies have been developed to enhance the safety of LLMs. However, our research finds that existing defense strategies lead LLMs to predominantly adopt a rejection-oriented stance, thereby diminishing the usability of their responses to benign instructions. To solve this problem, we introduce the MoGU framework, designed to enhance LLMs' safety while preserving their usability. Our MoGU framework transforms the base LLM into two variants: the usable LLM and the safe LLM, and further employs dynamic routing to balance their contribution. When encountering malicious instructions, the router will assign a higher weight to the safe LLM to ensure that responses are harmless. Conversely, for benign instructions, the router prioritizes the usable LLM, facilitating usable and helpful responses. On various open-sourced LLMs, we compare multiple defense strategies to verify the superiority of our MoGU framework. Besides, our analysis provides key insights into the effectiveness of MoGU and verifies that our designed routing mechanism can effectively balance the contribution of each variant by assigning weights. Our work released the safer Llama2, Vicuna, Falcon, Dolphin, and Baichuan2. Yanrui Du, Sendong Zhao, Danyang Zhao, Yuhan Chen 0002, Liangyu Huo, Qing Yang 0033, Dongliang Xu, Bing Qin 0001 |
NeurIPS | 2 |
| 2023 | UniCoRN: Unified Cognitive Signal ReconstructioN bridging cognitive signals and human languageabstractDecoding text stimuli from cognitive signals (e.g.fMRI) enhances our understanding of the human language system, paving the way for building versatile Brain-Computer Interface.However, existing studies largely focus on decoding individual word-level fMRI volumes from a restricted vocabulary, which is far too idealized for real-world application.In this paper, we propose fMRI2text, the first openvocabulary task aiming to bridge fMRI time series and human language.Furthermore, to explore the potential of this new task, we present a baseline solution, UniCoRN: the Unified Cognitive Signal ReconstructioN for Brain Decoding.By reconstructing both individual time points and time series, UniCoRN establishes a robust encoder for cognitive signals (fMRI & EEG).Leveraging a pre-trained language model as decoder, UniCoRN proves its efficacy in decoding coherent text from fMRI series across various split settings.Our model achieves a 34.77%BLEU score on fMRI2text, and a 37.04% BLEU when generalized to EEGto-text decoding, thereby surpassing the former baseline.Experimental results indicate the feasibility of decoding consecutive fMRI volumes, and the effectiveness of decoding different cognitive signals using a unified structure. Nuwa Xi, Sendong Zhao, Haochun Wang, Chi Liu 0003, Bing Qin 0001, Ting Liu 0001 |
ACL (1) | 2 |
| 2023 | Less Learn Shortcut: Analyzing and Mitigating Learning of Spurious Feature-Label CorrelationabstractRecent research has revealed that deep neural networks often take dataset biases as a shortcut to make decisions rather than understand tasks, leading to failures in real-world applications. In this study, we focus on the spurious correlation between word features and labels that models learn from the biased data distribution of training data. In particular, we define the word highly co-occurring with a specific label as biased word, and the example containing biased word as biased example. Our analysis shows that biased examples are easier for models to learn, while at the time of prediction, biased words make a significantly higher contribution to the models' predictions, and models tend to assign predicted labels over-relying on the spurious correlation between words and labels. To mitigate models' over-reliance on the shortcut (i.e. spurious correlation), we propose a training strategy Less-Learn-Shortcut (LLS): our strategy quantifies the biased degree of the biased examples and down-weights them accordingly. Experimental results on Question Matching, Natural Language Inference and Sentiment Analysis tasks show that LLS is a task-agnostic strategy and can improve the model performance on adversarial data while maintaining good performance on in-domain data. Yanrui Du, Jing Yan 0004, Jing Liu 0022, Sendong Zhao, Qiaoqiao She, Hua Wu 0003, Haifeng Wang 0001, Bing Qin 0001 |
IJCAI | 5 |
| 2023 | Global Prompt Cell: A Portable Control Module for Effective Prompt Tuning
Chi Liu 0003, Haochun Wang, Nuwa Xi, Sendong Zhao, Bing Qin 0001 |
NLPCC (1) | 4 |
| 2022 | Prompt Combines Paraphrase: Teaching Pre-trained Models to Understand Rare Biomedical WordsabstractPrompt-based fine-tuning for pre-trained models has proven effective for many natural language processing tasks under few-shot settings in general domain. However, tuning with prompt in biomedical domain has not been investigated thoroughly. Biomedical words are often rare in general domain, but quite ubiquitous in biomedical contexts, which dramatically deteriorates the performance of pre-trained models on downstream biomedical applications even after fine-tuning, especially in low-resource scenarios. We propose a simple yet effective approach to helping models learn rare biomedical words during tuning with prompt. Experimental results show that our method can achieve up to 6% improvement in biomedical natural language inference task without any extra parameters or training steps using few-shot vanilla prompt settings. Haochun Wang, Chi Liu 0003, Nuwa Xi, Sendong Zhao, Meizhi Ju, Yefeng Zheng 0001, Bing Qin 0001, Ting Liu 0001 |
COLING | 4 |
| 2022 | Biomedical evidence engineering for data-driven discoveryabstractMOTIVATION: With the rapid development of precision medicine, a large amount of health data (such as electronic health records, gene sequencing, medical images, etc.) has been produced. It encourages more and more interest in data-driven insight discovery from these data. A reasonable way to verify the derived insights is by checking evidence from biomedical literature. However, manual verification is inefficient and not scalable. Therefore, an intelligent technique is necessary to solve this problem. RESULTS: This article introduces a framework for biomedical evidence engineering, addressing this problem more effectively. The framework consists of a biomedical literature retrieval module and an evidence extraction module. The retrieval module ensembles several methods and achieves state-of-the-art performance in biomedical literature retrieval. A BERT-based evidence extraction model is proposed to extract evidence from literature in response to queries. Moreover, we create a dataset with 1 million examples of biomedical evidence, 10 000 of which are manually annotated. AVAILABILITY AND IMPLEMENTATION: Datasets are available at https://github.com/SendongZhao. Sendong Zhao, Aobo Wang, Bing Qin 0001, Fei Wang 0001 |
Bioinform. | 1 |
| 2022 | Combining Self-supervised Learning and Active Learning for Disfluency DetectionabstractSpoken language is fundamentally different from the written language in that it contains frequent disfluencies or parts of an utterance that are corrected by the speaker. Disfluency detection (removing these disfluencies) is desirable to clean the input for use in downstream NLP tasks. Most existing approaches to disfluency detection heavily rely on human-annotated data, which is scarce and expensive to obtain in practice. To tackle the training data bottleneck, in this work, we investigate methods for combining self-supervised learning and active learning for disfluency detection. First, we construct large-scale pseudo training data by randomly adding or deleting words from unlabeled data and propose two self-supervised pre-training tasks: (i) a tagging task to detect the added noisy words and (ii) sentence classification to distinguish original sentences from grammatically incorrect sentences. We then combine these two tasks to jointly pre-train a neural network. The pre-trained neural network is then fine-tuned using human-annotated disfluency detection training data. The self-supervised learning method can capture task-special knowledge for disfluency detection and achieve better performance when fine-tuning on a small annotated dataset compared to other supervised methods. However, limited in that the pseudo training data are generated based on simple heuristics and cannot fully cover all the disfluency patterns, there is still a performance gap compared to the supervised models trained on the full training dataset. We further explore how to bridge the performance gap by integrating active learning during the fine-tuning process. Active learning strives to reduce annotation costs by choosing the most critical examples to label and can address the weakness of self-supervised learning with a small annotated dataset. We show that by combining self-supervised learning with active learning, our model is able to match state-of-the-art performance with just about 10% of the original training data on both the commonly used English Switchboard test set and a set of in-house annotated Chinese data. Shaolei Wang, Wanxiang Che, Sendong Zhao, Ting Liu 0001 |
ACM Trans. Asian Low Resour. Lang. Inf. Process. | 4 |
| 2022 | Understanding Patient Query With Weak Supervision From Doctor ResponseabstractCurrently, the need for high-quality dialogue systems that assist users to conduct self-diagnosis is rapidly increasing. Slot filling for automatic diagnosis, which converts medical queries into structured representations, plays an important role in diagnostic dialogue systems. However, the lack of high-quality datasets limits the performance of slot filling. While medical communities like AskAPatient usually have multiple rounds of diagnostic dialogue containing colloquial input and professional responses from doctors. Therefore, the data of diagnostic dialogue in medical communities can be utilized to solve the main challenges in slot filling. This paper proposes a two-step training framework to make full use of these unlabeled dialogue data in medical communities. To promote further researches, we provide a Chinese dataset with 2,652 annotated samples and a large amount of unlabeled samples. Experimental results on the dataset demonstrate the effectiveness of the proposed method with an increase of 6.32% in Micro F1 and 8.20% in Macro F1 on average over strong baselines. Sendong Zhao, Yuxuan Wang 0001, Xi Chen 0003, Yefeng Zheng 0001, Wanxiang Che |
IEEE J. Biomed. Health Informatics | 2 |
| 2022 | Structural and Textual Information Fusion for Symptom and Disease Representation LearningabstractOnline medical consult and offline medical check-in have generated a large amount of health-related data in medical forums and patient records. However, exploiting the user-generated content for orienting patients online and assisting medical checkup offline is nontrivial due to the sparsity of symptom-disease associations. The serious sparsity is caused by the informal/chatty expressions of symptoms in the data. Sendong Zhao, Meng Jiang 0001, Bing Qin 0001, Ting Liu 0001, ChengXiang Zhai, Fei Wang 0001 |
IEEE Trans. Knowl. Data Eng. | 1 |
| 2021 | A Co-Interactive Transformer for Joint Slot Filling and Intent DetectionabstractIntent detection and slot filling are two main tasks for building a spoken language understanding (SLU) system. The two tasks are closely related and the information of one task can benefit the other. Previous studies either implicitly model the two tasks with multi-task framework or only explicitly consider the single information flow from intent to slot. None of the prior approaches model the bidirectional connection between the two tasks simultaneously in a unified framework. In this paper, we propose a Co-Interactive Transformer which considers the cross-impact between the two tasks. Instead of adopting the self-attention mechanism in vanilla Transformer, we propose a co-interactive module to consider the cross-impact by building a bidirectional connection between the two related tasks, where slot and intent can be able to attend on the corresponding mutual information. The experimental results on two public datasets show that our model achieves the state-of-the-art performance. Libo Qin 0001, Tailu Liu, Wanxiang Che, Bingbing Kang, Sendong Zhao, Ting Liu 0001 |
ICASSP | 5 |
| 2021 | Injecting Word Information with Multi-Level Word Adapter for Chinese Spoken Language UnderstandingabstractIn this paper, we improve Chinese spoken language understanding (SLU) by injecting word information. Previous studies on Chinese SLU do not consider the word information, failing to detect word boundaries that are beneficial for intent detection and slot filling. To address this issue, we propose a multi-level word adapter to inject word information for Chinese SLU, which consists of (1) sentence-level word adapter, which directly fuses the sentence representations of the word information and character information to perform intent detection and (2) character-level word adapter, which is applied at each character for selectively controlling weights on word information as well as character information. Experimental results on two Chinese SLU datasets show that our model can capture useful word information and achieve state-of-the-art performance. Dechuan Teng, Libo Qin 0001, Wanxiang Che, Sendong Zhao, Ting Liu 0001 |
ICASSP | 4 |
| 2021 | Recent advances in biomedical literature miningabstractThe recent years have witnessed a rapid increase in the number of scientific articles in biomedical domain. These literature are mostly available and readily accessible in electronic format. The domain knowledge hidden in them is critical for biomedical research and applications, which makes biomedical literature mining (BLM) techniques highly demanding. Numerous efforts have been made on this topic from both biomedical informatics (BMI) and computer science (CS) communities. The BMI community focuses more on the concrete application problems and thus prefer more interpretable and descriptive methods, while the CS community chases more on superior performance and generalization ability, thus more sophisticated and universal models are developed. The goal of this paper is to provide a review of the recent advances in BLM from both communities and inspire new research directions. Sendong Zhao, Chang Su 0002, Zhiyong Lu, Fei Wang 0001 |
Briefings Bioinform. | 1 |
| 2020 | Interactive Attention Networks for Semantic Text MatchingabstractSemantic text matching, which matches target texts to source texts, is a general problem in many areas, such as information retrieval, question answering, and recommendation. The challenges to existing research on this topic include 1) out-of-vocabulary and low-frequency keywords and 2) direct utilization of sparse matching matrix of source and target. The out-of-vocabulary and low-frequency keywords could lead to the mismatch of similar keywords in source and target texts. The sparse matching matrix cannot provide enough clues to match the source with the target. To address these challenges, we propose a novel deep neural semantic text matching model. Our model adopts an interactive attention network to achieve information exchange between the source text and the target text, and dynamically explores the matching matrix and learns new representations of source and target texts. Experimental results on three different text matching datasets demonstrate that our model can significantly outperform competitive baselines. Furthermore, our model demonstrates great advantage in alleviating the sparse matching problem and learning out-of-vocabulary words with the local context, which widely exists in a broad spectrum of NLP applications. Sendong Zhao, Chang Su 0002, Yuantong Li, Fei Wang 0001 |
ICDM | 1 |
| 2020 | Graph convolutional networks for computational drug development and discoveryabstractDespite the fact that deep learning has achieved remarkable success in various domains over the past decade, its application in molecular informatics and drug discovery is still limited. Recent advances in adapting deep architectures to structured data have opened a new paradigm for pharmaceutical research. In this survey, we provide a systematic review on the emerging field of graph convolutional networks and their applications in drug discovery and molecular informatics. Typically we are interested in why and how graph convolution networks can help in drug-related tasks. We elaborate the existing applications through four perspectives: molecular property and activity prediction, interaction prediction, synthesis prediction and de novo drug design. We briefly introduce the theoretical foundations behind graph convolutional networks and illustrate various architectures based on different formulations. Then we summarize the representative applications in drug-related problems. We also discuss the current challenges and future possibilities of applying graph convolutional networks to drug discovery. Mengying Sun, Sendong Zhao, Coryandar Gilvary, Olivier Elemento, Fei Wang 0001 |
Briefings Bioinform. | 2 |
| 2019 | A Neural Multi-Task Learning Framework to Jointly Model Medical Named Entity Recognition and NormalizationabstractState-of-the-art studies have demonstrated the superiority of joint modeling over pipeline implementation for medical named entity recognition and normalization due to the mutual benefits between the two processes. To exploit these benefits in a more sophisticated way, we propose a novel deep neural multi-task learning framework with explicit feedback strategies to jointly model recognition and normalization. On one hand, our method benefits from the general representations of both tasks provided by multi-task learning. On the other hand, our method successfully converts hierarchical tasks into a parallel multi-task setting while maintaining the mutual supports between tasks. Both of these aspects improve the model performance. Experimental results demonstrate that our method performs significantly better than state-of-theart approaches on two publicly available medical literature datasets. Sendong Zhao, Ting Liu 0001, Sicheng Zhao, Fei Wang 0001 |
AAAI | 1 |
| 2019 | CycleEmotionGAN: Emotional Semantic Consistency Preserved CycleGAN for Adapting Image EmotionsabstractDeep neural networks excel at learning from large-scale labeled training data, but cannot well generalize the learned knowledge to new domains or datasets. Domain adaptation studies how to transfer models trained on one labeled source domain to another sparsely labeled or unlabeled target domain. In this paper, we investigate the unsupervised domain adaptation (UDA) problem in image emotion classification. Specifically, we develop a novel cycle-consistent adversarial model, termed CycleEmotionGAN, by enforcing emotional semantic consistency while adapting images cycleconsistently. By alternately optimizing the CycleGAN loss, the emotional semantic consistency loss, and the target classification loss, CycleEmotionGAN can adapt source domain images to have similar distributions to the target domain without using aligned image pairs. Simultaneously, the annotation information of the source images is preserved. Extensive experiments are conducted on the ArtPhoto and FI datasets, and the results demonstrate that CycleEmotionGAN significantly outperforms the state-of-the-art UDA approaches. Sicheng Zhao, Chuang Lin 0003, Pengfei Xu 0013, Sendong Zhao, Ravi Krishna, Guiguang Ding, Kurt Keutzer |
AAAI | 4 |
| 2019 | GRAPHENE: A Precise Biomedical Literature Retrieval Engine with Graph Augmented Deep Learning and External Knowledge EmpowermentabstractEffective biomedical literature retrieval (BLR) plays a central role inprecision medicine informatics. In this paper, we propose GRAPHENE,which is a deep learning based framework for precise BLR. GRAPHENEconsists of three main different modules 1) graph-augmented doc-ument representation learning; 2) query expansion and represen-tation learning and 3) learning to rank biomedical articles. Thegraph-augmented document representation learning module con-structs a document-concept graph containing biomedical conceptnodes and document nodes so that global biomedical related con-cept from external knowledge source can be captured, which isfurther connected to a BiLSTM so both local and global topics canbe explored. Query expansion and representation learning moduleexpands the query with abbreviations and different names, and thenbuilds a CNN-based model to convolve the expanded query andobtain a vector representation for each query. Learning to rank min-imizes a ranking loss between biomedical articles with the queryto learn the retrieval function. Experimental results on applyingour system to TREC Precision Medicine track data are provided todemonstrate its effectiveness. Sendong Zhao, Chang Su 0002, Andrea Sboner, Fei Wang 0001 |
CIKM | 1 |
| 2017 | Mining Medical Causality for Diagnosis AssistanceabstractIn the medical context, causal knowledge usually refers to causal relations between diseases and symptoms, living habits and diseases, symptoms which get better and therapy, drugs and side-effects, etc [3]. All these causal relations are usually in medical literature, forum and clinical cases and compose the core part of medical diagnosis. Therefore, mining these causal knowledge to predict disease and recommend therapy is of great value for assisting patients and professionals. Sendong Zhao |
WSDM | 1 |
| 2017 | Constructing and Embedding Abstract Event Causality Networks from Text SnippetsabstractIn this paper, we formally define the problem of representing and leveraging abstract event causality to power downstream applications. We propose a novel solution to this problem, which build an abstract causality network and embed the causality network into a continuous vector space. The abstract causality network is generalized from a specific one, with abstract event nodes represented by frequently co-occurring word pairs. To perform the embedding task, we design a dual cause-effect transition model. Therefore, the proposed method can obtain general, frequent, and simple causality patterns, meanwhile, simplify event matching. Given the causality network and the learned embeddings, our model can be applied to a wide range of applications such as event prediction, event clustering and stock market movement prediction. Experimental results demonstrate that 1) the abstract causality network is effective for discovering high-level causality rules behind specific causal events; 2) the embedding models perform better than state-of-the-art link prediction techniques in predicting events; and 3) the event causality embedding is an easy-to-use and sophisticated feature for downstream applications such as stock market movement prediction. Sendong Zhao, Quan Wang 0002, Sean Massung, Bing Qin 0001, Ting Liu 0001, Bin Wang 0004, ChengXiang Zhai |
WSDM | 1 |
| 2016 | Event causality extraction based on connectives analysis
Sendong Zhao, Ting Liu 0001, Sicheng Zhao, Yiheng Chen, Jian-Yun Nie |
Neurocomputing | 1 |
| 2016 | Multi-modal microblog classification via multi-task learning
Sicheng Zhao, Hongxun Yao, Sendong Zhao, Xuesong Jiang, Xiaolei Jiang |
Multim. Tools Appl. | 3 |
| 2012 | Cookie-Proxy: A Scheme to Prevent SSLStrip Attack
Sendong Zhao, Ding Wang 0002, Sicheng Zhao, Chunguang Ma |
ICICS | 1 |
| 2012 | Breaking a Robust Remote User Authentication Scheme Using Smart Cards
Ding Wang 0002, Chunguang Ma, Sendong Zhao, Chang-li Zhou |
NPC | 3 |