Event Causality Identification (ECI) is a critical and challenging Natural Language Processing (NLP) task. Despite Large Language Models (LLMs) offer the potential for ground-breaking achievements through In-Context Learning (ICL), they still demonstrate significant weaknesses in ECI, as evidenced by series causal hallucinations. We argue that this is due to three major shortcomings of ICL for ECI: (1) conventional retrieval methods fail to ensure sufficient causal similarity between input queries and ICL demonstrations; (2) the lack of explanations of causal relationships in demonstrations leading to poor ICL effectiveness; and (3) limited non-causal knowledge in LLMs causing a misalignment between ICL and human causal cognition. In this paper, we propose CarICL to address the aforementioned issues by: (1) incorporating causality-aware representations in demonstration retrieval; (2) enriching the demonstrations with cause-and-effect reasoning; and (3) aligning LLMs with human causal cognition through causality preference optimization. Experimental results show that CarICL outperforms state-of-the-art baselines on three widely used sentence-level ECI benchmarks.

Biomedical relation extraction is crucial for many applications such as biomedical knowledge graph construction and question answering. It is difficult for a typical neural network model to clearly understand the meaning of the complex biomedical text without any knowledge. The knowledge includes external knowledge and inherent prior knowledge within the dataset. The existing methods always integrate the embedded external knowledge obtained through translation-based models like TransE, which is insufficient for a clear understanding of the biomedical entities and relations. In addition, the corpus itself contains abundant prior knowledge that can aid in learning discriminative features. However, this valuable knowledge is not fully utilized in current methods for biomedical relation extraction. In this work, we propose a multi-channel knowledge-enhanced model to extract biomedical relation. It incorporates the external knowledge and the inherent prior knowledge within the dataset into a neural network using multiple channels. On the one hand, the external entity knowledge is deeply exploited by the sequential and structural knowledge channels, respectively. On the other hand, we also explore the prior knowledge in the dataset using an external attention in the prior knowledge channel. The experimental results demonstrate that the proposed model is effective for the biomedical relation extraction.

In the Multimodal Sentiment Analysis task, most existing approaches focus on extracting modality-consistent information from raw unimodal data and integrating it into multimodal representations for sentiment classification. However, these methods often assume that all modalities contribute equally to model performance, prioritizing the extraction and enhancement of consistent information, while overlooking the adverse effects of noise caused by modality inconsistency. In contrast to these approaches, this paper introduces a novel approach namely text-guided Hierarchical Noise Eliminator (t-HNE). This model consists of a two-stage denoising phase and a feature recovery phase. Firstly, textual information is injected into both visual and acoustic modalities using an attention mechanism, aiming to reduce intra-modality noise in the visual and acoustic representations. Secondly, it further mitigates inter-modality noise by maximizing the mutual information between textual representations and the respective visual and acoustic representations. Finally, to address the potential loss of modality-invariant information during denoising, the fused multimodal representation is refined through contrastive learning with each unimodal representation except the textual. Extensive experiments conducted on the CMU-MOSI and CMU-MOSEI datasets demonstrate the efficacy of our approach.

Zero-shot event-relational reasoning is an important task in natural language processing, and existing methods jointly learn a variety of event-relational prefixes and inference-form prefixes to achieve such tasks. However, training prefixes consumes large computational resources and lacks interpretability. Additionally, learning various relational and inferential knowledge inefficiently exploits the connections between tasks. Therefore, we first propose a method for Reasoning-Oriented Locating and Editing (ROLE), which locates and edits the key modules of the language model for reasoning about event relations, enhancing interpretability and also resource-efficiently optimizing the reasoning ability. Subsequently, we propose a method for Analogy-Based Locating and Editing (ABLE), which efficiently exploits the similarities and differences between tasks to optimize the zero-shot reasoning capability. Experimental results show that ROLE improves interpretability and reasoning performance with reduced computational cost. ABLE achieves SOTA results in zero-shot reasoning.

The extraction and fusion of multimodal features are critical for the Multimodal Sentiment Analysis task. Attention mechanisms, known for their remarkable ability to capture key information, are widely used in this process. However, existing attention mechanisms rely on only one or more artificially isolated unimodal features during attention computation, which limits their ability to capture the holistic sentiment conveyed by multimodal inputs. In contrast, humans perceive all modalities simultaneously to more accurately assess sentiment. In this paper, we propose the Unified-Modality Attention Network (UAT). Our method first aligns modalities at feature-level, then we design partial graph attention network to transform unimodal features into unified-modality feature units. Self-attention is applied to these units to perform attention operations directly on unifed-modality like human. Finally, a contrastive learning network is constructed to recover modal continuity lost during alignment. Experiments on CMU-MOSI and CMU-MOSEI datasets demonstrate the effectiveness of our approach.

Biomedical document-level relation extraction (RE) aims to extract relation facts from unstructured biomedical documents and plays an important role in downstream tasks. Graph-based methods solve the problem that sequence-based methods cannot extract long-distance entity relationships, but ignore the fact that graph node connections should be dynamic rather than static. Besides, the existing methods usually introduce external knowledge to address the method's performance bottleneck caused by the limited information contained in the dataset itself. But they fail to consider utilizing external knowledge through explicitly enriching graph connectivity. For the above problems, we propose a novel document-level relation extraction model based on a knowledge-enhanced graph and dynamic generative adversarial network (KG-DGAN). Specifically, a knowledge-enhanced graph is constructed based on the documents and external knowledge information together, where the external knowledge is used to explicitly enhance the connectivity of the graph. Then, the dynamic generative adversarial network (DGAN) can dynamically soften the edge weights and node representations, which reduces the redundant information and enhances useful information during aggregation. We evaluate our method on the widely used CDR and CHR dataset. The final experimental results confirm that the proposed method achieves novel state-of-the-art performances.

Biomedical Event Causal Relation Extraction (BECRE) aims to identify event causal relations in biomedical literature. Current methods emphasize sample data optimization. First, although data augmentation tackles the low-resource issue, it only augments data from the lexical level. Second, integrating external knowledge enriches data diversity, yet fails to select targeted knowledge. Third, fine-grained sample denoising is often overlooked. To resolve the aforementioned issues, we introduce Multiple Optimization with Retrieval-Augmented Generation and Fine-Grained Denoising (MoRAG-FD) framework. It augments data from multiple perspectives, which mainly include Retrieval-Augmented Generation for semantic enrichment and targeted selection of external knowledge. Additionally, we accomplish fine-grained data denoising by assigning near-zero weights to entity pairs without syntactic dependencies in events, rather than simply filtering them based on word index distance. Experimental results show our framework outperforms current methods on Hahn-Powell’s dataset and BioCause dataset.

Biomedical event detection is a critical task that seeks to identify event triggers of certain types in texts. Most existing methods focus on supervised learning schemes, which require a large amount of annotated data and cannot generalize to rare or unseen event types that emerge with only a few or no annotated data available. Existing research on low-resource biomedical event detection has only been conducted in few-shot scenarios, and cannot simultaneously detect rare and unseen event types. In addition, previous work has focused on detecting flat events, neglecting the detection of overlapping events. To address the above problems, we propose LRBED, a unified meta learning framework for both few- and zero-shot biomedical event detection, which can learn from seen event types, yet with the meta objective to generalize in rare and unseen event types. To tackle the detection issues of flat and overlapping events, in our framework, we further propose a novel event-aware semantic matching model that uses event types as semantically rich event-aware prompts to extract candidate triggers from the input text. With our designed joint learning meta objective based on contrastive learning and attention mechanism, our model can better capture the semantic relevance between event-aware prompts and potential trigger words in the input text. In this way, an overlapping event will be combined with all event-aware prompts, which can match all its corresponding event types, solving the problem of flat and overlapping event detection in low-resource scenarios. Experiments on the benchmark datasets demonstrate the effectiveness of LRBED in both few-shot and zero-shot scenarios, and LRBED outperforms the existing method in the same few-shot scenarios. In addition, we also validate the ability of our model to detect overlapping events in low-resource scenarios.

Event representation learning plays a crucial role in numerous natural language processing (NLP) tasks, as it facilitates the extraction of semantic features associated with events. Current methods of learning event representation based on contrastive learning processes positive examples with single-grain random masked language model (MLM), but fall short in learn information inside events from multiple aspects. In this paper, we introduce multi-grained contrastive learning and triple-mixture of experts (MCTM) for event representation learning. Our proposed method extends the random MLM by incorporating a specialized MLM designed to capture different grammatical structures within events, which allows the model to learn token-level knowledge from multiple perspectives. Furthermore, we have observed that mask tokens with different granularities affect the model differently, therefore, we incorporate mixture of experts (MoE) to learn importance weights associated with different granularities. Our experiments demonstrate that MCTM outperforms other baselines in tasks such as hard similarity and transitive sentence similarity, highlighting the superiority of our method.

Few-shot Event Detection (FSED) is a meaningful task due to the limited labeled data and expensive manual labeling. Some prompt-based methods are used in FSED. However, these methods require large GPU memory due to the increased length of input tokens caused by concatenating prompts, as well as additional human effort for designing verbalizers. Moreover, they ignore instance and prompt biases arising from the confounding effects between prompts and texts. In this paper, we propose a prototype-based prompt-instance Interaction with causal Intervention (2xInter) model to conveniently utilize both prompts and verbalizers and effectively eliminate all biases. Specifically, 2xInter first presents a Prototype-based Prompt-Instance Interaction (PPII) module that applies an interactive approach for texts and prompts to reduce memory and regards class prototypes as verbalizers to avoid design costs. Next, 2xInter constructs a Structural Causal Model (SCM) to explain instance and prompt biases and designs a Double-View Causal Intervention (DVCI) module to eliminate these biases. Due to limited supervised information, DVCI devises a generation-based prompt adjustment for instance intervention and a Siamese network-based instance contrasting for prompt intervention. Finally, the experimental results show that 2xInter achieves state-of-the-art performance on RAMS and ACE datasets.

Biomedical causal relation extraction (BCRE) as a sub-task of biomedical information extraction aims to extract event causal relation facts from unstructured biomedical texts and plays an important role in some downstream tasks. The existing methods usually apply oversampling to solve the problems caused by the unbalanced distribution and limited knowledge of the datases, which may ignore the sample diversity. In addition, they usually encode the text by the pre-trained language model BioBERT, which can only obtain context information of the text and may limit the performance because of the insufficiently extracted text information. To solve the above mentioned problems, in this paper, we propose a Multi-source Knowledge Fusion Network (MKFN) to augment data as well as sufficiently extract and fuse the text information and the external knowledge for biomedical causal relation extraction. Specifically, we apply the large language model Roberta to augment samples in minority classes and filter the external knowledge from multiple knowledge bases with the relevance of the text to triples captured by the structure information. Afterward, the multi-source knowledge embedding including context information, structure information and the corresponding external knowledge is acquired by various different encoders. Additionally, we utilize the triplet attention, which is introduced into event relation extraction for the first time, to fuse the multi-source knowledge embedding. Extensive experimental results on Hahn-Powell’s and BioCause datasets confirm that the proposed method achieves novel state-of-the-art performance compared with the current advances.

Medical events, such as diagnostic events, treatment events and examination events, and the relationships between these medical events are of importance in medical research. Moreover, the temporal relationship is one of the most basic medical event relationships. However, people often cannot clearly obtain the relationship between medical events directly from electronic medical records. This defect can lead to inconvenient query construction, inefficient query execution, and poor query performance when queries need to be made for certain diseases or symptoms. In this paper, we propose a method to construct a time-guided medical event chain, which combines medical events of patients with the same disease or symptoms according to the temporal relationship. We use BERT and Bi-LSTM joint encoding to learn contextual information to detect medical event trigger words, and test the effectiveness of our model on MLEE dataset. Finally, we extract medical event chains on the CCKS2020 evaluation dataset. Through the event chain, patients and doctors can more easily understand the trend of the disease and analyze the disease and the treatment effect, and it is helpful for some complex clinical research. Moreover, medical event chain can be easily integrated into existing medical knowledge graph (such as CMeKG) for auxiliary diagnosis.

Extracting relationships among events typically aims to recognize the precise relation between two given events. For the task of event causal relation extraction in the biomedical domain, it has been extremely challenging since the issue of class imbalance. Existing work proposes to address that challenge with certain traditional data augmentation methods, but this approach has been of limited help. Furthermore, existing work lacks fine-grained event relation identification for available datasets, but focuses only on determining whether two given events are causally related or not. With the development of pre-trained language models in recent years, inspired by prompt learning, we propose a framework for biomedical event causal relation extraction based on dual-head prompt augmentation and prototypical network filtering. By comparing the framework with existing methods, we validate the effectiveness of our method.

Biomedical event relation extraction (BioERE) plays an important role in many downstream biological applications. Recent efforts based on supervised learning from small hand-labeled data usually suffer from low coverage of relation topics and limited scale. These shortages make supervised methods hard to achieve competitive performances and predict unseen relations in other topics. Thus, we explore the open-topic BioERE task to simultaneously extract event relations of multiple topics based on the automatically labeled large-scale training data via detecting key meta paths using distant supervision. The experimental results show that the quality of the generated data is competitive to enhance the performances of the open-topic BioERE models.

Biomedical entity and relation extraction has attracted increasing attention recently, whereas it remains challenging due to its domain-specific features for the biomedical corpus. Hence, many researchers consider utilizing external knowledge from large-scale databases to enhance the semantic understanding of models. However, these knowledge-enhanced methods usually enrich context information by incorporating the context-independent knowledge into entity representations and lack effective interaction. Actually, inspired by pre-trained language models, we argue that knowledge representations need to be trainable and adapted for different contexts. Therefore, we propose Knowledge-enhanced Dual-channel Iterative Model (KeDcIM), a novel end-to-end joint model for biomedical entity and relation extraction. Experiments show that KeDcIM achieves new state-of-the-art results on two benchmark datasets.

The structural knowledge graph is crucial external resource for Drug-Drug Interaction (DDI) extraction. However, it is challenging to combine the structural information of knowledge graph and the semantic representation derived from the neural network. Therefore, we propose to extract DDI by constructing a drug knowledge graph, and pre-train a model to learn drug knowledge embeddings adaptive for DDI. Our model performs well on DDIExtraction 2013 dataset, which demonstrates the effectiveness.

Document-level biomedical relation extraction (Bio-DocuRE) is an important branch of biomedical text mining that aims to automatically extract all relation facts from the biomedical text. Since there are a considerable number of relations in biomedical documents that need to be judged by other existing relations, logical reasoning has become a research hotspot in the past two years. However, current models with reasoning are single-granularity only based on one element information, ignoring the complementary fact of different granularity reasoning information. In addition, obtaining rich document information is a prerequisite for logical reasoning, but most of the previous models cannot sufficiently utilize document information, which limits the reasoning ability of the model. In this paper, we propose a novel Bio-DocuRE model called FILR, based on Multi-Dimensional Fusion Information and Multi-Granularity Logical Reasoning. Specifically, FILR presents a multi-dimensional information fusion module MDIF to extract sufficient global document information. Then FILR proposes a multi-granularity reasoning module MGLR to obtain rich inference information through the reasoning of both entity-pairs and mention-pairs. We evaluate our FILR model on two widely used biomedical corpora CDR and GDA. Experimental results show that FILR achieves state-of-the-art performance.

Document-level relation extraction (RE) aims to extract relations among entities within a document, which is more complex than its sentence-level counterpart, especially in biomedical text mining. Chemical-disease relation (CDR) extraction aims to extract complex semantic relationships between chemicals and diseases entities in documents. In order to identify the relations within and across multiple sentences at the same time, existing methods try to build different document-level heterogeneous graph. However, the entity relation representations captured by these models do not make full use of the document information and disregard the noise introduced in the process of integrating various information. In this paper, we propose a novel model DAM-GAN to document-level biomedical RE, which can extract entity-level and mention-level representations of relation instances with R-GCN and Dual-Attention Multi-Instance Learning (DAM) respectively, and eliminate the noise with Generative Adversarial Network (GAN). Entity-level representations of relation instances model the semantic information of all entity pairs from the perspective of the whole document, while the mention-level representations from the perspective of mention pairs related to these entity pairs in different sentences. Therefore, entity- and mention-level representations can be better integrated to represent relation instances. Experimental results demonstrate that our model achieves superior performance on public document-level biomedical RE dataset BioCreative V Chemical Disease Relation(CDR).

Extracting valuable information from the biomedical literature is gaining attention among researchers, and Biomedical Named Entity Recognition (BioNER) becomes one of the most essential tasks in text mining. Previous studies have shown that long-distance interactions between words play an important role in enhancing hidden representations in NER. However, the existing mainstream NER models treat texts as plain linear sequences, resulting in the loss of structural information in sentences, such as BiLSTM-CRF. To overcome this shortcoming, this paper proposes BGGF, a novel model that is able to capture both semantic information in text sequences and structural information in dependency trees. Experiments are conducted on BC2GM and NCBI disease datasets to demonstrate the effectiveness of the proposed model in improving BioNER. The resulting model achieves state-of-the-art performance on the BC2GM and NCBI disease datasets.

Biomedical event extraction is a challenging task in biomedical information extraction. There exist two main problems in previous works: (1) Existing methods are insufficient to capture the indicative information from distant context. (2) Existing methods are not skilled at generating the hierarchical representations for each sentence, which is important for the downstream classification task. In this paper, we propose a novel Hierarchical Distillation Network (HDN) for biomedical event extraction. Firstly, HDN encodes a given sentence from multiple perspectives: a bidirectional GRU (BI-GRU) is employed for sequential encoding and several graph convolution networks (GCN) are employed for multi-order syntactic encodings. Second, HDN integrates the distillation module which can emphasize the difference among all levels of encodings to reduce the redundant information, then HDN adopts residual connection to obtain more expressive sentence representation. Finally, we perform biomedical event extraction on the commonly used Multi-Level Event Extraction (MLEE) corpus and achieve an F1-score of 62.74% which is 3.13% higher than the previous best.

Biomedical named entity recognition (Bio-NER) is an important preliminary step for many biomedical text mining tasks. The current mainstream methods for NER are based on the neural networks to avoid the complex hand-designed features derived from various linguistic analyses. However, these methods ignore some potential sentence-level semantic information and general features of semantic and syntactic. Therefore, we propose a novel Long Short Term Memory (LSTM) Networks model integrating language model and sentence-level reading control gate (LS-BLSTM-CRF) for Bio-NER. In our model, a sentence-level reading control gate (SC) is inserted into the networks to integrate the implicit meaning of an entire sentence and the language model is integrated to our model to learn richer potential features. Besides, character-level embeddings are introduced as the input to deal with out-of-vocabulary words. The experimental results conducted on the BioCreative II GM corpus show that our method can achieve an F-score of 89.94 percent, which outperforms all state-of-the-art systems and is 1.33 percent higher than the best performing neural networks.

Biomedical event extraction is important for medical research and disease prevention, which has attracted much attention in recent years. Traditionally, most of the state-of-the-art systems have been based on shallow machine learning methods, which require many complex, hand-designed features. In addition, the words encoded by one-hot are unable to represent semantic information. Therefore, we utilize dependency-based embeddings to represent words semantically and syntactically. Then, we propose a parallel multi-pooling convolutional neural network (PMCNN) model to capture the compositional semantic features of sentences. Furthermore, we employ a rectified linear unit, which creates sparse representations with true zeros, and which is adapted to the biomedical event extraction, as a nonlinear function in PMCNN architecture. The experimental results from MLEE dataset show that our approach achieves an F1 score of 80.27 percent in trigger identification and an F1 score of 59.65 percent in biomedical event extraction, which performs better than other state-of-the-art methods.

Aspect-level sentiment analysis aims to identify the sentiment of a specific target in its context.Previous works have proved that the interactions between aspects and the contexts are important.On this basis, we also propose a succinct hierarchical attention based mechanism to fuse the information of targets and the contextual words.In addition, most existing methods ignore the position information of the aspect when encoding the sentence.In this paper, we argue that the position-aware representations are beneficial to this task.Therefore, we propose a hierarchical attention based position-aware network (HAPN), which introduces position embeddings to learn the position-aware representations of sentences and further generate the target-specific representations of contextual words.The experimental results on SemEval 2014 dataset show that our approach outperforms the state-of-theart methods.

BACKGROUND: Biomedical event extraction is a crucial task in biomedical text mining. As the primary forum for international evaluation of different biomedical event extraction technologies, BioNLP Shared Task represents a trend in biomedical text mining toward fine-grained information extraction (IE). The fourth series of BioNLP Shared Task in 2016 (BioNLP-ST'16) proposed three tasks, in which the Bacteria Biotope event extraction (BB) task has been put forward in the earlier BioNLP-ST. Deep learning methods provide an effective way to automatically extract more complex features and achieve notable results in various natural language processing tasks. RESULTS: The experimental results show that the presented approach can achieve an F-score of 57.42% in the test set, which outperforms previous state-of-the-art official submissions to BioNLP-ST 2016. CONCLUSIONS: In this paper, we propose a novel Gated Recurrent Unit Networks framework integrating attention mechanism for extracting biomedical events between biotope and bacteria from biomedical literature, utilizing the corpus from the BioNLP'16 Shared Task on Bacteria Biotope task. The experimental results demonstrate the potential and effectiveness of the proposed framework.

BACKGROUND: In biomedical information extraction, event extraction plays a crucial role. Biological events are used to describe the dynamic effects or relationships between biological entities such as proteins and genes. Event extraction is generally divided into trigger detection and argument recognition. The performance of trigger detection directly affects the results of the event extraction. In general, the traditional method is used to address the trigger detection as a classification task, as well as the use of machine learning or rules method, which construct many features to improve the classification results. Moreover, the classification model only recognizes triggers composed of single words, whereas for multiple words, the result is unsatisfactory. RESULTS: The corpus of our model is MLEE. If we were to only use the biomedical LSTM and CRF model without other features, the F-score would reach about 78.08%. Comparing entity to part of speech (POS), we find the entity features more conducive to the improvement of performance of detection, with the F-score potentially reaching about 80%. Furthermore, we also experiment on the other three corpora (BioNLP 2009, BioNLP 2011, and BioNLP 2013) to verify the generalization of our model. Hence, F-scores can reach more than 60%, which are better than the comparative experiments. CONCLUSIONS: The trigger recognition method based on the sequence annotation model does not require initial complex feature engineering, and only requires a simple labeling mechanism to complete the training. Therefore, generalization of our model is better compared to other traditional models. Secondly, this method can identify multi-word triggers, thereby improving the F-scores of trigger recognition. Thirdly, details on the entity have a crucial impact on trigger detection. Finally, the combination of character-level word embedding and word-level word embedding provides increasingly effective information for the model; therefore, it is a key to the success of the experiment.

Extracting biomedical events from biomedical literature plays an important role in the field of biomedical text mining, and the trigger detection is a key step in biomedical event extraction. We propose a two-stage method for trigger detection, which divides trigger detection into recognition stage and classification stage, and different features are selected in each stage. In the first stage, we select the features which are more suitable for recognition, and in the second stage, the features that are more helpful to classification are adopted. Furthermore, we integrate word embeddings to represent words semantically and syntactically. On the multi-level event extraction (MLEE) corpus test dataset, our method achieves an F-score of 79.75 percent, which outperforms the state-of-the-art systems.

Biomedical named entity recognition (Bio-NER) is an important preliminary step for many biomedical text mining tasks. The current mainstream methods for NER are based on the neural networks to avoid the complex hand-designed features derived from various linguistic analyses. However, the performance of these methods is always limited to exploring dependencies across output label and ignoring some potential word-level and sentence-level semantic information. Therefore, we propose a novel Long Short Term Memory (LSTM) Networks model SC-LSTM-CRF integrating two channels and the sentence-level embeddings for Bio-NER. In our model, two channels word embeddings are introduced as the input to obtain more abundant potential information, and the sentence-level reading control gate (SC) is inserted into the networks to integrate the implicit meaning of an entire sentence. The experimental results conducted on the BioCreative II GM corpus show that our method can achieve an F-score of 89.49%, which outperforms all state-of-the-art systems and is 0.88% higher than the best performing neural networks.

In biomedical research, events revealing complex relations between entities play an important role. Event trigger identification is a crucial and prerequisite step in the pipeline process of biomedical event extraction. There exist two main problems in the previous work: (1) Traditional feature-based methods often rely on human ingenuity, which is a time-consuming process. Though most representation-based methods overcome this problem, these methods usually depend on local sentence representation features only within a window. (2) In current biomedical event trigger identification methods, arguments annotated in training set which can provide significant clues are completely ignored or exploited in an indirect manner. In this paper, we propose a Recurrent Neural Networks (RNN) based model considering argument information achieved via supervised attention mechanisms, which can automatically extract context features across the sentence and arguments clues. Meanwhile, we also introduce the dependency-based word embeddings in order to represent more dependency-based semantic information. Experimental results on the Multi Level Event Extraction (MLEE) corpus show that 1.14% improvement on F1-score is achieved by the proposed model when compared to the state-of-the-art approach, demonstrating the effectiveness of the proposed method.

Trigger detection plays a key role in the extraction of biomedical events, so it will influence the results of biomedical events extraction directly. The traditional biomedical event trigger recognition method is based on artificial design features and construct feature vectors; Not only does it consume great amounts of manpower, it also lacks system generalization ability. Most of methods of trigger detection are based on the convolutional neural network that identify each word in the text, and regard it as a multi-classification task. However for the multi-word composed of the trigger, there is no useful recognition effect. In this paper, we will use the IBO format and consider the trigger detection as a task of sequence annotation, a solution that improves the recognition accuracy of multi-word triggers by bidirectional LSTM and CRF.

Extracting knowledge from unstructured text is one of the most important goals of Natural Language Processing, especially in biomedical event extraction domain. In this paper, we describe a system for extracting biomedical events among biotope and bacteria from biomedical literature, using the corpus from the BioNLP'16 Shared Task on Bacteria Biotope task. The current mainstream methods for event extraction are based on shallow machine learning methods. However, these methods mainly rely on domain experience and need enormous manual efforts to select features. Therefore, we propose a novel Long Short Term Memory (LSTM) Networks framework DETBLSTM for event extraction. In our framework, a dynamic extended tree is introduced as the input instead of the original sentences, which utilizes the syntactic information. Furthermore, the POS and distance embeddings are added to enrich input information and thus the complex feature extraction can be skipped. In final, we construct a bidirectional LSTM model to extract biomedical events and achieve 57.14% F-score in the test set. Our model obtains a better F-score than all official submissions to BioNLP-ST 2016, which is 1.34% higher than the best system.

In systems biology, filtering the discriminative features from complex high-dimensional data is a crucial issue. This paper proposes a feature selection algorithm based on feature overlapping and group overlapping (FS-FOGO) to calculate the feature importance. FS-FOGO weighs feature from two aspects: overlapping degree based on the ratio of overlapping area on the effective range of each class and the overlapping degree based on the proportion of heterogeneous samples in every sample's nearest neighbors. To show the validation of FS-FOGO, it is compared with effective range based gene selection (ERGS), which calculates the feature weights based on overlapping area of the effective range, on six public biological data sets and one serum metabolomics data set about liver disease. Naive Bayes and Support Vector Machine are used as classifiers, respectively. The experiment results show that the top ranked features by FS-FOGO are more discriminative and get higher classification accuracy rates than those by ERGS in most cases. And in the metabolomics data, the top ranked metabolites by FS-FOGO could separate different liver diseases well.

In biomedical text mining tasks, distributed word representation has succeeded in capturing semantic regularities, but most of them are shallow-window based models, which are not sufficient for expressing the meaning of words. To represent words using deeper information, we make explicit the semantic regularity to emerge in word relations, including dependency relations and context relations, and propose a novel architecture for computing continuous vector representation by leveraging those relations. The performance of our model is measured on word analogy task and Protein-Protein Interaction Extraction (PPIE) task. Experimental results show that our method performs overall better than other word representation models on word analogy task and have many advantages on biomedical text mining.

Extracting biomedical event from literatures has attracted much attention recently. By now, most of the state-of-the-art systems have been based on pipelines which suffer from cascading errors, and the words encoded by one-hot are unable to represent the semantic information. Joint inference with dual decomposition and novel word embeddings are adopted to address the two problems, respectively, in this work. Word embeddings are learnt from large scale unlabeled texts and integrated as an unsupervised feature into other rich features based on dependency parse graphs to detect triggers and arguments. The proposed system consists of four components: trigger detector, argument detector, jointly inference with dual decomposition, and rule-based semantic post-processing, and outperforms the state-of-the-art systems. On the development set of BioNLP'09, the F-score is 59.77 percent on the primary task, which is 0.96 percent higher than the best system. On the test set of BioNLP'11, the F-score is 56.09 and 0.89 percent higher than the best published result that do not adopt additional techniques. On the test set of BioNLP'13, the F-score reaches 53.19 percent which is 2.22 percent higher than the best result.

Most word embedding methods are proposed with general purpose which take a word as a basic unit and learn embeddings according to words' external contexts. However, in biomedical text mining, there are many biomedical entities and syntactic chunks which contain rich domain information, and the semantic meaning of a word is also strongly related to those information. Hence, we present a biomedical domain-specific word embedding model by incorporating stem, chunk and entity to train word embeddings. We also present two deep learning architectures respectively for two biomedical text mining tasks, by which we evaluate our word embeddings and compare them with other models. Experimental results show that our biomedical domain-specific word embeddings overall outperform other general-purpose word embeddings in these deep learning methods for biomedical text mining tasks.

Biomedical named entity recognition (bio-NER), which extracts important entities such as genes and proteins, has become one of the most fundamental tasks in biomedical knowledge acquisition. However, the performance of traditional NER systems is always limited to the construction of complex hand-designed features which are derived from various linguistic analyses and maybe only adapted to specified area. In this paper we mainly focus on building a simple and efficient system for bio-NER with the extended Recurrent Neural Network (RNN) which considers the predicted information from the prior node and external context information (topical information & clustering information). Extracting complex hand-designed features is skipped and replaced with word embeddings. The experiments conducted on the BioCreative II GM data set demonstrate RNN models outperform CRF model and deep neural networks (DNN); furthermore, the extended RNN model performs better than the original RNN model.

In this paper, we present an actively transfer learning framework to extract PPI. Experimental results show that the proposed ActTrAdaBoost method performs much better than the baseline SVM and the original transfer learning method. In PPIE transfer learning task, our ActTrAdaBoost method presents better performance.

As an important branch of biomedical information extraction, Protein-Protein Interaction extraction (PPIe) from biomedical literatures has been widely researched, and machine learning methods have achieved great success for this task. However, the word feature generally adopted in the existing methods suffers badly from vocabulary gap and data sparseness, weakening the classification performance. In this paper, the unsupervised word representation approach is introduced to address these problems. Three word representation methods are adopted to improve the performance of PPIe: distributed representation, vector clustering and Brown clusters representation. Experimental results show that our method outperforms the state-of-the-art methods on five publicly available corpora.

Previous researches have shown that supervised Protein-Protein Interaction Extraction (PPIE) can get high accuracies with elaborately selected features and kernels. However, most features and kernels rest upon domain knowledge and natural language analysis, which makes the supervised model expensive, heavy and brittle. Moreover, the one-hot encoding, a commonly used representation technique, fails to capture the semantic similarity between words. To reduce the manual labor and overcome the shortage of one-hot encoding, we put forward a general instance representation architecture for PPIE, which integrates word representation and vector composition. Our method obtains F-scores of 69.4%, 78.8%, 76.0%, 74.0% and 81.1% on AIMed, BioInfer, HPRD50, IEPA and LLL respectively.

Coreference resolution recently plays a more and more important role for many natural language processing tasks. In this paper, we propose two methods for the biomedical coreference resolution. One is the single machine learning method (SVM ranker-learning algorithm) which selects appropriate features for the pronoun and noun phrase coreference resolution respectively. The other one is the hybrid method which adopts the rule-based method or the machine learning method for relative pronouns, non-relative pronouns and noun phrases coreference resolution respectively. Experiments are carried out on Biomedical Natural Language Process Shared Task (BioNLP-ST)12011 coreference resolution corpus. In the first method (the single machine learning method), the F-score is 49.36%, higher than that using the same method with the features in the Reconcile system by 10.06%. In the second method (the hybrid method), the F-score is 68.61%, higher than that of the currently best system by 1.21%.

Protein-Protein Interaction (PPI) extraction from literatures is becoming a more and more significant task in the biomedical information extraction. Though many methods for PPI extraction have achieved promising results, they all concentrated on the abstracts of literatures rather than full texts. In this paper, we append full-text features, namely Location and Co-occurrence to extract PPIs from full texts. Location describes where the protein pair appears in the article. Co-occurrence is the frequency of each protein pair occurring in the article. In addition, syntactic patterns are extracted as features, and then feature selection is applied to improve the performance and reduce the dimension of feature vectors in SVM. Finally, the selected features are combined with two-level DET tree kernel. Experimental results show that the presented approach can achieve an F-score of 74.46% and an AUC of 78.50%.

Biomedical named entity recognition (Bio-NER) is a fundamental step in biomedical text mining. This paper presents a two-phase Bio-NER model targeting at JNLPBA task. Our two-phase method divides the task into two subtasks: named entity detection (NED) and named entity classification (NEC). The NED subtask is accomplished based on the two-layer stacking method in the first phase, where named entities (NEs) are distinguished from nonnamed-entities (NNEs) in biomedical literatures without identifying their types. Then six classifiers are constructed by four toolkits (CRF++, YamCha, maximum entropy, Mallet) with different training methods and integrated based on the two-layer stacking method. In the second phase for the NEC subtask, the multiagent strategy is introduced to determine the correct entity type for entities identified in the first phase. The experiment results show that the presented approach can achieve an F-score of 76.06 percent, which outperforms most of the state-of-the-art systems.

Bilingual Named Entity (NE) pairs are valuable resources for many NLP applications. Since comparable corpora are more accessible, abundant and up-to-date, recent researches have concentrated on mining bilingual lexicons using comparable corpora. Leveraging comparable corpora, this research presents a novel approach to mining English-Chinese NE translations by combining multi-dimension features from various information sources for every possible NE pair, which include the transliteration model, English-Chinese matching, Chinese-English matching, translation model, length, and context vector. These features are integrated into one model with linear combination and minimum sample risk (MSR) algorithm. As for the high type-dependence of NE translation, we integrate different features according to different NE types. We experiment with the above individual feature or integrated features to mine person NE (PN) pairs, location NE (LN) pairs and organization NE (ON) pairs. When using transliteration and length to mine PN pairs, we achieve the best performance of 84.9% ( F -score). The LN pairs can be mined with the features of transliteration model, length, translation model, English-Chinese matching and Chinese-English matching. And the best performance is 83.4% ( F -score). The ON pairs can be mined with the features of English-Chinese matching and Chinese-English matching. It reaches the best performance with 84.1% ( F -score).