Medical named entity recognition is a fundamental task in biomedical information extraction. Considering the inherent scarcity of labeled data in medical texts, recent studies adopt few-shot learning for the task of medical named entity recognition. However, the existing methods mostly design various models and features according to the original text, without making use of external knowledge and information such as Wikipedia, which is publicly available. In this paper, we propose a knowledgeenhanced few-shot learning method that leverages the contextual information of the original text from external knowledge source, for medical named entity recognition. Experimental results on the I2B2 dataset demonstrate that our proposed model outperforms the existing methods and strong baselines.

As an important research topic in biomedical field, adverse drug reaction detection has received extensive research attention in the past ten years. Recent studies begin to design various neural frameworks with external knowledge to improve the task performance. However, these efforts mainly focus on extracting contextual information or knowledge according to the whole input text, which may introduce a lot of redundant information. In this paper, we propose a hybrid neural network model, which automatically integrates entity-related keywords from external knowledge base, for predicting adverse drug reaction. Experimental results on the publicly available dataset show our proposed model can achieve 93.71% and 88.07% F1 score on entity recognition and relationship extraction, respectively, outperforming the existing methods and strong neural baselines by a large margin.

Genomic variant detection is essential for cancer research and treatment, enabling precise diagnosis and personalized therapies. Considering the scarcity of labeled data, recent studies adopt few-shot learning for the task of genomic variant detection. However, the existing methods face significant challenges, including sensitivity to label distribution variability and difficulties in predicting long-span entities. To address two issues, we propose an enhanced few-shot learning method that integrates adaptive memory module, supervised contrastive learning, and orthogonal transfer module for genomic variant detection. Experimental results on the tmVar dataset demonstrate that our proposed model outperforms the existing methods and strong baselines significantly.

Extracting adverse drug events (ADEs) is one of the most challenging tasks in biomedical field. However, the existing studies fail to fully utilize the syntactic information and capture the long-range dependencies between entities, limiting the performance of the task. To address this issue, this paper introduces a tree-structured neural network model for jointly extracting ADEs entities and their relationships. Results on the publicly available dataset show that our proposed model can achieve 93.34% and 87.40% F1 scores on entity recognition and relation extraction, respectively, outperforming the current best models by a large margin.

Aspect-based sentiment triplet extraction (ASTE) aims at recognizing the joint triplets from texts, i.e., aspect terms, opinion expressions, and correlated sentiment polarities. As a newly proposed task, ASTE depicts the complete sentiment picture from different perspectives to better facilitate real-world applications. Unfortunately, several major challenges, such as the overlapping issue and long-distance dependency, have not been addressed effectively by the existing ASTE methods, which limits the performance of the task. In this article, we present an innovative encoder-decoder framework for end-to-end ASTE. Specifically, the ASTE task is first modeled as an unordered triplet set prediction problem, which is satisfied with a nonautoregressive decoding paradigm with a pointer network. Second, a novel high-order aggregation mechanism is proposed for fully integrating the underlying interactions between the overlapping structure of aspect and opinion terms. Third, a bipartite matching loss is introduced for facilitating the training of our nonautoregressive system. Experimental results on benchmark datasets show that our proposed framework significantly outperforms the state-of-the-art methods. Further analysis demonstrates the advantages of the proposed framework in handling the overlapping issue, relieving long-distance dependency and decoding efficiency.

Aspect-based sentiment analysis (ABSA) aims at automatically inferring the specific sentiment polarities toward certain aspects of products or services behind the social media texts or reviews, which has been a fundamental application to the real-world society. Since the early 2010s, ABSA has achieved extraordinarily high accuracy with various deep neural models. However, existing ABSA models with strong in-house performances may fail to generalize to some challenging cases where the contexts are variable, i.e., low robustness to real-world environments. In this study, we propose to enhance the ABSA robustness by systematically rethinking the bottlenecks from all possible angles, including model, data, and training. First, we strengthen the current best-robust syntax-aware models by further incorporating the rich external syntactic dependencies and the labels with aspect simultaneously with a universal-syntax graph convolutional network. In the corpus perspective, we propose to automatically induce high-quality synthetic training data with various types, allowing models to learn sufficient inductive bias for better robustness. Last, we based on the rich pseudo data perform adversarial training to enhance the resistance to the context perturbation and meanwhile employ contrastive learning to reinforce the representations of instances with contrastive sentiments. Extensive robustness evaluations are conducted. The results demonstrate that our enhanced syntax-aware model achieves better robustness performances than all the state-of-the-art baselines. By additionally incorporating our synthetic corpus, the robust testing results are pushed with around 10% accuracy, which are then further improved by installing the advanced training strategies. In-depth analyses are presented for revealing the factors influencing the ABSA robustness.

Many natural language processing (NLP) tasks appear in dual forms, which are generally solved by dual learning technique that models the dualities between the coupled tasks. In this work, we propose to further enhance dual learning with structure matching that explicitly builds structural connections in between. Starting with the dual text$\leftrightarrow$text generation, we perform dually-syntactic structure co-echoing of the region of interest (RoI) between the task pair, together with a syntax cross-reconstruction at the decoding side. We next extend the idea to a text$\leftrightarrow$non-text setup, making alignment between the syntactic-semantic structure. Over 2*14 tasks covering 5 dual learning scenarios, the proposed structure matching method shows its significant effectiveness in enhancing existing dual learning. Our method can retrieve the key RoIs that are highly crucial to the task performance. Besides NLP tasks, it is also revealed that our approach has great potential in facilitating more non-text$\leftrightarrow$non-text scenarios.

Conversational semantic role labeling (CSRL) is a newly proposed task that uncovers the shallow semantic structures in a dialogue text. Unfortunately several important characteristics of the CSRL task have been overlooked by the existing works, such as the structural information integration, near-neighbor influence. In this work, we investigate the integration of a latent graph for CSRL. We propose to automatically induce a predicate-oriented latent graph (POLar) with a predicate-centered gaussian mechanism, by which the nearer and informative words to the predicate will be allocated with more attention. The POLar structure is then dynamically pruned and refined so as to best fit the task need. We additionally introduce an effective dialogue-level pre-trained language model, CoDiaBERT, for better supporting multiple utterance sentences and handling the speaker coreference issue in CSRL. Our system outperforms best-performing baselines on three benchmark CSRL datasets with big margins, especially achieving over 4% F1 score improvements on the cross-utterance argument detection. Further analyses are presented to better understand the effectiveness of our proposed methods.

Joint aspect category sentiment analysis (ACSA) and rating prediction (RP) is a newly proposed task (namely ASAP) that integrates the characteristics of both fine-grained and coarse-grained sentiment analysis. However, the prior joint models for the ASAP task only consider the shallow interaction between the two granularities. In this work, we gain the inspiration from human intuition, presenting an innovative from-fine-to-coarse reasoning framework for better joint task performance. Our system advances mainly in three aspects. First, we additionally make use of the category label text features, co-encoding them with the input document texts, allowing to accurately capture the key clues of each category. Second, we build a fine-to-coarse hierarchical label graph, modeling the aspect categories and the overall rating as a hierarchical structure for full interaction of the two granularities. Third, we propose to perform global iterative reasoning with a cross-collaboration between the hierarchical label graph and the context graphs, enabling sufficient communication between categories and review contexts. Based on the ASAP dataset, experimental results demonstrate that our proposed framework outperforms state-of-the-art baselines by large margins. Further in-depth analyses prove that our method is effective on addressing both the unbalanced data distribution and the long-text issue.

Attention has been shown highly effective for modeling sequences, capturing the more informative parts in learning a deep representation. However, recent studies show that the attention values do not always coincide with intuition in tasks, such as machine translation and sentiment classification. In this study, we consider using deep reinforcement learning to automatically optimize attention distribution during the minimization of end task training losses. With more sufficient environment states, iterative actions are taken to adjust attention weights so that more informative words receive more attention automatically. Results on different tasks and different attention networks demonstrate that our model is of great effectiveness in improving the end task performances, yielding more reasonable attention distribution. The more in-depth analysis further reveals that our retrofitting method can help to bring explainability for baseline attention.

A majority of research interests in irregular (e.g., nested or discontinuous) named entity recognition (NER) have been paid on nested entities, while discontinuous entities received limited attention. Existing work for discontinuous NER, however, either suffers from decoding ambiguity or predicting using token-level local features. In this work, we present an innovative model for discontinuous NER based on pointer networks, where the pointer simultaneously decides whether a token at each decoding frame constitutes an entity mention and where the next constituent token is. Our model has three major merits compared with previous work: (1) The pointer mechanism is memory-augmented, which enhances the mention boundary detection and interactions between the current decision and prior recognized mentions. (2) The encoder-decoder architecture can linearize the complexity of structure prediction, and thus reduce search costs. (3) The model makes every decision using global information, i.e., by consulting all the input, encoder and previous decoder output in a global view. Experimental results on the CADEC and ShARe13 datasets show that our model outperforms flat and hypergraph models as well as a state-of-the-art transition-based model for discontinuous NER. Further in-depth analysis demonstrates that our model performs well in recognizing various entities including flat, overlapping and discontinuous ones. More crucially, our model is effective on boundary detection, which is the kernel source to NER.

In this paper, we propose to enhance the pair-wise aspect and opinion terms extraction (PAOTE) task by incorporating rich syntactic knowledge. We first build a syntax fusion encoder for encoding syntactic features, including a label-aware graph convolutional network (LAGCN) for modeling the dependency edges and labels, as well as the POS tags unifiedly, and a local-attention module encoding POS tags for better term boundary detection. During pairing, we then adopt Biaffine and Triaffine scoring for high-order aspect-opinion term pairing, in the meantime re-harnessing the syntax-enriched representations in LAGCN for syntactic-aware scoring. Experimental results on four benchmark datasets demonstrate that our model outperforms current state-of-the-art baselines, meanwhile yielding explainable predictions with syntactic knowledge.

Existing works for document-level sentiment classification task treat the review document as an overall text unit, performing feature extraction with various sophisticated model architectures. In this paper, we draw inspiration from fine-grained sentiment analysis, proposing to first learn the latent target-opinion distribution behind the documents, and then leverage such fine-grained prior knowledge into the classification process. We model the latent target-opinion distribution as hierarchical variables, where global-level variable captures the overall target and opinion, and local-level variables retrieve the detailed opinion clues at the word level. The proposed method consists of two main parts: a variational module and a classification module. We employ the conditional variational autoencoder to make reconstructions of the document, during which the user and product information can be integrated. In the classification module, we build a hierarchical model based on Transformer encoders, where the local-level and global-level prior distribution representations induced from the variational module are injected into the word-level and sentence-level Transformers, respectively. Experimental results on benchmark datasets show that the proposed method significantly outperforms strong baselines, achieving the state-of-the-art performance. Further analysis shows that our model is capable of capturing the latent fine-grained target and opinion prior information, which is highly effective for improving the task performance.

Biomedical information extraction (BioIE) is an important task. The aim is to analyze biomedical texts and extract structured information such as named entities and semantic relations between them. In recent years, pre-trained language models have largely improved the performance of BioIE. However, they neglect to incorporate external structural knowledge, which can provide rich factual information to support the underlying understanding and reasoning for biomedical information extraction. In this paper, we first evaluate current extraction methods, including vanilla neural networks, general language models and pre-trained contextualized language models on biomedical information extraction tasks, including named entity recognition, relation extraction and event extraction. We then propose to enrich a contextualized language model by integrating a large scale of biomedical knowledge graphs (namely, BioKGLM). In order to effectively encode knowledge, we explore a three-stage training procedure and introduce different fusion strategies to facilitate knowledge injection. Experimental results on multiple tasks show that BioKGLM consistently outperforms state-of-the-art extraction models. A further analysis proves that BioKGLM can capture the underlying relations between biomedical knowledge concepts, which are crucial for BioIE.

MOTIVATION: Entity relation extraction is one of the fundamental tasks in biomedical text mining, which is usually solved by the models from natural language processing. Compared with traditional pipeline methods, joint methods can avoid the error propagation from entity to relation, giving better performances. However, the existing joint models are built upon sequential scheme, and fail to detect overlapping entity and relation, which are ubiquitous in biomedical texts. The main reason is that sequential models have relatively weaker power in capturing long-range dependencies, which results in lower performance in encoding longer sentences. In this article, we propose a novel span-graph neural model for jointly extracting overlapping entity relation in biomedical texts. Our model treats the task as relation triplets prediction, and builds the entity-graph by enumerating possible candidate entity spans. The proposed model captures the relationship between the correlated entities via a span scorer and a relation scorer, respectively, and finally outputs all valid relational triplets. RESULTS: Experimental results on two biomedical entity relation extraction tasks, including drug-drug interaction detection and protein-protein interaction detection, show that the proposed method outperforms previous models by a substantial margin, demonstrating the effectiveness of span-graph-based method for overlapping relation extraction in biomedical texts. Further in-depth analysis proves that our model is more effective in capturing the long-range dependencies for relation extraction compared with the sequential models. AVAILABILITY AND IMPLEMENTATION: Related codes are made publicly available at http://github.com/Baxelyne/SpanBioER.

Semantic role labeling (SRL) is one of the important tasks in natural language processing. Current end-to-end SRL, compared with traditional pipeline SRL, has achieved competitive performance via graph-based neural models. However, these are all first-order methods, where decisions for detecting predicate-argument pairs are made in isolation with local short-term features, thus being error-prone. Besides, they may potentially suffer from long-distance dependency, and perform poorly on the data-scarce scenario. In this paper, we explore a second-order end-to-end SRL model that considers simultaneously two pairs of predicate-argument when making scoring. We next propose a structural refinement mechanism to further model higher-order interactions at a global scope. After iterations of refinement, the underlying token representations are refined to be more informative. Experimental results show that our model significantly outperforms the state-of-the-art baselines on multiple benchmarks (i.e., CoNLL08, CoNLL09 and CoNLL12). Further analysis demonstrates that our framework is able to learn latent SRL constraints, being more robust on data-scarce settings, meanwhile relieving the long-range dependency issue.

Pair-wise aspect and opinion terms extraction (PAOTE), aiming at detecting the pair of the correlated aspect and opinion terms jointly, recently has drawn increasing research attention in the community of sentiment analysis and opinion mining. Recent works largely employ joint methods for the task, while they do not sufficiently incorporate the external syntactic knowledge, such as the dependency edges and labels. Besides, these methods fail to capture the underlying shared interactions among the overlapping aspect-opinion pair structures. In this paper we address the above issues for better PAOTE. Specifically, we present a span graph-based model for joint PAOTE, based on which we propose a edge-enhanced syntactic graph convolutional network (ESGCN) to simultaneously encode the syntactic dependency edges and the corresponding labels, for enhancing the extraction and pairing of aspect and opinion terms. During relational pairing, we employ a Biaffine and a Triaffine scorer respectively for high-order scoring, fully exploring the mutual interactions among different aspect-opinion pairs. Experimental results on four benchmark datasets show that our proposed method significantly outperforms all the baseline systems, giving the state-of-the-art performances. Further analysis proves the effectiveness of our framework on modeling the syntactic edge and label knowledge, as well as the advance of the high-order pairing mechanism for the task.

Identifying multiple emotions in a sentence is an important research topic. Existing methods usually model the problem as multi-label classification task. However, previous methods have two issues, limiting the performance of the task. First, these models do not consider prior emotion distribution in a sentence. Second, they fail to effectively capture the context information closely related to the corresponding emotion. In this paper, we propose a Latent Emotion Memory network (LEM) for multi-label emotion classification. The proposed model can learn the latent emotion distribution without external knowledge, and can effectively leverage it into the classification network. Experimental results on two benchmark datasets show that the proposed model outperforms strong baselines, achieving the state-of-the-art performance.

We consider retrofitting structure-aware Transformer language model for facilitating end tasks by proposing to exploit syntactic distance to encode both the phrasal constituency and dependency connection into the language model.A middle-layer structural learning strategy is leveraged for structure integration, accomplished with main semantic task training under multi-task learning scheme.Experimental results show that the retrofitted structure-aware Transformer language model achieves improved perplexity, meanwhile inducing accurate syntactic phrases.By performing structure-aware fine-tuning, our model achieves significant improvements for both semantic-and syntactic-dependent tasks.

Identifying multiple emotions in a piece of text is an important research topic in the NLP community.Existing methods usually model the task as a multi-label classification problem, while these work has two issues. First, these methods fail to leverage the topic information of the text, which has been shown to be effective for sentiment analysis task. Second, different parts of the text can contribute differently to predicting different emotion labels, so the proposed model needs to capture effective features for each corresponding emotion, which is not considered by existing models. To tackle these problems, we propose a topic-enhanced capsule network, which contains two main parts: a variational autoencoder and a capsule module, for multi-label emotion detection task. Specifically, the variational autoencoder can learn the latent topic information of the text, and the capsule module can capture rich features for corresponding emotion. Experimental results on two benchmark datasets show that the proposed model achieves the current best performance, outperforming previous methods and strong baselines by a large margin.

Many named entities usually contain nested entities in biomedical texts. Nested entities pose challenge to the task of named entity recognition. Traditional methods try to solve the problem as a graph-structure prediction problem. However, these methods fail to sufficiently capture the boundaries information between nested entities, which limits the performance of the task. In this paper, we take a different view by solving each unique entity type as a separate task, using multi-task learning with dispatched attention to facilitate information exchange between tasks. Results on GENIA corpus show that the proposed method is highly effective, obtaining the best results in the literature.

Drug-drug interaction (DDI) extraction is one of important tasks in biomedical text mining. Compared with traditional pipeline extraction, joint methods enjoy lower error propagation, achieving better performances. However, biomedical texts contain a large number of overlapped relations, and existing joint extraction models fail to capture these entity relations. In this paper, we propose a novel joint model for DDI extraction. Different from the existing sequential models, our span-based neural model treats the task as a relation triplets prediction problem, and builds the entity-graph by enumerating all possible candidate spans, and simultaneously models the relation between the correlated entities. Experimental results show that the proposed method significantly outperforms the strong baselines and previous methods, achieving the state-of-the-art performances.

BACKGROUND: Biomedical named entity recognition(BNER) is a crucial initial step of information extraction in biomedical domain. The task is typically modeled as a sequence labeling problem. Various machine learning algorithms, such as Conditional Random Fields (CRFs), have been successfully used for this task. However, these state-of-the-art BNER systems largely depend on hand-crafted features. RESULTS: We present a recurrent neural network (RNN) framework based on word embeddings and character representation. On top of the neural network architecture, we use a CRF layer to jointly decode labels for the whole sentence. In our approach, contextual information from both directions and long-range dependencies in the sequence, which is useful for this task, can be well modeled by bidirectional variation and long short-term memory (LSTM) unit, respectively. Although our models use word embeddings and character embeddings as the only features, the bidirectional LSTM-RNN (BLSTM-RNN) model achieves state-of-the-art performance - 86.55% F1 on BioCreative II gene mention (GM) corpus and 73.79% F1 on JNLPBA 2004 corpus. CONCLUSIONS: Our neural network architecture can be successfully used for BNER without any manual feature engineering. Experimental results show that domain-specific pre-trained word embeddings and character-level representation can improve the performance of the LSTM-RNN models. On the GM corpus, we achieve comparable performance compared with other systems using complex hand-crafted features. Considering the JNLPBA corpus, our model achieves the best results, outperforming the previously top performing systems. The source code of our method is freely available under GPL at https://github.com/lvchen1989/BNER .

Sentiment classification on Twitter has attracted increasing research in recent years.Most existing work focuses on feature engineering according to the tweet content itself.In this paper, we propose a context-based neural network model for Twitter sentiment analysis, incorporating contextualized features from relevant Tweets into the model in the form of word embedding vectors.Experiments on both balanced and unbalanced datasets show that our proposed models outperform the current state-of-the-art.

It has been shown that learning distributed word representations is highly useful for Twitter sentiment classification.Most existing models rely on a single distributed representation for each word.This is problematic for sentiment classification because words are often polysemous and each word can contain different sentiment polarities under different topics.We address this issue by learning topic-enriched multi-prototype word embeddings (TMWE).In particular, we develop two neural networks which 1) learn word embeddings that better capture tweet context by incorporating topic information, and 2) learn topic-enriched multiple prototype embeddings for each word.Experiments on Twitter sentiment benchmark datasets in SemEval 2013 show that TMWE outperforms the top system with hand-crafted features, and the current best neural network model.

Deceptive opinion spam detection has attracted significant attention from both business and research communities. Existing approaches are based on manual discrete features, which can capture linguistic and psychological cues. However, such features fail to encode the semantic meaning of a document from the discourse perspective, which limits the performance. In this paper, we empirically explore a neural network model to learn document-level representation for detecting deceptive opinion spam. In particular, given a document, the model learns sentence representations with a convolutional neural network, which are combined using a gated recurrent neural network with attention mechanism to model discourse information and yield a document vector. Finally, the document representation is used directly as features to identify deceptive opinion spam. Experimental results on three domains (Hotel, Restaurant, and Doctor) show that our proposed method outperforms state-of-the-art methods.

We propose a transition-based model for joint word segmentation, POS tagging and text normalization.Different from previous methods, the model can be trained on standard text corpora, overcoming the lack of annotated microblog corpora.To evaluate our model, we develop an annotated corpus based on microblogs.Experimental results show that our joint model can help improve the performance of word segmentation on microblogs, giving an error reduction in segmentation accuracy of 12.02%, compared to the traditional approach.

Deceptive reviews detection has attracted significant attention from both business and research communities.However, due to the difficulty of human labeling needed for supervised learning, the problem remains to be highly challenging.This paper proposed a novel angle to the problem by modeling PU (positive unlabeled) learning.A semi-supervised model, called mixing population and individual property PU learning (MPIPUL), is proposed.Firstly, some reliable negative examples are identified from the unlabeled dataset.Secondly, some representative positive examples and negative examples are generated based on LDA (Latent Dirichlet Allocation).Thirdly, for the remaining unlabeled examples (we call them spy examples), which can not be explicitly identified as positive and negative, two similarity weights are assigned, by which the probability of a spy example belonging to the positive class and the negative class are displayed.Finally, spy examples and their similarity weights are incorporated into SVM (Support Vector Machine) to build an accurate classifier.Experiments on gold-standard dataset demonstrate the effectiveness of MPIPUL which outperforms the state-of-the-art baselines.