Multimodal Sentiment Analysis (MSA) seeks to understand human emotions by integrating textual, acoustic, and visual signals. Although multimodal fusion is designed to leverage cross-modal complementarity, real-world scenarios often exhibit modality competition: dominant modalities tend to overshadow weaker ones, leading to suboptimal performance. In this paper, we propose PaSE, a novel Prototype-aligned Calibration and Shapley-optimized Equilibrium framework, which enhances collaboration while explicitly mitigating modality competition. PaSE first applies Prototype-guided Calibration Learning (PCL) to refine unimodal representations and align them through an Entropic Optimal Transport mechanism that ensures semantic consistency. To further stabilize optimization, we introduce a Dual-Phase Optimization strategy. A prototype-gated fusion module is first used to extract shared representations, followed by Shapley-based Gradient Modulation (SGM), which adaptively adjusts gradients according to the contribution of each modality. Extensive experiments on IEMOCAP, MOSI, and MOSEI confirm that PaSE achieves the superior performance and effectively alleviates modality competition.

Temporal reasoning is crucial for large language models (LLMs) to understand event concurrency and complex temporal interactions in natural language.Recent approaches rely on the LLM to infer temporal relations between events and largely overlook the inherent structural nature of temporal relationships.In this work, we propose ODL-TempLLM (Ontology-Guided and Description Logic-Constrained Temporal Reasoning with LLMs), a novel paradigm for temporal reasoning with LLMs that shifts focus from internal inference to the explicit modeling of temporal structure.ODL-TempLLM leverages ontology learning to explicitly construct structured temporal knowledge, employs a symbolic reasoner to deductively reason about temporal relations and uses logic-constrained retrieval augmentation to obtain relevant facts.Experiments results evaluated across three datasets via various LLM backbones show that our method outperforms state-of-the-art methods by 2.07-31.83F1 points and 1.00-30.73EM points, exhibiting strong generalization and highlighting the potential of explicit temporal reasoning.

Li Zheng, Yanyi Luo, Hao Fei, Yuzhe Ding, Yujie Huang, Fei Li, Chong Teng, Donghong Ji. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Li Zheng, Xin Zhang, Shuyi He, Fei Li, Chong Teng, Jiang-Ming Yang, Donghong Ji, Zhuang Li. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Multimodal information extraction (IE) tasks have attracted increasing attention because many studies have shown that multimodal information benefits text information extraction. However, existing multimodal IE datasets mainly focus on sentence-level image-facilitated IE in English text, and pay little attention to video-based multimodal IE and fine-grained visual grounding. Therefore, in order to promote the development of multimodal IE, we constructed a multimodal multilingual multitask dataset, named M$^{3}$3D, which has the following features: (1) It contains paired document-level text and video to enrich multimodal information; (2) It supports two widely-used languages, namely English and Chinese; (3) It includes more multimodal IE tasks such as entity recognition, entity chain extraction, relation extraction and visual grounding. In addition, our dataset introduces an unexplored theme, i.e., biography, enriching the domains of multimodal IE resources. To establish a benchmark for our dataset, we propose an innovative hierarchical multimodal IE model. This model effectively leverages and integrates multimodal information through a Denoised Feature Fusion Module (DFFM). Furthermore, in non-ideal scenarios, modal information is often incomplete. Thus, we designed a Missing Modality Construction Module (MMCM) to alleviate the issues caused by missing modalities. Our model achieved an average performance of 53.80% and 53.77% on four tasks in English and Chinese datasets, respectively, which set a reasonable standard for subsequent research. In addition, we conducted more analytical experiments to verify the effectiveness of our proposed module. We believe that our work can promote the development of the field of multimodal IE.

Emotion and intent joint understanding in multimodal conversations (MC-EIU) aims to model the semantic dependencies among multimodal conversations while inferring emotion and intent information. Despite making progress, existing methods overlook the differential contributions of modalities and rely on single-round interactions for emotion and intent recognition, resulting in suboptimal model understanding performance. To overcome these limitations, we propose MEIPro, a novel progressive interaction and adaptive weight fusion based multimodal joint understanding of emotion and intent framework. We first design a hierarchical denoising module to effectively remove noise and redundant information from multimodal data. Then, we propose an adaptive weight fusion mechanism that dynamically fuses multimodal features by taking the true classification probabilities of each modality as their respective contributions, thus enhancing the fusion process. Additionally, we present a progressive dual task interaction module to capture the deep seated interactions between emotion and intent through a step-by-step multi-round iteration. Experiments on the benchmark MC-EIU bilingual dataset demonstrate that our MEI-Pro framework significantly outperforms state-of-the-art baselines in both emotion and intent tasks. Specifically, on the English dataset, the F1-scores of the multimodal emotion and intent understanding tasks have increased by 6.12% and 7.25% respectively.

With the continuous emergence of various social media platforms frequently used in daily life, the multimodal meme understanding (MMU) task has been garnering increasing attention. MMU aims to explore and comprehend the meanings of memes from various perspectives by performing tasks such as metaphor recognition, sentiment analysis, intention detection, and offensiveness detection. Despite making progress, limitations persist due to the loss of fine-grained metaphorical visual clue and the neglect of multimodal text-image weak correlation. To overcome these limitations, we propose a multi-granular multimodal clue fusion model (MGMCF) to advance MMU. Firstly, we design an object-level semantic mining module to extract object-level image feature clues, achieving fine-grained feature clue extraction and enhancing the model's ability to capture metaphorical details and semantics. Secondly, we propose a brand-new global-local cross-modal interaction model to address the weak correlation between text and images. This model facilitates effective interaction between global multimodal contextual clues and local unimodal feature clues, strengthening their representations through a bidirectional cross-modal attention mechanism. Finally, we devise a dual-semantic guided training strategy to enhance the model's understanding and alignment of multimodal representations in the semantic space. Experiments conducted on the widely-used MET-MEME bilingual dataset demonstrate significant improvements over state-of-the-art baselines. Specifically, there is an 8.14% increase in precision for offensiveness detection task, and respective accuracy enhancements of 3.53%, 3.89%, and 3.52% for metaphor recognition, sentiment analysis, and intention detection tasks. These results, underpinned by in-depth analyses, underscore the effectiveness and potential of our approach for advancing MMU.

Xiaorui Wu, Xiaofeng Mao, Fei Li, Xin Zhang, Xuanhong Li, Chong Teng, Donghong Ji, Zhuang Li. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Li Zheng, Sihang Wang, Hao Fei, Zuquan Peng, Fei Li, Jianming Fu, Chong Teng, Donghong Ji. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Entity and relation extraction is a conventional task in the field of information extraction. Existing work primarily focuses on detecting specific relations between entities, often constrained to particular fields and lacking general applicability. In response, we propose a novel task: nominal compound relation extraction (NCRE), which concentrates on abstract and broadly applicable relation extraction between noun phrases. This task diverges significantly from traditional entity and relation extraction in two key respects. Firstly, our task involves general nominal compounds rather than named entities, which are longer and encompass a broader scope, presenting significant challenges for extraction. Secondly, relation extraction in NCRE demands an in-depth understanding of context to detect abstract relations. We manually annotate a high-quality Chinese dataset for the NCRE task and develop a model incorporating the rotary position-enhanced word pair (RoWP) detection schema. Experimental results demonstrate the efficiency of our RoWP model over previous baselines, while the suboptimal F1 scores indicate that NCRE remains a challenging task. Our code and data are available at https://github.com/yeecjc/NCRE.

Vision-Language Models (VLMs) generate discourse-level, multi-sentence visual descriptions, challenging text scene graph parsers built for single-sentence caption-to-graph mapping.Current approaches typically merge sentencelevel parsing outputs for discourse input, often missing phenomena like cross-sentence coreference, resulting in fragmented graphs and degraded downstream VLM task performance.We introduce a new task, Discourse-level text Scene Graph parsing (DiscoSG), and release DiscoSG-DS, a dataset of 400 expert-annotated and 8,430 synthesised multi-sentence captiongraph pairs.Each caption averages 9 sentences, and each graph contains at least 3× more triples than those in existing datasets.Fine-tuning GPT-4o on DiscoSG-DS yields over 40% higher SPICE metric than the best sentence-merging baseline.However, its high inference cost and licensing restrict opensource use.Smaller fine-tuned open-source models (e.g., Flan-T5) perform well on simpler graphs yet degrade on denser, more complex graphs.To bridge this gap, we introduce DiscoSG-Refiner, a lightweight open-source parser that drafts a seed graph and iteratively refines it with a novel learned graph-editing model, achieving 30% higher SPICE than the baseline while delivering 86× faster inference than GPT-4o.It generalises from simple to dense graphs, thereby consistently improving downstream VLM tasks, including discourselevel caption evaluation and hallucination detection, outperforming alternative open-source parsers.

Natural Language Inference (NLI) is a fundamental task in natural language processing.While NLI has developed many subdirections such as sentence-level NLI, documentlevel NLI and cross-lingual NLI, Cross-Document Cross-Lingual NLI (CDCL-NLI) remains largely unexplored.In this paper, we propose a novel paradigm: CDCL-NLI, which extends traditional NLI capabilities to multidocument, multilingual scenarios.To support this task, we construct a high-quality CDCL-NLI dataset including 25,410 instances and spanning 26 languages.To address the limitations of previous methods on CDCL-NLI task, we further propose an innovative method that integrates RST-enhanced graph fusion with interpretability-aware prediction.Our approach leverages RST (Rhetorical Structure Theory) within heterogeneous graph neural networks for cross-document context modeling, and employs a structure-aware semantic alignment based on lexical chains for crosslingual understanding.For NLI interpretability, we develop an EDU (Elementary Discourse Unit)-level attribution framework that produces extractive explanations.Extensive experiments demonstrate our approach's superior performance, achieving significant improvements over both conventional NLI models as well as large language models.Our work sheds light on the study of NLI and will bring research interest on cross-document cross-lingual context understanding, hallucination elimination and interpretability inference.Our code and dataset are available at CDCL- NLI-link.

Unsupervised sentence embedding learning excels through positive sample construction and instance-level contrastive learning (ICL). However, this approach can lead to over-compression and dimensional contamination from noisy data augmentation and unconstrained ICL processes. To mitigate these issues, we propose a novel enhancement method, MSSE, which incorporates an Information Compensation Mechanism (ICM) and a Dimensional-Level Contrastive Learning Mechanism (DCM). ICM, inspired by the information bottleneck principle, prevents excessive compression in representation learning. DCM constrains ICL and minimizes information leakage across dimensions. Experimental results show that MSSE surpasses existing competitive baselines across seven STS tasks, including unsupervised and few-shot scenarios. The source code is available at https://github.com/Hekang001/MSSE.

Coma is a prolonged state of unconsciousness in which a patient exhibits no response to external stimuli. The electroencephalogram (EEG) is a non-invasive exam that measures electrical brain activity through electrodes placed on the scalp, providing real-time insights into neural dynamics. EEG is essential for assessing coma depth, detecting neurological deterioration, and predicting patient outcomes. While deep learning techniques such as convolutional neural networks (CNNs) and long short-term memory (LSTM) networks have been applied to coma prognosis, state-of-the-art architectures like transformers remain largely unexplored in this context. To address this gap, we propose EEGTransformer, a transformer-based architecture designed for the prognostic assessment of comatose patients using EEG signals. EEGTransformer leverages the self-attention mechanism to capture long-range dependencies in EEG sequences while benefiting from multiple attention heads to enhance contextual representation. We evaluate the proposed approach on a real-world dataset comprising dozens of EEG recordings. The results show that EEGTransformer achieves a macro F1 score of 0.88, outperforming state-of-the-art deep learning models, including a CNN-LSTM model and another transformer-based approach. Furthermore, a visual analysis of the learned latent representations reveals a significantly improved class separability compared to existing methods. These findings highlight the potential of transformers for EEG-based biomedical analysis, representing a significant advancement in coma prognosis.

The rapid development of social platforms and multimedia technologies has led to the proliferation of multimodal rumors on these platforms. Therefore, automatic multimodal rumor detection has received extensive attention from researchers. Although many existing methods exhibit strong capabilities to identify multimodal rumors, they still have shortcomings in fully utilizing the information contained in images and performing fine-grained fusion of features from different modalities. In this paper, we propose an effective model via Image Semantic Enhancement and Hierarchical Fusion (ISEHF) for multimodal rumor detection. Specifically, ISEHF performs image semantic enhancement by employing a large vision-language model to obtain image captions, which is critical to fully utilizing the information contained in images. Moreover, we propose a hierarchical fusion module within the ISEHF model, which consists of a shallow fusion module and a deep fusion module, to generate richer features at different levels for fine-grained fusion of features from different modalities. Extensive experiments conducted on two public datasets demonstrate the superiority of our model in comparison with the state-of-the-art baselines.

Large language models (LLMs) frequently refuse to respond to pseudo-malicious instructions: semantically harmless input queries triggering unnecessary LLM refusals due to conservative safety alignment, significantly impairing user experience. Collecting such instructions is crucial for evaluating and mitigating over-refusals, but existing instruction curation methods, like manual creation or instruction rewriting, either lack scalability or fail to produce sufficiently diverse and effective refusal-inducing prompts. To address these limitations, we introduce EVOREFUSE, a prompt optimization approach that generates diverse pseudo-malicious instructions consistently eliciting confident refusals across LLMs. EVOREFUSE employs an evolutionary algorithm exploring the instruction space in more diverse directions than existing methods via mutation strategies and recombination, and iteratively evolves seed instructions to maximize evidence lower bound on LLM refusal probability. Using EVOREFUSE, we create two novel datasets: EVOREFUSE-TEST, a benchmark of 582 pseudo-malicious instructions that outperforms the next-best benchmark with 85.34% higher average refusal triggering rate across 9 LLMs without a safety-prior system prompt, 34.86% greater lexical diversity, and 40.03% improved LLM response confidence scores; and EVOREFUSE-ALIGN, which provides 3,000 pseudo-malicious instructions with responses for supervised and preference-based alignment training. With supervised fine-tuning on EVOREFUSE-ALIGN, LLAMA3.1-8B-INSTRUCT achieves up to 29.85% fewer over-refusals than models trained on the second-best alignment dataset, without compromising safety. Our analysis with EVOREFUSE-TEST reveals models trigger over-refusals by overly focusing on sensitive keywords while ignoring broader context. Our code and datasets are available at https://github.com/FishT0ucher/EVOREFUSE .

Biomedical event extraction, a cornerstone of information extraction, has increasingly attracted attention within the biomedical research community. Moreover, it is a highly complex task, which not only deals with many sub-tasks but also involves nested events. Currently, the research on biomedical event extraction, whether pipelined model or joint method, needs to be processed for each sub-task. The process of processing each sub-task one by one lead to the degradation of event extraction performance. In addition, most studies focus on extracting sentence-level events and ignore cross-sentence event information. To solve these problems, we simplify the process of event extraction, reduce the processing steps, and combine the two sub-tasks of relation extraction and argument combination as one sub-task. In addition, we consider document-level event extraction, which not only extracts cross-sentence events but also considers broader context information. Experimental results indicate that our novel approach outperforms prior studies. Additionally, the document-level event extraction model attains the top performance on the BioNLP’11 test data and achieves near-leading performance on the BioNLP’13 test data.

Abstract Character Coreference Resolution in Movie Screenplays (MovieCoref) is a newly emerging task for understanding complex movie plots and character relationships. This task poses greater challenges than traditional coreference resolution, due to the intricate narrative structures and character interactions unique to screenplays. In light of these challenges, we introduce a novel approach: a Structure-aware Triaffine Parser (STPar) for the MovieCoref task. STPar combines discourse and syntactic structures in the feature encoding process, enabling comprehensive analysis of ternary relationships and complex interactions. During the pairing process, STPar utilizes a triaffine scorer to consider high-order relations between candidate mention pairs, thus enhancing its ability to capture detailed narrative structures. In addition, STPar incorporates multi-task learning, encompassing singleton and span detection tasks, to further improve coreference resolution performance. Our evaluations on the MovieCoref dataset demonstrate that STPar significantly outperforms the best baseline by 7.4%, 21.5%, 7.1%, and 10.2% in F1 scores of B3, CEAFe, LEA, and CoNLL. Further analysis highlights the benefits of integrating structural discourse and syntactic information as well as the combined approaches of triaffine and multi-task learning.1

Dialogue disentanglement aims to detach the chronologically ordered utterances into several independent sessions. Conversation utterances are essentially organized and described by the underlying discourse, and thus dialogue disentanglement requires the full understanding and harnessing of the intrinsic discourse attribute. In this article, we propose enhancing dialogue disentanglement by taking full advantage of the dialogue discourse characteristics. First of all, in feature encoding stage , we construct the heterogeneous graph representations to model the various dialogue-specific discourse structural features, including the static speaker-role structures (i.e., speaker-utterance and speaker-mentioning structure) and the dynamic contextual structures (i.e., the utterance-distance and partial-replying structure). We then develop a structure-aware framework to integrate the rich structural features for better modeling the conversational semantic context. Second, in model learning stage , we perform optimization with a hierarchical ranking loss mechanism, which groups dialogue utterances into different discourse levels and carries training covering pairwise and session-wise levels hierarchically. Third, in inference stage , we devise an easy-first decoding algorithm, which performs utterance pairing under the easy-to-hard manner with a global context, breaking the constraint of traditional sequential decoding order. On two benchmark datasets, our overall system achieves new state-of-the-art performances on all evaluations. In-depth analyses further demonstrate the efficacy of each proposed idea and also reveal how our methods help advance the task. Our work has great potential to facilitate broader multi-party multi-thread dialogue applications.

The multimodal emotion-cause pair extraction (MECPE) task aims to detect the emotions, causes, and emotion-cause pairs from multimodal conversations. Existing methods for this task typically concatenate representations of each utterance from distinct modalities and then predict emotion-cause pairs directly. This approach struggles to effectively integrate multimodal features and capture the subtleties of emotion transitions, which are crucial for accurately identifying causes—thereby limiting overall performance. To address these challenges, we propose a novel model that captures holistic interaction and label constraint (HiLo) features for the MECPE task. HiLo facilitates cross-modality and cross-utterance feature interaction with various attention mechanisms, establishing a robust foundation for precise cause extraction. Notably, our model innovatively leverages emotion transition features as pivotal cues to enhance causal inference within conversations. The experimental results demonstrate the superior performance of HiLo, evidenced by an increase of more than 2% in the F1 score compared to existing benchmarks. Further analysis reveals that our approach adeptly utilizes multimodal and dialogue features, making a significant contribution to the field of emotion-cause analysis. Our code is publicly available at https://is.gd/MVdYmx .

Despite significant advancements in multi-label text classification, the ability of existing models to generalize to novel and seldom-encountered complex concepts, which are compositions of elementary ones, remains underexplored. This research addresses this gap. By creating unique data splits across three benchmarks, we assess the compositional generalization ability of existing multi-label text classification models. Our results show that these models often fail to generalize to compositional concepts encountered infrequently during training, leading to inferior performance on tests with these new combinations. To address this, we introduce a data augmentation method that leverages two innovative text generation models designed to enhance the classification models' capacity for compositional generalization. Our experiments show that this data augmentation approach significantly improves the compositional generalization capabilities of classification models on our benchmarks, with both generation models surpassing other text generation baselines. Our codes available at https://github.com/yychai74/LD-VAE.

Dialogue Aspect-based Sentiment Quadruple (DiaASQ) is a newly-emergent task aiming to extract the sentiment quadruple (i.e., targets, aspects, opinions, and sentiments) from conversations. While showing promising performance, the prior DiaASQ approach unfortunately falls prey to the key crux of DiaASQ, including insufficient modeling of discourse features, and lacking quadruple extraction, which hinders further task improvement. To this end, we introduce a novel framework that not only capitalizes on comprehensive discourse feature modeling, but also captures the intrinsic interaction for optimal quadruple extraction. On the one hand, drawing upon multiple discourse features, our approach constructs a token-level heterogeneous graph and enhances token interactions through a heterogeneous attention network. We further propose a novel triadic scorer, strengthening weak token relations within a quadruple, thereby enhancing the cohesion of the quadruple extraction. Experimental results on the DiaASQ benchmark showcase that our model significantly outperforms existing baselines across both English and Chinese datasets. Our code is available at https://bit.ly/3v27pqA.

With the proliferation of dialogic data across the Internet, the Dialogue Commonsense Multi-choice Question Answering (DC-MCQ) task has emerged as a response to the challenge of comprehending user queries and intentions. Although prevailing methodologies exhibit effectiveness in addressing single-choice questions, they encounter difficulties in handling multi-choice queries due to the heightened intricacy and informational density. In this paper, inspired by the human cognitive process of progressively excluding options, we propose a three-step Reverse Exclusion Graph-of-Thought (ReX-GoT) framework, including Option Exclusion, Error Analysis, and Combine Information. Specifically, our ReX-GoT mimics human reasoning by gradually excluding irrelevant options and learning the reasons for option errors to choose the optimal path of the GoT and ultimately infer the correct answer. By progressively integrating intricate clues, our method effectively reduces the difficulty of multi-choice reasoning and provides a novel solution for DC-MCQ. Extensive experiments on the CICERO and CICERO_v2 datasets validate the significant improvement of our approach on DC-MCQ task. On zero-shot setting, our model outperform the best baseline by 17.67% in terms of F1 score for the multi-choice task. Most strikingly, our GPT3.5-based ReX-GoT framework achieves a remarkable 39.44% increase in F1 score.

Structured Sentiment Analysis (SSA) was cast as a problem of bi-lexical dependency graph parsing by prior studies.Multiple formulations have been proposed to construct the graph, which share several intrinsic drawbacks: (1) The internal structures of spans are neglected, thus only the boundary tokens of spans are used for relation prediction and span recognition, thus hindering the model's expressiveness; (2) Long spans occupy a significant proportion in the SSA datasets, which further exacerbates the problem of internal structure neglect.In this paper, we treat the SSA task as a dependency parsing task on partially-observed dependency trees, regarding flat spans without determined tree annotations as latent subtrees to consider internal structures of spans.We propose a twostage parsing method and leverage TreeCRFs with a novel constrained inside algorithm to model latent structures explicitly, which also takes advantages of joint scoring graph arcs and headed spans for global optimization and inference.Results of extensive experiments on five benchmark datasets reveal that our method performs significantly better than all previous bi-lexical methods, achieving new state-of-theart.

Large Language Models (LLMs) are now being considered as judges of high efficiency to evaluate the quality of answers generated by candidate models. However, their judgments may be influenced by complex scenarios and inherent biases, raising concerns about their reliability. This study aims to bridge this gap by introducing four unexplored factors and examining the performance of LLMs as judges, namely answer quantity, inducing statements, judging strategy, and judging style. Additionally, we introduce a new dimension of question difficulty to provide a more comprehensive understanding of LLMs’ judgments across varying question intricacies. We employ ChatGPT, GPT-4, Gemini, and Claude-2 as judges and conduct experiments on Vicuna Benchmark and MT-bench. Our study reveals that LLMs’ judging abilities are susceptible to the influence of these four factors, and analyzing from the newly proposed dimension of question difficulty is highly necessary. We also provide valuable insights into optimizing LLMs’ performance as judges, enhancing their reliability and adaptability across diverse evaluation scenarios.

Existing cross-document event coreference resolution models, which either compute mention similarity directly or enhance mention representation by extracting event arguments (such as location, time, agent, and patient), lackingmthe ability to utilize document-level information. As a result, they struggle to capture long-distance dependencies. This shortcoming leads to their underwhelming performance in determining coreference for the events where their argument information relies on long-distance dependencies. In light of these limitations, we propose the construction of document-level Rhetorical Structure Theory (RST) trees and cross-document Lexical Chains to model the structural and semantic information of documents. Subsequently, cross-document heterogeneous graphs are constructed and GAT is utilized to learn the representations of events. Finally, a pair scorer calculates the similarity between each pair of events and co-referred events can be recognized using standard clustering algorithm. Additionally, as the existing cross-document event coreference datasets are limited to English, we have developed a large-scale Chinese cross-document event coreference dataset to fill this gap, which comprises 53,066 event mentions and 4,476 clusters. After applying our model on the English and Chinese datasets respectively, it outperforms all baselines by large margins.

Optical Coherence Tomography (OCT) is an advanced biomedical imaging technology that provides noninvasive, real-time, high-resolution images of tissues with high scattering properties. Widely employed in ophthalmology, OCT conducts diagnostic imaging of the anterior eye and retina, particularly in clinical studies evaluating its efficacy for various retinal diseases. Despite its significance, the multiple images generated by OCT pose a time-consuming challenge for ophthalmologists during the analysis process. In this paper, the pre-trained visual transformer is extended and trained with adversarial noise images. The proposed approach demonstrates the ability to effectively classify a broad spectrum of adversarial attacks while operating without prior knowledge of the attackers and without compromising classification performance. The results achieved on retinal OCT images, susceptible to adversarial attacks, exhibit a high accuracy of 98.1%.

In this study, we unveil an innovative framework for aspect-based sentiment analysis specifically designed for Urdu news headlines. Our approach uniquely combines the power of Graph Attention Networks (GAT) with the multilingual BERT model to navigate the linguistic intricacies of Urdu, a language often overlooked in computational sentiment analysis. Central to our work is the development of a first-of-its-kind, manually annotated dataset for Urdu, meticulously crafted to reflect its diverse linguistic attributes. This dataset serves as a critical tool in enhancing our model’s accuracy and comprehensiveness. By merging the contextual capabilities of BERT with the relational insights provided by GAT, we have created a model that excels in extracting and classifying sentiments from Urdu news content. Our methodology not only involves extensive preprocessing of data, including tokenization, POS tagging, and lemmatization using Stanza, but also sets a new benchmark in aspect extraction and sentiment analysis for Urdu. This pioneering research contributes significantly to narrowing the gap in NLP resources for low-resource languages and facilitate regional language processing, offering valuable insights for linguists, media analysts, and NLP practitioners.

This work investigates high-level classification techniques based on properties and measures of complex networks for the salivary detection of oral cancer from Attenuated Total Reflectance by Fourier Transform Infrared Spectroscopy (ATR-FTIR). Saliva biomarkers are alternative to surrogate other invasive samples in the early detection and monitoring of systemic diseases. Saliva also allows convenient and easy collection. ATR-FTIR is a sustainable, rapid and non-invasive platform able to contribute to the detection of several diseases. Traditional machine learning techniques have already been considered in the analysis of salivary ATR-FTIR data. However, such techniques are able to perform only low-level classification of the spectra data by considering physical features such as similarity, distance or distribution. On the other hand, high-level techniques are able to consider the semantic meaning of the input data by analyzing their structural and topological properties. In this paper, we investigate the hypothesis that the high-level classification of the ATR-FTIR spectra obtained via learning systems based on complex networks measures can achieve better predictive performance in comparison with low-level classification techniques widely adopted in the literature, such as linear discriminant analysis and support vector machines (SVM). Experiments conducted on real-world data confirmed our hypothesis. Our high-level classification techniques achieved 71% of accuracy and 81% of sensitivity in the detection of oral cancer after considering properties and structural patterns captured by the Clustering Coefficient network measure. Moreover, such results also outperformed those obtained by state-of-the-art classifiers like convolutional neural networks.

The field of Graph Neural Networks (GNNs) has garnered significant attention for leveraging Neural Networks to analyze graph data. However, the inherent limitations of GNNs, arising from their aggregation function and injectivity constraints, restrict their ability to distinguish various topological neighborhood structures accurately. Consequently, after aggregation, different underlying graph structures may not be distinctly identified, impeding the model’s representation capabilities. In this study, we introduce a novel GNN model that incorporates Self-Organizing Maps (SOM) to enhance the representation of node attributes and characterize the graph’s neighborhood structure. Leveraging SOM, we transform the node’s attribute space into a fixed-size grid, enabling the representation of neighborhood topology by overlaying activated regions, which allows the model to learn association rules based on the presence or absence of neighboring attributes. This process generates an injective graph embedding that uniquely captures different neighborhood structures. We evaluate the model on real-world datasets to assess its’ performance, analyzing how SOM parameters influence its behavior and effectiveness. Through extensive experimentation, we demonstrate that our model achieves competitive and robust results with well-established GNN models. Our approach showcases the potential of using SOM to overcome limitations in current GNN architectures, providing a more effective and nuanced representation of graph data.

Coreference resolution, an essential task in natural language processing, is particularly challenging in multi-modal scenarios where data comes in various forms and modalities. Despite advancements, limitations due to scarce labeled data and underleveraged unlabeled data persist. We address these issues with a self-adaptive fine-grained multi-modal data augmentation framework for semi-supervised MCR, focusing on enriching training data from labeled datasets and tapping into the untapped potential of unlabeled data. Regarding the former issue, we first leverage text coreference resolution datasets and diffusion models,to perform fine-grained text-to-image generation with aligned text entities and image bounding boxes. We then introduce a self-adaptive selection strategy, meticulously curating the augmented data to enhance the diversity and volume of the training set without compromising its quality. For the latter issue, we design a self-adaptive threshold strategy that dynamically adjusts the confidence threshold based on the model's learning status and performance, enabling effective utilization of valuable information from unlabeled data. Additionally, we incorporate a distance smoothing term, which smooths distances between positive and negative samples, enhancing discriminative power of the model?s feature representations and addressing noise and uncertainty in the unlabeled data. Our experiments on the widely-used CIN dataset show that our framework significantly outperforms state-of-the-art baselines by at least 9.57% on MUC F1 score and 4.92% on CoNLL F1 score. Remarkably, against weakly-supervised baselines, our framework achieves a staggering 22.24% enhancement in MUC F1 score. These results, underpinned by in-depth analyses, underscore the effectiveness and potential of our approach for advancing MCR tasks.

With the increasing popularity of live streaming, the interactions from viewers during a live streaming can provide more specific and constructive feedback for both the streamer and platform. In such scenario, the primary and most direct feedback method from the audience is through comments. Thus, mining these live streaming comments to unearth the intentions behind them and, in turn, aiding streamers to enhance their live streaming quality is significant for the well development of live streaming ecosystem. To this end, we introduce the MMLSCU dataset, containing 50,129 intention-annotated comments across multiple modalities (text, images, vi-deos, audio) from eight streaming domains. Using multimodal pretrained large model and drawing inspiration from the Chain of Thoughts (CoT) concept, we implement an end-to-end model to sequentially perform the following tasks: viewer comment intent detection ➛ intent cause mining ➛ viewer comment explanation ➛ streamer policy suggestion. We employ distinct branches for video and audio to process their respective modalities. After obtaining the video and audio representations, we conduct a multimodal fusion with the comment. This integrated data is then fed into the large language model to perform inference across the four tasks following the CoT framework. Experimental results indicate that our model outperforms three multimodal classification baselines on comment intent detection and streamer policy suggestion, and one multimodal generation baselines on intent cause mining and viewer comment explanation. Compared to the models using only text, our multimodal setting yields superior outcomes. Moreover, incorporating CoT allows our model to enhance comment interpretation and more precise suggestions for the streamers. Our proposed dataset and model will bring new research attention on multimodal live streaming comment understanding.

Empathetic response generation is a crucial task in natural language processing , enabling emotionally resonant machine–human interactions. In this paper, we introduce the InfRa ( In tegrating Discourse F eatures and R esponse A ssessment) model to address limitations in traditional methods for this task, such as the lack of deep dialogue comprehension and response control. InfRa integrates discourse features to augment structural dialogue understanding, with a novel edge pruning and mutual information learning module to further refine the representation. The model also employs a response evaluation module for dynamic optimization , ensuring emotional and semantic consistency between the generated response and its context . Our experiments demonstrate that InfRa outperforms existing baselines, reducing the Perplexity (PPL) score by approximately 9 points and excelling in all three fine-grained aspects of human evaluation. This research not only advances the development of empathetic chatbots but also provides valuable insights for broader text generation tasks.

The dependency syntactic structure is widely used in event extraction. However, the dependency structure reflecting syntactic features is essentially different from the event structure that reflects semantic features, leading to the performance degradation. In this article, we propose to use Event Trigger Structure for Event Extraction (ETSEE), which can compensate the inconsistency between two structures. First, we leverage the ACE2005 dataset as case study, and annotate three kinds of ETSs, that is, “light verb + trigger”, “preposition structures” and “tense + trigger”. Then we design a graph-based event extraction model that jointly identifies triggers and arguments, where the graph consists of both the dependency structure and ETSs. Experiments show that our model significantly outperforms the state-of-the-art methods. Through empirical analysis and manual observation, we find that the ETSs can bring the following benefits: (1) enriching trigger identification features by introducing structural event information; (2) enriching dependency structures with event semantic information; (3) enhancing the interactions between triggers and candidate arguments by shortening their distances in the dependency graph.

Aspect-based Sentiment Analysis (ABSA) comprises numerous subtasks including aspect term extraction (AE), opinion term extraction (OE), opinion pair extraction (PE), and triplet extraction (TE). Current research in Chinese ABSA primarily concentrates on aspect terms and sentiment polarity, with insufficient emphasis on opinion terms. This article aims to provide a viable solution for the unannotated Chinese ABSA subtasks such as OE, PE, and TE. First, we develop an English-Chinese parallel dataset for ABSA using a semi-automatic process involving machine translation and a word aligner. Second, we examine the efficacy of cross-lingual transfer methods. Third, we propose a plug-and-play transfer method based on sememe knowledge. Sememes are the language-independent smallest semantic units that encapsulate the components, commonalities, and attributes of things extracted from the real world, which can bridge the gap between English and Chinese. Experimental results show that our proposed method brings significant improvements for Chinese ABSA, and achieves a maximum increase of 8% on the F1 metric for model transfer and label transfer on OE, PE, and TE.

Financial sentiment analysis is a fine-grained sentiment analysis task that needs to predict the sentiment value toward a given target entity. Recently, dependency-based graph neural networks have been introduced for target-based sentiment analysis. However, financial sentiment analysis with implicit sentiment expression is more challenging than target-based explicit sentiment analysis, requiring a deep understanding of the complex association between the sentiment clue in context and the target entity. In previous work related to financial sentiment analysis, most methods focused on learning the simple word-to-word relations between the contextual words and the target entity based on the dependency tree of the sentence, ignoring the exploitation of span-boundary information and phrase-level syntactic knowledge with regard to the target entity. In this paper, we perform financial implicit sentiment analysis by taking phrases as basic semantic units and proposing a graph attention network ($PhraseGAT$) based on the constituent tree to leverage the phrase syntactic knowledge. To enhance the information flow between the nodes in the graph, we construct a heterogeneous graph based on the constituent tree and encode higher-order neighbor information. In addition, we introduce a multi-edge-type graph attention network ($MET$-$GAT$) to take full consideration of syntax and semantic interactions for the final prediction on the sentiment value of the target entity. Our proposed approach achieves 85.56% and 84.37% cosine similarity on public benchmark HEADLINE and MICROBLOG datasets, outperforms several strong baselines and achieves new state-of-the-art performance, verifying its effectiveness.

Currently, biomedical event extraction has received considerable attention in various fields, including natural language processing, bioinformatics, and computational biomedicine. This has led to the emergence of numerous machine learning and deep learning models that have been proposed and applied to tackle this complex task. While existing models typically adopt an extraction-based approach, which requires breaking down the extraction of biomedical events into multiple subtasks for sequential processing, making it prone to cascading errors. This paper presents a novel approach by constructing a biomedical event generation model based on the framework of the pre-trained language model T5. We employ a sequence-to-sequence generation paradigm to obtain events, the model utilizes constrained decoding algorithm to guide sequence generation, and a curriculum learning algorithm for efficient model learning. To demonstrate the effectiveness of our model, we evaluate it on two public benchmark datasets, Genia 2011 and Genia 2013. Our model achieves superior performance, illustrating the effectiveness of generative modeling of biomedical events.

Few-shot named entity recognition (NER) exploits limited annotated instances to identify named mentions. Effectively transferring the internal or external resources thus becomes the key to few-shot NER. While the existing prompt tuning methods have shown remarkable few-shot performances, they still fail to make full use of knowledge. In this work, we investigate the integration of rich knowledge to prompt tuning for stronger few-shot NER. We propose incorporating the deep prompt tuning framework with threefold knowledge (namelyTKDP), including the internal 1)context knowledgeand the external 2)label knowledge& 3)sememe knowledge. TKDP encodes the three feature sources and incorporates them into soft prompt embeddings, which are further injected into an existing pre-trained language model to facilitate predictions. On five benchmark datasets, the performance of our knowledge-enriched model was boosted by at most 11.53% F1 over the raw deep prompt method, and it significantly outperforms 9 strong-performing baseline systems in 5-/10-/20-shot settings, showing great potential in few-shot NER. Our TKDP framework can be broadly adapted to other few-shot tasks without much effort.

Graph Neural Networks (GNNs) have become a rapidly growing field, due to their ability to capture the relationship among data, instead of only learning from the attribute of the data. The core of any GNN is the graph embedding generation by message passing mechanisms. In this work we propose a new message passing technique based on the Particle Competition and Cooperation (PCC) model, originally developed for community detection in graphs. The proposed framework performs a transductive learning in the network and passes the learned information to the nodes, prior to the inductive learning performed by traditional GNN schemes. The new GNN presents attractive features which overcomes the over-smoothing problem of traditional GNNs and shows promising results in terms of classification accuracy, computational cost and learning with very small quantity of labeled data.

High-level classification are supervised learning techniques able to consider topological and structural features of the input data. Several high-level techniques have been proposed in the last years with different strategies, such as the pattern conformation technique which represents the input data as a network and perform classification by analyzing the variation of complex network measures. Such techniques have contributed in several tasks, but their contribution to the classification of sequential patterns has not been investigated yet. In this paper, we propose a high-level technique based on the complex network measures assortativity and average shortest path length to the analysis of electroencephalogram (EEG) data. To be specific, we consider two formulation of the problem of prognosis of patients in coma: binary and multi-class. The problem is very difficult and challenging as the data contains patients from different etiologies. Experimental results with nine other techniques including state-of-the-art ones like convolutional neural networks revealed that our high-level approach has the potential to improve (statistically) the predictive performance of those techniques, especially when considering the results with the assortativity measure for both binary and multi-class formulations. Moreover, this study paves a way in the adoption of complex network measures besides the extraction of features from EEG records, but also for the classification itself.

It has been a hot research topic to enable machines to understand human emotions in multimodal contexts under dialogue scenarios, which is tasked with multimodal emotion analysis in conversation (MM-ERC). MM-ERC has received consistent attention in recent years, where a diverse range of methods has been proposed for securing better task performance. Most existing works treat MM-ERC as a standard multimodal classification problem and perform multimodal feature disentanglement and fusion for maximizing feature utility. Yet after revisiting the characteristic of MM-ERC, we argue that both the feature multimodality and conversational contextualization should be properly modeled simultaneously during the feature disentanglement and fusion steps. In this work, we target further pushing the task performance by taking full consideration of the above insights. On the one hand, during feature disentanglement, based on the contrastive learning technique, we devise a Dual-level Disentanglement Mechanism (DDM) to decouple the features into both the modality space and utterance space. On the other hand, during the feature fusion stage, we propose a Contribution-aware Fusion Mechanism (CFM) and a Context Refusion Mechanism (CRM) for multimodal and context integration, respectively. They together schedule the proper integrations of multimodal and context features. Specifically, CFM explicitly manages the multimodal feature contributions dynamically, while CRM flexibly coordinates the introduction of dialogue contexts. On two public MM-ERC datasets, our system achieves new state-of-the-art performance consistently. Further analyses demonstrate that all our proposed mechanisms greatly facilitate the MM-ERC task by making full use of the multimodal and context features adaptively. Note that our proposed methods have the great potential to facilitate a broader range of other conversational multimodal tasks.

The spread of fake news is a significant social problem that can have disastrous impacts on various domains, such as politics and the economy.Therefore, detecting fake news has become a major concern.However, prior research has relied solely on news text to derive news representation, which is inadequate because different news items under the same publisher are interconnected.To address this limitation, we propose an innovative approach called the Publisher-oriented Multi-view Graph Model (PMGM) that leverages the context awareness of the publisher to detect fake news.Our approach enriches the news representation by incorporating publisher profiles and text style features extracted from the news.Specifically, we construct a multi-view graph that encodes various relationships between news items from the same publisher, such as news topics and occasions in which they were released.Furthermore, we leverage a multi-layer Graph Convolutional Network in conjunction with jumping knowledge networks to model the multi-view graph and produce a publisher-oriented contextualized representation of news.Experimental results on two widely used fake news datasets, namely LIAR and Weibo21, demonstrate the effectiveness of our approach.Specifically, the PMGM model outperforms the state-ofthe-art methods significantly.Overall, our proposed model unifies various heterogeneous features and information related to news based on a publisher-oriented approach, thereby offering a novel idea to enhance fake news detection.

The dissemination of disinformation on social media platforms has a significant impact on personal reputation and public trust.There has been a recent surge of interest in fake news detection.However, detecting low-resource fake news, particularly those pertaining to recent events that have not yet been disseminated by users and are typically in short text, remains challenging due to the lack of training data and prior knowledge.In this paper, we introduce a novel framework named the Heterogeneous Graph Augmented Prompt-based Tuning framework (HGAPT) that can leverage the metadata of news such as publisher and topic to construct a heterogeneous graph in same batch, which improve the performance of low-resource fake news detection.We have conducted extensive experiments on two low-resource fake news datasets that were collected from real-world sources.The results demonstrate that our proposed framework outperforms state-of-the-art methods, with superior detection performance at the zero-shot setting.

Anaphora resolution is a traditional task in the natural language processing community, defined as a cohesion phenomenon where one entity points back to a previous entity. Event discourse deixis (EDD) is a kind of more complex anaphora in which the anaphors refer to event descriptions such as sentences or clauses. Event discourse deixis resolution (EDDR) is able to help machines understand the richer linguistic and semantic information in the discourse. However, compared to anaphora resolution, EDDR has received relatively less research attention. In this work, we investigate the EDDR task by designing the corresponding dataset and model. First, we manually construct a high-quality Chinese corpus for EDDR, including 4,417 documents and 5,929 event chains that consist of event antecedents and anaphors. Second, we propose a deep neural network model for EDDR, which formulates the task into two subtasks, namely event anaphor recognition and event antecedent recognition. Our model is trained under the two subtasks jointly so that the EDDR task can be performed end to end. Besides our final model, we also build seven pipeline and joint models as baselines to build comprehensive benchmarks for follow-up research. Experimental results on our EDDR dataset show that our model outperforms all the baselines and achieves about 53%, 44%, 53%, and 63% F1s using standard anaphora resolution metrics such as CoNLL, MUC, B3, and Ceafe. The performances show that EDDR is a challenging task and worth researching in the future. Our dataset and model will be released to facilitate follow-up research.

So far, discontinuous named entity recognition (NER) has received increasing research attention and many related methods have surged such as hypergraph-based methods, span-based methods, and sequence-to-sequence (Seq2Seq) methods, etc. However, these methods more or less suffer from some problems such as decoding ambiguity and efficiency, which limit their performance. Recently, grid-tagging methods, which benefit from the flexible design of tagging systems and model architectures, have shown superiority to adapt for various information extraction tasks. In this paper, we follow the line of such methods and propose a competitive grid-tagging model for discontinuous NER. We call our model TOE because we incorporate two kinds of Tag-Oriented Enhancement mechanisms into a state-of-the-art (SOTA) grid-tagging model that casts the NER problem into word-word relationship prediction. First, we design a Tag Representation Embedding Module (TREM) to force our model to consider not only word-word relationships but also word-tag and tag-tag relationships. Concretely, we construct tag representations and embed them into TREM, so that TREM can treat tag and word representations as queries/keys/values and utilize self-attention to model their relationships. On the other hand, motivated by the Next-Neighboring-Word (NNW) and Tail-Head-Word (THW) tags in the SOTA model, we add two new symmetric tags, namely Previous-Neighboring-Word (PNW) and Head-Tail-Word (HTW), to model more fine-grained word-word relationships and alleviate error propagation from tag prediction. In the experiments of three benchmark datasets, namely CADEC, ShARe13 and ShARe14, our TOE model pushes the SOTA results by about 0.83%, 0.05% and 0.66% in F1, demonstrating its effectiveness.

Aspect-based sentiment triplet extraction (ASTE) aims at recognizing the joint triplets from texts, i.e., aspect terms, opinion expressions, and correlated sentiment polarities. As a newly proposed task, ASTE depicts the complete sentiment picture from different perspectives to better facilitate real-world applications. Unfortunately, several major challenges, such as the overlapping issue and long-distance dependency, have not been addressed effectively by the existing ASTE methods, which limits the performance of the task. In this article, we present an innovative encoder-decoder framework for end-to-end ASTE. Specifically, the ASTE task is first modeled as an unordered triplet set prediction problem, which is satisfied with a nonautoregressive decoding paradigm with a pointer network. Second, a novel high-order aggregation mechanism is proposed for fully integrating the underlying interactions between the overlapping structure of aspect and opinion terms. Third, a bipartite matching loss is introduced for facilitating the training of our nonautoregressive system. Experimental results on benchmark datasets show that our proposed framework significantly outperforms the state-of-the-art methods. Further analysis demonstrates the advantages of the proposed framework in handling the overlapping issue, relieving long-distance dependency and decoding efficiency.

Aspect-based sentiment analysis (ABSA) aims at automatically inferring the specific sentiment polarities toward certain aspects of products or services behind the social media texts or reviews, which has been a fundamental application to the real-world society. Since the early 2010s, ABSA has achieved extraordinarily high accuracy with various deep neural models. However, existing ABSA models with strong in-house performances may fail to generalize to some challenging cases where the contexts are variable, i.e., low robustness to real-world environments. In this study, we propose to enhance the ABSA robustness by systematically rethinking the bottlenecks from all possible angles, including model, data, and training. First, we strengthen the current best-robust syntax-aware models by further incorporating the rich external syntactic dependencies and the labels with aspect simultaneously with a universal-syntax graph convolutional network. In the corpus perspective, we propose to automatically induce high-quality synthetic training data with various types, allowing models to learn sufficient inductive bias for better robustness. Last, we based on the rich pseudo data perform adversarial training to enhance the resistance to the context perturbation and meanwhile employ contrastive learning to reinforce the representations of instances with contrastive sentiments. Extensive robustness evaluations are conducted. The results demonstrate that our enhanced syntax-aware model achieves better robustness performances than all the state-of-the-art baselines. By additionally incorporating our synthetic corpus, the robust testing results are pushed with around 10% accuracy, which are then further improved by installing the advanced training strategies. In-depth analyses are presented for revealing the factors influencing the ABSA robustness.

So far, named entity recognition (NER) has been involved with three major types, including flat, overlapped (aka. nested), and discontinuous NER, which have mostly been studied individually. Recently, a growing interest has been built for unified NER, tackling the above three jobs concurrently with one single model. Current best-performing methods mainly include span-based and sequence-to-sequence models, where unfortunately the former merely focus on boundary identification and the latter may suffer from exposure bias. In this work, we present a novel alternative by modeling the unified NER as word-word relation classification, namely W^2NER. The architecture resolves the kernel bottleneck of unified NER by effectively modeling the neighboring relations between entity words with Next-Neighboring-Word (NNW) and Tail-Head-Word-* (THW-*) relations. Based on the W^2NER scheme we develop a neural framework, in which the unified NER is modeled as a 2D grid of word pairs. We then propose multi-granularity 2D convolutions for better refining the grid representations. Finally, a co-predictor is used to sufficiently reason the word-word relations. We perform extensive experiments on 14 widely-used benchmark datasets for flat, overlapped, and discontinuous NER (8 English and 6 Chinese datasets), where our model beats all the current top-performing baselines, pushing the state-of-the-art performances of unified NER.

Unified opinion role labeling (ORL) aims to detect all possible opinion structures of 'opinion-holder-target' in one shot, given a text. The existing transition-based unified method, unfortunately, is subject to longer opinion terms and fails to solve the term overlap issue. Current top performance has been achieved by employing the span-based graph model, which however still suffers from both high model complexity and insufficient interaction among opinions and roles. In this work, we investigate a novel solution by revisiting the transition architecture, and augmenting it with a pointer network (PointNet). The framework parses out all opinion structures in linear-time complexity, meanwhile breaks through the limitation of any length of terms with PointNet. To achieve the explicit opinion-role interactions, we further propose a unified dependency-opinion graph (UDOG), co-modeling the syntactic dependency structure and the partial opinion-role structure. We then devise a relation-centered graph aggregator (RCGA) to encode the multi-relational UDOG, where the resulting high-order representations are used to promote the predictions in the vanilla transition system. Our model achieves new state-of-the-art results on the MPQA benchmark. Analyses further demonstrate the superiority of our methods on both efficacy and efficiency.

The state-of-the-art model for structured sentiment analysis casts the task as a dependency parsing problem, which has some limitations:(1) The label proportions for span prediction and span relation prediction are imbalanced.(2) The span lengths of sentiment tuple components may be very large in this task, which will further exacerbates the imbalance problem.(3) Two nodes in a dependency graph cannot have multiple arcs, therefore some overlapped sentiment tuples cannot be recognized.In this work, we propose nichetargeting solutions for these issues.First, we introduce a novel labeling strategy, which contains two sets of token pair labels, namely essential label set and whole label set.The essential label set consists of the basic labels for this task, which are relatively balanced and applied in the prediction layer.The whole label set includes rich labels to help our model capture various token relations, which are applied in the hidden layer to softly influence our model.Moreover, we also propose an effective model to well collaborate with our labeling strategy, which is equipped with the graph attention networks to iteratively refine token representations, and the adaptive multi-label classifier to dynamically predict multiple relations between token pairs.We perform extensive experiments on 5 benchmark datasets in four languages.Experimental results show that our model outperforms previous SOTA models by a large margin. 1

Event extraction (EE) is an essential task of information extraction, which aims to extract structured event information from unstructured text. Most prior work focuses on extracting flat events while neglecting overlapped or nested ones. A few models for overlapped and nested EE includes several successive stages to extract event triggers and arguments,which suffer from error propagation. Therefore, we design a simple yet effective tagging scheme and model to formulate EE as word-word relation recognition, called OneEE. The relations between trigger or argument words are simultaneously recognized in one stage with parallel grid tagging, thus yielding a very fast event extraction speed. The model is equipped with an adaptive event fusion module to generate event-aware representations and a distance-aware predictor to integrate relative distance information for word-word relation recognition, which are empirically demonstrated to be effective mechanisms. Experiments on 3 overlapped and nested EE benchmarks, namely FewFC, Genia11, and Genia13, show that OneEE achieves the state-of-the-art (SOTA) results. Moreover, the inference speed of OneEE is faster than those of baselines in the same condition, and can be further substantially improved since it supports parallel inference.

Emotion cause pair extraction (ECPE), as one of the derived subtasks of emotion cause analysis (ECA), shares rich inter-related features with emotion extraction (EE) and cause extraction (CE). Therefore EE and CE are frequently utilized as auxiliary tasks for better feature learning, modeled via multi-task learning (MTL) framework by prior works to achieve state-of-the-art (SoTA) ECPE results. However, existing MTL-based methods either fail to simultaneously model the specific features and the interactive feature in between, or suffer from the inconsistency of label prediction. In this work, we consider addressing the above challenges for improving ECPE by performing two alignment mechanisms with a novel Aˆ2Net model. We first propose a feature-task alignment to explicitly model the specific emotion-&cause-specific features and the shared interactive feature. Besides, an inter-task alignment is implemented, in which the label distance between the ECPE and the combinations of EE&CE are learned to be narrowed for better label consistency. Evaluations of benchmarks show that our methods outperform current best-performing systems on all ECA subtasks. Further analysis proves the importance of our proposed alignment mechanisms for the task.

Relation Extraction (RE) is a fundamental task of information extraction, which has attracted a large amount of research attention.Previous studies focus on extracting the relations within a sentence or document, while currently researchers begin to explore cross-document RE.However, current cross-document RE methods directly utilize text snippets surrounding the target entities in multiple given documents, which brings considerable noisy and non-relevant sentences.Moreover, they utilize all the text paths in a document bag in a coarse-grained way, without considering the connections between these text paths.In this paper, we aim to address both of these shortages and push the stateof-the-art for cross-document RE.First, we focus on input construction for our RE model and propose an entity-based document-context filter to retain useful information in the given documents by using the bridge entities in the text paths.Second, we propose a cross-document RE model based on cross-path entity relation attention, which allows the entity relations across text paths to interact with each other.We compare our cross-document RE method with the state-of-the-art methods in the dataset CodRED.Our method outperforms them by at least 10% in F1, thus demonstrating its effectiveness.

So far, aspect-based sentiment analysis (ABSA) has involved with total seven subtasks, in which, however the interactions among them have been left unexplored sufficiently. This work presents a novel multiplex cascade framework for unified ABSA and maintaining such interactions. First, we model total seven subtasks as a hierarchical dependency in the easy-to-hard order, based on which we then propose a multiplex decoding mechanism, transferring the sentiment layouts and clues in lower tasks to upper ones. The multiplex strategy enables highly-efficient subtask interflows and avoids repetitive training; meanwhile it sufficiently utilizes the existing data without requiring any further annotation. Further, based on the characteristics of aspect-opinion term extraction and pairing, we enhance our multiplex framework by integrating POS tag and syntactic dependency information for term boundary and pairing identification. The proposed Syntax-aware Multiplex (SyMux) framework enhances the ABSA performances on 28 subtasks (7×4 datasets) with big margins.

Recent research attention for relation extraction has been paid to the dialogue scenario, i.e., dialogue-level relation extraction (DiaRE). Existing DiaRE methods either simply concatenate the utterances in a dialogue into a long piece of text, or employ naive words, sentences or entities to build dialogue graphs, while the structural characteristics in dialogues have not been fully utilized. In this work, we investigate a novel dialogue-level mixed dependency graph (D2G) and an argument reasoning graph (ARG) for DiaRE with a global relation reasoning mechanism. First, we model the entire dialogue into a unified and coherent D2G by explicitly integrating both syntactic and discourse structures, which enables richer semantic and feature learning for relation extraction. Second, we stack an ARG graph on top of D2G to further focus on argument inter-dependency learning and argument representation refinement, for sufficient argument relation inference. In our global reasoning framework, D2G and ARG work collaboratively, iteratively performing lexical, syntactic and semantic information exchange and representation learning over the entire dialogue context. On two DiaRE benchmarks, our framework shows considerable improvements over the current state-of-the-art baselines. Further analyses show that the model effectively solves the long-range dependence issue, and meanwhile gives explainable predictions.

Conversational semantic role labeling (CSRL) is a newly proposed task that uncovers the shallow semantic structures in a dialogue text. Unfortunately several important characteristics of the CSRL task have been overlooked by the existing works, such as the structural information integration, near-neighbor influence. In this work, we investigate the integration of a latent graph for CSRL. We propose to automatically induce a predicate-oriented latent graph (POLar) with a predicate-centered gaussian mechanism, by which the nearer and informative words to the predicate will be allocated with more attention. The POLar structure is then dynamically pruned and refined so as to best fit the task need. We additionally introduce an effective dialogue-level pre-trained language model, CoDiaBERT, for better supporting multiple utterance sentences and handling the speaker coreference issue in CSRL. Our system outperforms best-performing baselines on three benchmark CSRL datasets with big margins, especially achieving over 4% F1 score improvements on the cross-utterance argument detection. Further analyses are presented to better understand the effectiveness of our proposed methods.

Fine-grained sentiment classification (FGSC) task and fine-grained controllable text generation (FGSG) task are two representative applications of sentiment analysis, two of which together can actually form an inverse task prediction, i.e., the former aims to infer the fine-grained sentiment polarities given a text piece, while the latter generates text content that describes the input fine-grained opinions. Most of the existing work solves the FGSC and the FGSG tasks in isolation, while ignoring the complementary benefits in between. This paper combines FGSC and FGSG as a joint dual learning system, encouraging them to learn the advantages from each other. Based on the dual learning framework, we further propose decoupling the feature representations in two tasks into fine-grained aspect-oriented opinion variables and content variables respectively, by performing mutual disentanglement learning upon them. We also propose to transform the difficult "data-to-text'' generation fashion widely used in FGSG into an easier text-to-text generation fashion by creating surrogate natural language text as the model inputs. Experimental results on 7 sentiment analysis benchmarks including both the document-level and sentence-level datasets show that our method significantly outperforms the current strong-performing baselines on both the FGSC and FGSG tasks. Automatic and human evaluations demonstrate that our FGSG model successfully generates fluent, diverse and rich content conditioned on fine-grained sentiments.

Joint aspect category sentiment analysis (ACSA) and rating prediction (RP) is a newly proposed task (namely ASAP) that integrates the characteristics of both fine-grained and coarse-grained sentiment analysis. However, the prior joint models for the ASAP task only consider the shallow interaction between the two granularities. In this work, we gain the inspiration from human intuition, presenting an innovative from-fine-to-coarse reasoning framework for better joint task performance. Our system advances mainly in three aspects. First, we additionally make use of the category label text features, co-encoding them with the input document texts, allowing to accurately capture the key clues of each category. Second, we build a fine-to-coarse hierarchical label graph, modeling the aspect categories and the overall rating as a hierarchical structure for full interaction of the two granularities. Third, we propose to perform global iterative reasoning with a cross-collaboration between the hierarchical label graph and the context graphs, enabling sufficient communication between categories and review contexts. Based on the ASAP dataset, experimental results demonstrate that our proposed framework outperforms state-of-the-art baselines by large margins. Further in-depth analyses prove that our method is effective on addressing both the unbalanced data distribution and the long-text issue.

Biomedicalevent extraction is an essential task in the biomedical research. Existing models suffer from the issue of low recall due to the large proportion of unrecognized events and inflexible event argument combination. To address this issue, we propose an end-to-end multi-task approach for biomedical event extraction. Our model is able to achieve balanced precision and recall with several nichetargeting designs. First, neural encoders with rich lexical and syntactic features are used and shared by multiple subtasks such as event trigger recognition and argument relation extraction, in order to enhance the generalizability of the model. Second, a novel auxiliary subtask is added to identify the proteins that participate in the events, which helps decreasing the challenge of mining event-related proteins from the large candidate space. Third, event argument combination is performed using a strong neural network rather than inflexible rules or templates, to further increase the recall, especially for complex nested events. To demonstrate the effectiveness of our model, we evaluate it on two widely-used biomedical event extraction datasets used in the BioNLP 2011 and 2013 shared tasks. Our model achieves the state-of-the-art results (63.15% and 55.67% in F1 score) by significantly improving the recalls (compared withDeepEvnetMine$_{SciBERT}$, 4.65% and 5.0%) on the two datasets. Further experiments and analyses show the effectiveness of our proposed features and modules in the model.

Aspect-based sentiment information extraction has attracted increasing attention in the research community of natural language processing. Various methods, such as sequence tagging, sequence-to-sequence generation and span-based extraction, have been proposed, which own different advantages and disadvantages. In this article, we revisit the span-based method for aspect-sentiment-opinion triplet extraction, by designing and analyzing a simple yet effective Span-based Model, called SMTFASTE. Our model leverages a tidily three-layer architecture, including a BERT-based encoding layer, a span representation layer and an aspect-sentiment-opinion prediction layer. In the experiments of two widely-used benchmarks (ASTE-V2 and ASOTE-V2), we find that our model outperforms a number of complicated state-of-the-art models in most evaluation metrics. Therefore, we conduct detailed analyses for our model, such as ablation studies of the core components of our model and the benefit of explicitly using context information, and obtain some insightful findings and conclusion. Through this study, we show that a simple span-based model is able to achieve competitive results without much feature and architecture engineering. Our model is easy to follow and we have opened our code to facilitate related research.

Attention has been shown highly effective for modeling sequences, capturing the more informative parts in learning a deep representation. However, recent studies show that the attention values do not always coincide with intuition in tasks, such as machine translation and sentiment classification. In this study, we consider using deep reinforcement learning to automatically optimize attention distribution during the minimization of end task training losses. With more sufficient environment states, iterative actions are taken to adjust attention weights so that more informative words receive more attention automatically. Results on different tasks and different attention networks demonstrate that our model is of great effectiveness in improving the end task performances, yielding more reasonable attention distribution. The more in-depth analysis further reveals that our retrofitting method can help to bring explainability for baseline attention.

A majority of research interests in irregular (e.g., nested or discontinuous) named entity recognition (NER) have been paid on nested entities, while discontinuous entities received limited attention. Existing work for discontinuous NER, however, either suffers from decoding ambiguity or predicting using token-level local features. In this work, we present an innovative model for discontinuous NER based on pointer networks, where the pointer simultaneously decides whether a token at each decoding frame constitutes an entity mention and where the next constituent token is. Our model has three major merits compared with previous work: (1) The pointer mechanism is memory-augmented, which enhances the mention boundary detection and interactions between the current decision and prior recognized mentions. (2) The encoder-decoder architecture can linearize the complexity of structure prediction, and thus reduce search costs. (3) The model makes every decision using global information, i.e., by consulting all the input, encoder and previous decoder output in a global view. Experimental results on the CADEC and ShARe13 datasets show that our model outperforms flat and hypergraph models as well as a state-of-the-art transition-based model for discontinuous NER. Further in-depth analysis demonstrates that our model performs well in recognizing various entities including flat, overlapping and discontinuous ones. More crucially, our model is effective on boundary detection, which is the kernel source to NER.

Currently the unified semantic role labeling (SRL) that achieves predicate identification and argument role labeling in an end-to-end manner has received growing interests. Recent works show that leveraging the syntax knowledge significantly enhances the SRL performances. In this paper, we investigate a novel unified SRL framework based on the sequence-to-sequence architecture with double enhancement in both the encoder and decoder sides. In the encoder side, we propose a novel label-aware graph convolutional network (LA-GCN) to encode both the syntactic dependent arcs and labels into BERT-based word representations. In the decoder side, we creatively design a pointer-network-based model for detecting predicates, arguments and roles jointly. Our pointer-net decoder is able to make decisions by consulting all the input elements in a global view, and meanwhile it is syntactic-aware by incorporating the syntax information from LA-GCN. Besides, a high-order interacted attention is introduced into the decoder for leveraging previously recognized triplets to help the current decision. Empirical experiments show that our framework significantly outperforms all existing graph-based methods on the CoNLL09 and Universal Proposition Bank datasets. In-depth analysis demonstrates that our model can effectively capture the correlations between syntactic and SRL structures.

End-to-end semantic role labeling (SRL) has been received increasing interest. It performs the two subtasks of SRL: predicate identification and argument role labeling, jointly. Recent work is mostly focused on graph-based neural models, while the transition-based framework with neural networks which has been widely used in a number of closely-related tasks, has not been studied for the joint task yet. In this paper, we present the first work of transition-based neural models for end-to-end SRL. Our transition model incrementally discovers all sentential predicates as well as their arguments by a set of transition actions. The actions of the two subtasks are executed mutually for full interactions. Besides, we suggest high-order compositions to extract non-local features, which can enhance the proposed transition model further. Experimental results on CoNLL09 and Universal Proposition Bank show that our final model can produce state-of-the-art performance, and meanwhile keeps highly efficient in decoding. We also conduct detailed experimental analysis for a deep understanding of our proposed model.

Fei Li, ZhiChao Lin, Meishan Zhang, Donghong Ji. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

In this paper, we propose to enhance the pair-wise aspect and opinion terms extraction (PAOTE) task by incorporating rich syntactic knowledge. We first build a syntax fusion encoder for encoding syntactic features, including a label-aware graph convolutional network (LAGCN) for modeling the dependency edges and labels, as well as the POS tags unifiedly, and a local-attention module encoding POS tags for better term boundary detection. During pairing, we then adopt Biaffine and Triaffine scoring for high-order aspect-opinion term pairing, in the meantime re-harnessing the syntax-enriched representations in LAGCN for syntactic-aware scoring. Experimental results on four benchmark datasets demonstrate that our model outperforms current state-of-the-art baselines, meanwhile yielding explainable predictions with syntactic knowledge.

Existing works for document-level sentiment classification task treat the review document as an overall text unit, performing feature extraction with various sophisticated model architectures. In this paper, we draw inspiration from fine-grained sentiment analysis, proposing to first learn the latent target-opinion distribution behind the documents, and then leverage such fine-grained prior knowledge into the classification process. We model the latent target-opinion distribution as hierarchical variables, where global-level variable captures the overall target and opinion, and local-level variables retrieve the detailed opinion clues at the word level. The proposed method consists of two main parts: a variational module and a classification module. We employ the conditional variational autoencoder to make reconstructions of the document, during which the user and product information can be integrated. In the classification module, we build a hierarchical model based on Transformer encoders, where the local-level and global-level prior distribution representations induced from the variational module are injected into the word-level and sentence-level Transformers, respectively. Experimental results on benchmark datasets show that the proposed method significantly outperforms strong baselines, achieving the state-of-the-art performance. Further analysis shows that our model is capable of capturing the latent fine-grained target and opinion prior information, which is highly effective for improving the task performance.

Biomedical information extraction (BioIE) is an important task. The aim is to analyze biomedical texts and extract structured information such as named entities and semantic relations between them. In recent years, pre-trained language models have largely improved the performance of BioIE. However, they neglect to incorporate external structural knowledge, which can provide rich factual information to support the underlying understanding and reasoning for biomedical information extraction. In this paper, we first evaluate current extraction methods, including vanilla neural networks, general language models and pre-trained contextualized language models on biomedical information extraction tasks, including named entity recognition, relation extraction and event extraction. We then propose to enrich a contextualized language model by integrating a large scale of biomedical knowledge graphs (namely, BioKGLM). In order to effectively encode knowledge, we explore a three-stage training procedure and introduce different fusion strategies to facilitate knowledge injection. Experimental results on multiple tasks show that BioKGLM consistently outperforms state-of-the-art extraction models. A further analysis proves that BioKGLM can capture the underlying relations between biomedical knowledge concepts, which are crucial for BioIE.

MOTIVATION: Entity relation extraction is one of the fundamental tasks in biomedical text mining, which is usually solved by the models from natural language processing. Compared with traditional pipeline methods, joint methods can avoid the error propagation from entity to relation, giving better performances. However, the existing joint models are built upon sequential scheme, and fail to detect overlapping entity and relation, which are ubiquitous in biomedical texts. The main reason is that sequential models have relatively weaker power in capturing long-range dependencies, which results in lower performance in encoding longer sentences. In this article, we propose a novel span-graph neural model for jointly extracting overlapping entity relation in biomedical texts. Our model treats the task as relation triplets prediction, and builds the entity-graph by enumerating possible candidate entity spans. The proposed model captures the relationship between the correlated entities via a span scorer and a relation scorer, respectively, and finally outputs all valid relational triplets. RESULTS: Experimental results on two biomedical entity relation extraction tasks, including drug-drug interaction detection and protein-protein interaction detection, show that the proposed method outperforms previous models by a substantial margin, demonstrating the effectiveness of span-graph-based method for overlapping relation extraction in biomedical texts. Further in-depth analysis proves that our model is more effective in capturing the long-range dependencies for relation extraction compared with the sequential models. AVAILABILITY AND IMPLEMENTATION: Related codes are made publicly available at http://github.com/Baxelyne/SpanBioER.

Semantic role labeling (SRL) is one of the important tasks in natural language processing. Current end-to-end SRL, compared with traditional pipeline SRL, has achieved competitive performance via graph-based neural models. However, these are all first-order methods, where decisions for detecting predicate-argument pairs are made in isolation with local short-term features, thus being error-prone. Besides, they may potentially suffer from long-distance dependency, and perform poorly on the data-scarce scenario. In this paper, we explore a second-order end-to-end SRL model that considers simultaneously two pairs of predicate-argument when making scoring. We next propose a structural refinement mechanism to further model higher-order interactions at a global scope. After iterations of refinement, the underlying token representations are refined to be more informative. Experimental results show that our model significantly outperforms the state-of-the-art baselines on multiple benchmarks (i.e., CoNLL08, CoNLL09 and CoNLL12). Further analysis demonstrates that our framework is able to learn latent SRL constraints, being more robust on data-scarce settings, meanwhile relieving the long-range dependency issue.

Pair-wise aspect and opinion terms extraction (PAOTE), aiming at detecting the pair of the correlated aspect and opinion terms jointly, recently has drawn increasing research attention in the community of sentiment analysis and opinion mining. Recent works largely employ joint methods for the task, while they do not sufficiently incorporate the external syntactic knowledge, such as the dependency edges and labels. Besides, these methods fail to capture the underlying shared interactions among the overlapping aspect-opinion pair structures. In this paper we address the above issues for better PAOTE. Specifically, we present a span graph-based model for joint PAOTE, based on which we propose a edge-enhanced syntactic graph convolutional network (ESGCN) to simultaneously encode the syntactic dependency edges and the corresponding labels, for enhancing the extraction and pairing of aspect and opinion terms. During relational pairing, we employ a Biaffine and a Triaffine scorer respectively for high-order scoring, fully exploring the mutual interactions among different aspect-opinion pairs. Experimental results on four benchmark datasets show that our proposed method significantly outperforms all the baseline systems, giving the state-of-the-art performances. Further analysis proves the effectiveness of our framework on modeling the syntactic edge and label knowledge, as well as the advance of the high-order pairing mechanism for the task.

Relation extraction from dialogue text is an innovative task in natural language processing. In addition to the general characteristics of general relation extraction from news or scientific publication text, the task is of certain special features. For example, the context in dialogues frequently switches between speakers, and there exist rich pronoun anaphora in the dialogue text. Thus, it is important for the model to be aware of such features to improve the performance. Taking these factors together, we propose an end to-end neural model for dialogue-based relation extraction, which includes four modules to handle the problems existing in the task from different aspects: (1) the word-relation attention to model a natural intuition that different words contribute differently for the identification of different relations; (2) the graph reasoning to consider the global context information in the dialogue that contains many inter-sentence relations; (3) the speaker embeddings to incorporate speaker information into our model; (4) the speaker coreference to associate pronouns with speakers and enrich the information of graph reasoning. Our model was evaluated on a recently-proposed dataset for dialogue-based relation extraction, and achieved the state of the-art performance. We show that our proposed modules are effective through ablation studies. Our work can be a competitive benchmark for the study of dialogue based relation extraction.

Identifying multiple emotions in a sentence is an important research topic. Existing methods usually model the problem as multi-label classification task. However, previous methods have two issues, limiting the performance of the task. First, these models do not consider prior emotion distribution in a sentence. Second, they fail to effectively capture the context information closely related to the corresponding emotion. In this paper, we propose a Latent Emotion Memory network (LEM) for multi-label emotion classification. The proposed model can learn the latent emotion distribution without external knowledge, and can effectively leverage it into the classification network. Experimental results on two benchmark datasets show that the proposed model outperforms strong baselines, achieving the state-of-the-art performance.

Many efforts of research are devoted to semantic role labeling (SRL) which is crucial for natural language understanding.Supervised approaches have achieved impressing performances when large-scale corpora are available for resource-rich languages such as English.While for the low-resource languages with no annotated SRL dataset, it is still challenging to obtain competitive performances.Cross-lingual SRL is one promising way to address the problem, which has achieved great advances with the help of model transferring and annotation projection.In this paper, we propose a novel alternative based on corpus translation, constructing high-quality training datasets for the target languages from the source gold-standard SRL annotations.Experimental results on Universal Proposition Bank show that the translation-based method is highly effective, and the automatic pseudo datasets can improve the target-language SRL performances significantly.

Aspect-based sentiment classification is a popular task aimed at identifying the corresponding emotion of a specific aspect.One sentence may contain various sentiments for different aspects.Many sophisticated methods such as attention mechanism and Convolutional Neural Networks (CNN) have been widely employed for handling this challenge.Recently, semantic dependency tree implemented by Graph Convolutional Networks (GCN) is introduced to describe the inner connection between aspects and the associated emotion words.But the improvement is limited due to the noise and instability of dependency trees.To this end, we propose a dependency graph enhanced dual-transformer network (named DGEDT) by jointly considering the flat representations learnt from Transformer and graphbased representations learnt from the corresponding dependency graph in an iterative interaction manner.Specifically, a dualtransformer structure is devised in DGEDT to support mutual reinforcement between the flat representation learning and graph-based representation learning.The idea is to allow the dependency graph to guide the representation learning of the transformer encoder and vice versa.The results on five datasets demonstrate that the proposed DGEDT outperforms all state-of-the-art alternatives with a large margin.

Meaning Representations (AMRs) capture sentence-level semantics structural representations to broad-coverage natural sentences.We investigate parsing AMR with explicit dependency structures and interpretable latent structures.We generate the latent soft structure without additional annotations, and fuse both dependency and latent structure via an extended graph neural networks.The fused structural information helps our experiments results to achieve the best reported results on both AMR 2.0 (77.5% Smatch F1 on LDC2017T10) and AMR 1.0 (71.8% Smatch F1 on LDC2014T12).

In this paper, we target improving the joint dialogue act recognition (DAR) and sentiment classification (SC) tasks by fully modeling the local contexts of utterances.First, we employ the dynamic convolution network (DCN) as the utterance encoder to capture the dialogue contexts.Further, we propose a novel context-aware dynamic convolution network (CDCN) to better leverage the local contexts when dynamically generating kernels.We extended our frameworks into bi-channel version (i.e., BDCN and BCDCN) under multi-task learning to achieve the joint DAR and SC.Two channels can learn their own feature representations for DAR and SC, respectively, but with latent interaction.Besides, we suggest enhancing the tasks by employing the DiaBERT language model.Our frameworks 1 obtain state-of-the-art performances against all baselines on two benchmark datasets, demonstrating the importance of modeling the local contexts.

Emotion detection in conversations (EDC) is to detect the emotion for each utterance in conversations that have multiple speakers. Different from the traditional non-conversational emotion detection, the model for EDC should be context-sensitive (e.g., understanding the whole conversation rather than one utterance) and speaker-sensitive (e.g., understanding which utterance belongs to which speaker). In this paper, we propose a transformer-based context- and speaker-sensitive model for EDC, namely HiTrans, which consists of two hierarchical transformers. We utilize BERT as the low-level transformer to generate local utterance representations, and feed them into another high-level transformer so that utterance representations could be sensitive to the global context of the conversation. Moreover, we exploit an auxiliary task to make our model speaker-sensitive, called pairwise utterance speaker verification (PUSV), which aims to classify whether two utterances belong to the same speaker. We evaluate our model on three benchmark datasets, namely EmoryNLP, MELD and IEMOCAP. Results show that our model outperforms previous state-of-the-art models.

In this paper, we present Chinese lexical fusion recognition, a new task which could be regarded as one kind of coreference recognition.First, we introduce the task in detail, showing the relationship with coreference recognition and differences from the existing tasks.Second, we propose an end-to-end model for the task, handling mentions as well as coreference relationship jointly.The model exploits the state-of-the-art contextualized BERT representations as the encoder, and is further enhanced with the sememe knowledge from HowNet by graph attention networks.We manually annotate a benchmark dataset for the task and then conduct experiments on it.Results demonstrate that our final model is effective and competitive for the task.Detailed analysis is offered for comprehensively understanding the new task and our proposed model.

We consider retrofitting structure-aware Transformer language model for facilitating end tasks by proposing to exploit syntactic distance to encode both the phrasal constituency and dependency connection into the language model.A middle-layer structural learning strategy is leveraged for structure integration, accomplished with main semantic task training under multi-task learning scheme.Experimental results show that the retrofitted structure-aware Transformer language model achieves improved perplexity, meanwhile inducing accurate syntactic phrases.By performing structure-aware fine-tuning, our model achieves significant improvements for both semantic-and syntactic-dependent tasks.

In recent years, the joint model of entity recognition (ER) and relation extraction (RE) has attracted more and more attention in the healthcare and medical domains. However, there are some problems with the prior work. The joint model cannot extract all the relations for a specific entity, and the majority of joint models heavily rely on complex artificial features or professional natural language processing (NLP) tools. In this article, we construct a novel joint model that can simultaneously extract all medical entities and relations from medicine Chinese instructions. Moreover, the self-attention mechanism is introduced to the joint model to learn word intra-sentence dependencies. The proposed model is evaluated using a medicine Chinese instruction dataset that we collect and an open dataset provided in CoNLL-2004. Experimental results show that the model with self-attention achieves the state-of-the-art performance.

Emojis are frequently used to express moods, emotions, and feelings in social media. There has been much research on emojis and sentiments. However, existing methods mainly face two limitations. First, they treat emojis as binary indicator features and rely on handcrafted features for emoji-based sentiment analysis. Second, they consider the sentiment of emojis and texts separately, not fully exploring the impact of emojis on the sentiment polarity of texts. In this article, we investigate a sentiment analysis model based on bidirectional long short-term memory, and the model has two advantages compared with the existing work. First, it does not need feature engineering. Second, it utilizes the attention approach to model the impact of emojis on text. An evaluation on 10,042 manually labeled Sina Weibo showed that our model achieves much better performance compared with several strong baselines. To facilitate the related research, our corpus will be publicly available at https://github.com/yx100/emoji.

Identifying multiple emotions in a piece of text is an important research topic in the NLP community.Existing methods usually model the task as a multi-label classification problem, while these work has two issues. First, these methods fail to leverage the topic information of the text, which has been shown to be effective for sentiment analysis task. Second, different parts of the text can contribute differently to predicting different emotion labels, so the proposed model needs to capture effective features for each corresponding emotion, which is not considered by existing models. To tackle these problems, we propose a topic-enhanced capsule network, which contains two main parts: a variational autoencoder and a capsule module, for multi-label emotion detection task. Specifically, the variational autoencoder can learn the latent topic information of the text, and the capsule module can capture rich features for corresponding emotion. Experimental results on two benchmark datasets show that the proposed model achieves the current best performance, outperforming previous methods and strong baselines by a large margin.

Prior studies show that cross-lingual semantic role labeling (SRL) can be achieved by model transfer under the help of universal features. In this article, we fill the gap of cross-lingual SRL by proposing an end-to-end SRL model that incorporates a variety of universal features and transfer methods. We study both the bilingual transfer and multi-source transfer, under gold or machine-generated syntactic inputs, pre-trained high-order abstract features, and contextualized multilingual word representations. Experimental results on the Universal Proposition Bank corpus indicate that performances of the cross-lingual SRL can vary by leveraging different cross-lingual features. In addition, whether the features are gold-standard also has an impact on performances. Precisely, we find that gold syntax features are much more crucial for cross-lingual SRL, compared with the automatically-generated ones. Moreover, universal dependency structure features are able to give the best help, and both pre-trained high-order features and contextualized word representations can further bring significant improvements.

Many named entities usually contain nested entities in biomedical texts. Nested entities pose challenge to the task of named entity recognition. Traditional methods try to solve the problem as a graph-structure prediction problem. However, these methods fail to sufficiently capture the boundaries information between nested entities, which limits the performance of the task. In this paper, we take a different view by solving each unique entity type as a separate task, using multi-task learning with dispatched attention to facilitate information exchange between tasks. Results on GENIA corpus show that the proposed method is highly effective, obtaining the best results in the literature.

Drug-drug interaction (DDI) extraction is one of important tasks in biomedical text mining. Compared with traditional pipeline extraction, joint methods enjoy lower error propagation, achieving better performances. However, biomedical texts contain a large number of overlapped relations, and existing joint extraction models fail to capture these entity relations. In this paper, we propose a novel joint model for DDI extraction. Different from the existing sequential models, our span-based neural model treats the task as a relation triplets prediction problem, and builds the entity-graph by enumerating all possible candidate spans, and simultaneously models the relation between the correlated entities. Experimental results show that the proposed method significantly outperforms the strong baselines and previous methods, achieving the state-of-the-art performances.

Drug-drug interactions (DDIs) play a key role in various applications of biomedicine such as pharmacovigilance. DDIs are frequently reported in biomedical publications, making them an effective source for DDI extraction. Although neural networks have achieved competitive performances in DDI extraction, previous work depended on dependency paths to remove noise from the sentences of biomedical publications. However, such method may ignore crucial information about DDIs. Effectively exploiting much dependency information can improve DDI extraction. In this article, we proposed a model that combines the graph convolution neural network (GCNN) and bidirectional long short-term memory (BiLSTM) to extract DDI interactions from entire dependency graphs of sentences. We evaluated our model in the benchmark corpus for this domain, namely the DDIExtraction 2013 corpus. Our model achieved the state-of-the-art result (77.0% in F1), which are superior to the results reported in previous work. The code is available at https://github.com/woodyXwt/DDI_extraction.

The task of event extraction contains subtasks including detections for entity mentions, event triggers and argument roles. Traditional methods solve them as a pipeline, which does not make use of task correlation for their mutual benefits. There have been recent efforts towards building a joint model for all tasks. However, due to technical challenges, there has not been work predicting the joint output structure as a single task. We build a first model to this end using a neural transition-based framework, incrementally predicting complex joint structures in a state-transition process. Results on standard benchmarks show the benefits of the joint model, which gives the best result in the literature.

In recent years, with the prevalence of social media and smart devices, people causally reveal their locations such as shops, hotels, and restaurants in their tweets. Recognizing and linking such fine-grained location mentions to well-defined location profiles are beneficial for retrieval and recommendation systems. In this paper, we propose DLocRL, a new deep learning pipeline for fine-grained location recognition and linking in tweets, and verify its effectiveness on a real-world Twitter dataset.

Review-based methods are one of the dominant methods to address the data sparsity problem of recommender system. However, the performance of most existing review-based methods will degrade when the review is also sparse. To this end, we propose a method to exploit user-item p air-dependent features from a uxiliary r eviews written by l ike-minded users (PARL) to address such problem. That is, both the reviews written by the user and the reviews written for the item are incorporated to highlight the useful features covered by the auxiliary reviews. PARL not only alleviates the sparsity problem of reviews but also produce extra informative features to further improve the accuracy of rating prediction. More importantly, it is designed as a plug-and-play model which can be plugged into various deep recommender systems to improve recommendations provided by them. Extensive experiments on five real-world datasets show that PARL achieves better prediction accuracy than other state-of-the-art alternatives. Also, with the exploitation of auxiliary reviews, the performance of PARL is robust on datasets with different characteristics.

In machine learning, traditional data classification techniques analyze only physical features of the input data (e.g., distance or distribution) in order to identify the main differences among them. This type of approach is referred to as low level classification. However, the human (animal) brain is able to perform not only low orders of learning, but it is also able to identify patterns according to the semantic meaning of the input data. Data classification that considers both physical attributes and also the pattern formation, is referred to as high level classification. Previous high level classification techniques require a low level technique to work together. Such an approach does not fully highlight the ability of feature extraction embedded in high-level schemes. In this paper, we propose a pure network- based high level classification technique which aims to identify the classes of new instances by detecting and comparing the impact that each one of them has on the topological structure of the network components, which represent each class of the input data set. Eight artificially generated data sets, along with other nine different real classification data sets, were used in order to test this technique, as well as to compare its performance with those obtained by nine traditional and well-known classification models. The results of these tests are very stimulating, indicating that the novel technique proposed in this work may have great potential for further development and application. Moreover, the peculiarity of the concept of pattern based classification provides a new general approach for raw data feature extraction.

MOTIVATION: Disease named entities play a central role in many areas of biomedical research, and automatic recognition and normalization of such entities have received increasing attention in biomedical research communities. Existing methods typically used pipeline models with two independent phases: (i) a disease named entity recognition (DER) system is used to find the boundaries of mentions in text and (ii) a disease named entity normalization (DEN) system is used to connect the mentions recognized to concepts in a controlled vocabulary. The main problems of such models are: (i) there is error propagation from DER to DEN and (ii) DEN is useful for DER, but pipeline models cannot utilize this. METHODS: We propose a transition-based model to jointly perform disease named entity recognition and normalization, casting the output construction process into an incremental state transition process, learning sequences of transition actions globally, which correspond to joint structural outputs. Beam search and online structured learning are used, with learning being designed to guide search. Compared with the only existing method for joint DEN and DER, our method allows non-local features to be used, which significantly improves the accuracies. RESULTS: We evaluate our model on two corpora: the BioCreative V Chemical Disease Relation (CDR) corpus and the NCBI disease corpus. Experiments show that our joint framework achieves significantly higher performances compared to competitive pipeline baselines. Our method compares favourably to other state-of-the-art approaches. AVAILABILITY AND IMPLEMENTATION: Data and code are available at https://github.com/louyinxia/jointRN. CONTACT: [email protected].

BACKGROUND: Extracting biomedical entities and their relations from text has important applications on biomedical research. Previous work primarily utilized feature-based pipeline models to process this task. Many efforts need to be made on feature engineering when feature-based models are employed. Moreover, pipeline models may suffer error propagation and are not able to utilize the interactions between subtasks. Therefore, we propose a neural joint model to extract biomedical entities as well as their relations simultaneously, and it can alleviate the problems above. RESULTS: Our model was evaluated on two tasks, i.e., the task of extracting adverse drug events between drug and disease entities, and the task of extracting resident relations between bacteria and location entities. Compared with the state-of-the-art systems in these tasks, our model improved the F1 scores of the first task by 5.1% in entity recognition and 8.0% in relation extraction, and that of the second task by 9.2% in relation extraction. CONCLUSIONS: The proposed model achieves competitive performances with less work on feature engineering. We demonstrate that the model based on neural networks is effective for biomedical entity and relation extraction. In addition, parameter sharing is an alternative method for neural models to jointly process this task. Our work can facilitate the research on biomedical text mining.

Information extraction in clinical texts enables medical workers to find out problems of patients faster as well as makes intelligent diagnosis possible in the future. There has been a lot of work about disorder mention recognition in clinical narratives. But recognition of some more complicated disorder mentions like overlapping ones is still an open issue. This paper proposes a multi-label structured Support Vector Machine (SVM) based method for disorder mention recognition. We present a multi-label scheme which could be used in complicated entity recognition tasks. We performed three sets of experiments to evaluate our model. Our best F 1 -Score on the 2013 Conference and Labs of the Evaluation Forum data set is 0.7343. There are six types of labels in our multi-label scheme, all of which are represented by 24-bit binary numbers. The binary digits of each label contain information about different disorder mentions. Our multi-label method can recognize not only disorder mentions in the form of contiguous or discontiguous words but also mentions whose spans overlap with each other. The experiments indicate that our multi-label structured SVM model outperforms the condition random field (CRF) model for this disorder mention recognition task. The experiments show that our multi-label scheme surpasses the baseline. Especially for overlapping disorder mentions, the F 1 -Score of our multi-label scheme is 0.1428 higher than the baseline BIOHD1234 scheme. This multi-label structured SVM based approach is demonstrated to work well with this disorder recognition task. The novel multi-label scheme we presented is superior to the baseline and it can be used in other models to solve various types of complicated entity recognition tasks as well.

BACKGROUND: Biomedical named entity recognition(BNER) is a crucial initial step of information extraction in biomedical domain. The task is typically modeled as a sequence labeling problem. Various machine learning algorithms, such as Conditional Random Fields (CRFs), have been successfully used for this task. However, these state-of-the-art BNER systems largely depend on hand-crafted features. RESULTS: We present a recurrent neural network (RNN) framework based on word embeddings and character representation. On top of the neural network architecture, we use a CRF layer to jointly decode labels for the whole sentence. In our approach, contextual information from both directions and long-range dependencies in the sequence, which is useful for this task, can be well modeled by bidirectional variation and long short-term memory (LSTM) unit, respectively. Although our models use word embeddings and character embeddings as the only features, the bidirectional LSTM-RNN (BLSTM-RNN) model achieves state-of-the-art performance - 86.55% F1 on BioCreative II gene mention (GM) corpus and 73.79% F1 on JNLPBA 2004 corpus. CONCLUSIONS: Our neural network architecture can be successfully used for BNER without any manual feature engineering. Experimental results show that domain-specific pre-trained word embeddings and character-level representation can improve the performance of the LSTM-RNN models. On the GM corpus, we achieve comparable performance compared with other systems using complex hand-crafted features. Considering the JNLPBA corpus, our model achieves the best results, outperforming the previously top performing systems. The source code of our method is freely available under GPL at https://github.com/lvchen1989/BNER .

Chinese chunking has traditionally been solved by assuming gold standard word segmentation.We find that the accuracies drop drastically when automatic segmentation is used.Inspired by the fact that chunking knowledge can potentially improve segmentation, we explore a joint model that performs segmentation, POS-tagging and chunking simultaneously.In addition, to address the sparsity of full chunk features, we employ a semi-supervised method to derive chunk cluster features from large-scale automatically-chunked data.Results show the effectiveness of the joint model with semi-supervised features.

Sentiment classification on Twitter has attracted increasing research in recent years.Most existing work focuses on feature engineering according to the tweet content itself.In this paper, we propose a context-based neural network model for Twitter sentiment analysis, incorporating contextualized features from relevant Tweets into the model in the form of word embedding vectors.Experiments on both balanced and unbalanced datasets show that our proposed models outperform the current state-of-the-art.

It has been shown that learning distributed word representations is highly useful for Twitter sentiment classification.Most existing models rely on a single distributed representation for each word.This is problematic for sentiment classification because words are often polysemous and each word can contain different sentiment polarities under different topics.We address this issue by learning topic-enriched multi-prototype word embeddings (TMWE).In particular, we develop two neural networks which 1) learn word embeddings that better capture tweet context by incorporating topic information, and 2) learn topic-enriched multiple prototype embeddings for each word.Experiments on Twitter sentiment benchmark datasets in SemEval 2013 show that TMWE outperforms the top system with hand-crafted features, and the current best neural network model.

Aspect phrase grouping is an important task in aspect-level sentiment analysis. It is a challenging problem due to polysemy and context dependency. We propose an Attention-based Deep Distance Metric Learning (ADDML) method, by considering aspect phrase representation as well as context representation. First, leveraging the characteristics of the review text, we automatically generate aspect phrase sample pairs for distant supervision. Second, we feed word embeddings of aspect phrases and their contexts into an attention-based neural network to learn feature representation of contexts. Both aspect phrase embedding and context embedding are used to learn a deep feature subspace for measure the distances between aspect phrases for K-means clustering. Experiments on four review datasets show that the proposed method outperforms state-of-the-art strong baseline methods.

Recent studies show structured atomic event information is beneficial to represent the discourse semantic. However, extracting useful structured representation of events from open domain is a challenging problem. On one hand, previous event extraction methods on special domain, cannot be directly used for open domain because of domain limitation and predefined event pattern. On the other hand, atomic event extraction is simply regarded as a preprocessing step in previous related work, and few studies focus on atomic event extraction in open domain. In this paper, we propose an unsupervised method for Chinese event extraction in open domain. Being directed against the ellipsis and flexible sentence structure in Chinese text, the proposed method exploits double propagation (DP) to combine event extraction and event pattern generation, which does not require seed events or seed event patterns and is able to eliminate noise from syntactic parsing. Experimental results on standard benchmark show that our proposed method outperforms state-of-the-art algorithm.

Rui Sun, Yue Zhang, Meishan Zhang, Donghong Ji. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2015.

Extracting adverse drug events (a.k.a. drug-induced diseases or drug side effects) from the raw text has been widely studied in the biomedical area. It is usually assumed that entities of drugs and diseases are given by a separated named entity recognition model. In this paper, we propose a joint model to extract drugs, diseases and adverse drug events simultaneously. In our model, the structured perceptron is leveraged for training and multiple-beam search algorithm is used for decoding. The search algorithm is transition-based, i.e., an input sentence is processed in left-to-right order and predefined actions transit the sentence from one state to another which corresponds to a predicted result about drugs, diseases and adverse drug events in that sentence. Experimental results show that our joint approach obtains comparable performance compared with the baseline or state-of-the-art approaches and achieves 55.20% precision, 47.97% recall and 51.14% F1-measure in extraction of adverse drug events. We demonstrate that the joint approach is effective and can be easily extended to other entity-relation extraction systems such as protein-protein interactions and gene-disease relations. To facilitate the related research, our code is available online at: https://github.com/foxlf823/ade.

We propose a transition-based model for joint word segmentation, POS tagging and text normalization.Different from previous methods, the model can be trained on standard text corpora, overcoming the lack of annotated microblog corpora.To evaluate our model, we develop an annotated corpus based on microblogs.Experimental results show that our joint model can help improve the performance of word segmentation on microblogs, giving an error reduction in segmentation accuracy of 12.02%, compared to the traditional approach.

In this paper we propose a novel two-step procedure to recognize textual entailment. Firstly, we build a joint Restricted Boltzmann Machines (RBM) layer to learn the joint representation of the text-hypothesis pairs. Then the reconstruction error is calculated by comparing the original representation with reconstructed representation derived from the joint layer for each pair to recognize textual entailment. The joint RBM training data is automatically generated from a large news corpus. Experiment results show the contribution of the idea to the performance on textual entailment.

Aspect-phrase clustering is an important task for aspect finding in aspect-level sentiment analysis. Most of existing methods for this problem are based on a context model which aggregates related sentences that contains assigned aspect-phrase as context. In this paper, we explore a novel idea, capacity limitation , which states that the number of aggregated sentences in an aspect-phrase group has upper bound. And we propose a capacity constrained K-means algorithm to cluster aspect-phrases which encodes the capacity limitation as constraint. Empirical evaluation shows that the proposed method outperforms existing state-of-the-art methods.

Deceptive reviews detection has attracted significant attention from both business and research communities.However, due to the difficulty of human labeling needed for supervised learning, the problem remains to be highly challenging.This paper proposed a novel angle to the problem by modeling PU (positive unlabeled) learning.A semi-supervised model, called mixing population and individual property PU learning (MPIPUL), is proposed.Firstly, some reliable negative examples are identified from the unlabeled dataset.Secondly, some representative positive examples and negative examples are generated based on LDA (Latent Dirichlet Allocation).Thirdly, for the remaining unlabeled examples (we call them spy examples), which can not be explicitly identified as positive and negative, two similarity weights are assigned, by which the probability of a spy example belonging to the positive class and the negative class are displayed.Finally, spy examples and their similarity weights are incorporated into SVM (Support Vector Machine) to build an accurate classifier.Experiments on gold-standard dataset demonstrate the effectiveness of MPIPUL which outperforms the state-of-the-art baselines.

Cross-lingual textual entailment is a relatively new problem that detects the entailment relationship between two text fragments written in different languages. Previous work adopted machine learning algorithms and similarity measures as features to address this task. In order to overcome the high cost of human annotation and further improve the recognition performance, we present a novel co-training approach to solve this problem. We first use an off-the-shelf machine translation tool to eliminate the language gap between two texts. Then we measure the similarities and differences between two texts and regard them as sufficient and redundant views. We use those two views to conduct the co-training procedure to perform classification. Besides, a new effective Kullback-Leibler (KL) based criterion is proposed to select the results from all possible iterations. Experiments on cross-lingual datasets provided by SemEval 2013 show that our method significantly outperforms the baseline systems and previous work.

Shortage of manually labeled data is an obstacle to supervised relation extraction methods. In this paper we investigate a graph based semi-supervised learning algorithm, a label propagation (LP) algorithm, for relation extraction. It represents labeled and unlabeled examples and their distances as the nodes and the weights of edges of a graph, and tries to obtain a labeling function to satisfy two constraints: 1) it should be fixed on the labeled nodes, 2) it should be smooth on the whole graph. Experiment results on the ACE corpus showed that this LP algorithm achieves better performance than SVM when only very few labeled examples are available, and it also performs better than bootstrapping for the relation extraction task.

This paper proposes a novel document re-ranking approach in information retrieval, which is done by a label propagation-based semi-supervised learning algorithm to utilize the intrinsic structure underlying in the large document data. Since no labeled relevant or irrelevant documents are generally available in IR, our approach tries to extract some pseudo labeled documents from the ranking list of the initial retrieval. For pseudo relevant documents, we determine a cluster of documents from the top ones via cluster validation-based k-means clustering; for pseudo irrelevant ones, we pick a set of documents from the bottom ones. Then the ranking of the documents can be conducted via label propagation. Evaluation on benchmark corpora shows that the approach can achieve significant improvement over standard baselines and performs better than other related approaches.

Shortage of manually sense-tagged data is an obstacle to supervised word sense disambiguation methods. In this paper we investigate a label propagation based semi-supervised learning algorithm for WSD, which combines labeled and unlabeled data in learning process to fully realize a global consistency assumption: similar examples should have similar labels. Our experimental results on benchmark corpora indicate that it consistently outperforms SVM when only very few labeled examples are available, and its performance is also better than monolingual bootstrapping, and comparable to bilingual bootstrapping.

In this article we describe our approach to Chinese information retrieval, where a query is a short natural language description. First, we use automatically extracted short terms from document sets to build indexes and use the short terms in both the query and documents to do initial retrieval. Next, we use long terms extracted from the document collection to reorder the top N retrieved documents to improve precision. Finally, we acquire the relevant terms of the short terms from the Internet and the top retrieved documents and use them to do query expansion. Experiments on the NTCIR-4 CLIR Chinese SLIR sub-collection show that document reranking can both improve the retrieval performance on its own and make a significant contribution to query expansion. The experiments also show that the extended query expansion proposed in this article is more effective than the standard Rocchio query expansion.

This paper presents an unsupervised word sense learning algorithm, which induces senses of target word by grouping its occurrences into a "natural" number of clusters based on the similarity of their contexts. For removing noisy words in feature set, feature selection is conducted by optimizing a cluster validation criterion subject to some constraint in an unsupervised manner. Gaussian mixture model and Minimum Description Length criterion are used to estimate cluster structure and cluster number. Experimental results show that our algorithm can find important feature subset, estimate model order (cluster number) and achieve better performance than another algorithm which requires cluster number to be provided.

International Conference on Information and Knowledge Management, Proceedings