Document-level event argument extraction (DEAE) is essential for knowledge acquisition, aiming to extract participants of events from documents. In the zero-shot setting, existing methods employ LLMs to generate synthetic data to address the challenge posed by the scarcity of annotated data. However, relying solely on Event-type-only prompts makes it difficult for the generated content to accurately capture the contextual and structural relationships of unseen events. Moreover, ensuring the reliability and usability of synthetic data remains a significant challenge due to the absence of quality evaluation mechanisms. To this end, we introduce a multi-agent collaboration framework for zero-shot document-level event argument extraction (ZS-DEAE), which simulates the human collaborative cognitive process of “Propose–Evaluate–Revise.” Specifically, the framework comprises a generation agent and an evaluation agent. The generation agent synthesizes data for unseen events by leveraging knowledge from seen events, while the evaluation agent extracts arguments from the synthetic data and assesses their semantic consistency with the context. The evaluation results are subsequently converted into reward signals, with event structure constraints incorporated into the reward design to enable iterative optimization of both agents via reinforcement learning. In three zero-shot scenarios constructed from the RAMS and WikiEvents datasets, our method achieves improvements both in data generation quality and argument extraction performance, while the generated data also effectively enhances the zero-shot performance of other DEAE models.

Ya Su, Hu Zhang, Dan Qiao, YuJie Wang, Yunxiao Zhao, Yue Fan, Shike Li, Ru Li, Hongye Tan. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Despite significant advancements in large visual-language models (LVLMs), hallucinations remain a major bottleneck in their practical applications. One key factor contributing to hallucinations is the over-reliance on language priors during the autoregressive text generation process. Visual Contrastive Decoding (VCD), a popular technique for mitigating hallucinations, perturbs the visual input and compares the perturbed output with the original. However, it often overlooks the gradual attenuation of visual information within the decoder, limiting the model's ability to generate text based on actual visual content. We propose a novel, training-free method—Visual-Enhanced Contrastive Decoding (VECD)—which addresses this issue by amplifying visual information within the decoder, thereby reducing hallucinations caused by excessive reliance on language priors. VECD dynamically selects later layers for visual injection, while retaining only essential visual tokens in early layers. This approach enhances the generation process by adaptively balancing visual and language priors. By comparing outputs with and without visual amplification, we derive a refined probability distribution for the next token. Moreover, we improve the beam search algorithm by introducing a visually guided token selection strategy, enabling the generation of text that aligns more closely with the image content. Our extensive experiments show that VECD significantly reduces hallucinations and improves the quality of generated text, demonstrating its effectiveness as a practical solution.

Event Causality Identification (ECI) aims to identify fine-grained causal relationships between events in an unstructured text. Existing ECI methods primarily rely on knowledge enhanced and graph-based reasoning approaches, but they often overlook the dependencies between similar events. Additionally, the connection between unstructured text and structured knowledge is relatively weak. Therefore, this paper proposes an ECI method enhanced by LLM Knowledge and Concept-Level Event Relations (LKCER). Specifically, LKCER constructs a conceptual-level heterogeneous event graph by leveraging the local contextual information of related event mentions, generating a more comprehensive global semantic representation of event concepts. At the same time, the knowledge generated by COMET is filtered and enriched using LLM, strengthening the associations between event pairs and knowledge. Finally, the joint event conceptual representation and knowledge-enhanced event representation are used to uncover potential causal relationships between events. The experimental results show that our method outperforms previous state-of-the-art methods on both benchmarks, EventStoryLine and Causal-TimeBank.

Data-driven pre-trained language models typically perform shortcut learning wherein they rely on the spurious correlations between the data and the ground truth. This reliance can undermine the robustness and generalization of the model. To address this issue, data augmentation emerges as a promising solution. By integrating anti-shortcut data to the training set, the models’ shortcut-induced biases can be mitigated. However, existing methods encounter three challenges: 1) Manual definition of shortcuts is tailored to particular datasets, restricting generalization. 2) The inherent confirmation bias during model training hampers the effectiveness of data augmentation. 3) Insufficient exploration of the relationship between the model performance and the augmented data quantity may result in excessive data consumption. To tackle these challenges, we propose a method of Smart Data Augmentation based on Large Language Models (SAug-LLM). It leverages the LLMs to autonomously identify shortcuts and generate their anti-shortcut counterparts. In addition, the dual validation is employed to mitigate the confirmation bias during the model retraining. Furthermore, the data augmentation process is optimized to effectively rectify model biases while minimizing data consumption. We validate the effectiveness and generalization of our method through extensive experiments across various natural language processing tasks, demonstrating an average performance improvement of 5.61%.

Event Causal Identification (ECI) aims to identify fine-grained causal relationships between events from unstructured text. Contrastive learning has shown promise in enhancing ECI by optimizing representation distances between positive and negative samples. However, existing methods often rely on rule-based or random sampling strategies, which may introduce spurious causal positives. Moreover, static negative samples often fail to approximate actual decision boundaries, thus limiting discriminative performance. Therefore, we propose an ECI method enhanced by Dynamic Energy-based Contrastive Learning with multi-stage knowledge Verification (DECLV). Specifically, we integrate multi-source knowledge validation and LLM-driven causal inference to construct a multi-stage knowledge validation mechanism, which generates high-quality contrastive samples and effectively suppresses spurious causal disturbances. Meanwhile, we introduce the Stochastic Gradient Langevin Dynamics (SGLD) method to dynamically generate adversarial negative samples, and employ an energy-based function to model the causal boundary between positive and negative samples. The experimental results show that our method outperforms previous state-of-the-art methods on both benchmarks, EventStoryLine and Causal-TimeBank.

Visual story generation, which involves generating short stories from sequential images, has become a core task at the intersection of computer vision and natural language processing. However, existing methods suffer from a bias in the concept predicates predicted, leading to a semantic gap between the generated stories and the images. This article proposes a novel visual story generation model that utilizes multi granularity image information to guide the generation process and correct the bias in concept predicates, resulting in more image-consistent stories. The proposed model consists of two stages: In the first stage, a set of concepts predicates is predicted from the image and enriched with external knowledge, and the most suitable concepts for story generation are selected. In the second stage, fine-grained image information are utilized to integrate image information into the story generation module, improving the bias in concept predicates. The image theme information and the generated results of previous moments are used as prompts to guide the story generation module. Experimental results show that the proposed model outperforms baseline models in all evaluation metrics. Specifically, the Bilingual Evaluation Understudy 1 (BLEU-1), BLEU-2, BLEU-3, and BLEU-4 metrics are improved by 4.0, 3.8, 3.02, and 1.98 percentage points, respectively, and the METEOR metric is improved by 1.4 percentage points. The generated stories are more consistent with the image content, maintain a consistent theme, and enhance coherence between contexts.

Event Argument Extraction (EAE) aims to extract arguments for specified events from a text.Previous research has mainly focused on addressing long-distance dependencies of arguments, modeling co-occurrence relationships between roles and events, but overlooking potential inductive biases: (i) semantic differences among arguments of the same type and (ii) large margin separation between arguments of the different types.Inspired by prototype networks, we introduce a new model named HMPEAE, which takes the two inductive biases above as targets to locate prototypes and guide the model to learn argument representations based on these prototypes.Specifically, we set multiple prototypes to represent each role to capture intra-class differences.Simultaneously, we use hypersphere as the output space for prototypes, defining large margin separation between prototypes to encourage the model to learn significant differences between different types of arguments effectively.We solve the "argument-prototype" assignment as an optimal transport problem to optimize the argument representation and minimize the absolute distance between arguments and prototypes to achieve compactness within sub-clusters.Experimental results on the RAMS and WikiEvents datasets show that HMPEAE achieves state-of-the-art performances.

Graph Transformer has attracted widespread attention in network representation learning recently. It effectively overcomes several limitations of graph neural networks (GNNs) and learns richer graph representations by extending the attention mechanism to graph data. However, most of the existing graph Transformers suffer from two issues. One is that they do not adequately consider the graph structure, and the other is that the original node features are not used in the attention computation. To this end, we propose a novel graph Transformer method for network representation learning, named Substructure-augmented Graph Transformer (SAGT), which exploits substructures to extract graph structural information. Specifically, SAGT first utilizes predefined geometric substructures to conduct structural matching within the original graph and records the number of matches for each node. Subsequently, it merges the node-matching information with the initial node features as inputs to the Transformer, thereby helping to integrate graph structure information into node representations. Moreover, We employ a synchronous learning framework of GNN and Transformer to update the node embeddings. Experimental results show that our method outperforms the existing models on seven graph prediction benchmarks.

Document-level event extraction aims to identify event types and arguments from one document. However, existing methods fail to consider semantic distinctions between multiple mentions of one entity and ignore dynamic representation of entities across multiple events simultaneously. Therefore, the models cannot capture flexible and specific entity representations in different event types. In this article, we propose EADRE ( E vent-type- A ware D ynamic R epresentation of E ntities). Specifically, we use cross-attention between mentions and event-type prototypes to obtain event-type-aware mention features. Then, we propose ASGate ( A daptive S oft G ate), which adaptively selects mention features to reduce the influence of event-unrelated mentions. EADRE introduces no more than 1% new parameters compared with the base model and has good transportability. Experiments on two public datasets show that EADRE improves the performance of multi-event extraction by 2.6% and 3.1%, as well as outperforms previous state-of-the-art baselines by 0.2% and 1.6%, with lower resource consumption without the use of pre-trained models. Further experimental analysis shows that EADRE significantly improves extraction performance in O2M and M2M multi-event scenarios.

Commonsense question answering (CQA) generally means that the machine uses its mastered commonsense to answer questions without relevant background material, which is a challenging task in natural language processing. Existing methods focus on retrieving relevant subgraphs from knowledge graphs based on key entities and designing complex graph neural networks to perform reasoning over the subgraphs. However, they have the following problems: i) the nested entities in key entities lead to the introduction of irrelevant knowledge; ii) the QA context is not well integrated with the subgraphs; and iii) insufficient context knowledge hinders subgraph nodes understanding. In this paper, we present a heterogeneous-graph reasoning with context paraphrase method (HCP), which introduces the paraphrase knowledge from the dictionary into key entity recognition and subgraphs construction, and effectively fuses QA context and subgraphs during the encoding phase of the pre-trained language model (PTLM). Specifically, HCP filters the nested entities through the dictionary's vocabulary and constructs the Heterogeneous Path-Paraphrase (HPP) graph by connecting the paraphrase descriptions11The paraphrase descriptions are English explanations of words or phrases in WordNet and Wiktionary.with the key entity nodes in the subgraphs. Then, by constructing the visible matrices in the PTLM encoding phase, we fuse the QA context representation into the HPP graph. Finally, to get the answer, we perform reasoning on the HPP graph by Mask Self-Attention. Experimental results on CommonsenseQA and OpenBookQA show that fusing QA context with HPP graph in the encoding stage and enhancing the HPP graph representation by using context paraphrase can improve the machine's commonsense reasoning ability.

Recently, knowledge graphs (KGs) have won noteworthy success in commonsense question answering.Existing methods retrieve relevant subgraphs in the KGs through key entities and reason about the answer with language models (LMs) and graph neural networks.However, they ignore (i) optimizing the knowledge representation and structure of subgraphs and (ii) deeply fusing heterogeneous QA context with subgraphs.In this paper, we propose a dynamic heterogeneous-graph reasoning method with LMs and knowledge representation learning (DHLK), which constructs a heterogeneous knowledge graph (HKG) based on multiple knowledge sources and optimizes the structure and knowledge representation of the HKG using a two-stage pruning strategy and knowledge representation learning (KRL).It then performs joint reasoning by LMs and Relation Mask Self-Attention (RMSA).Specifically, DHLK filters key entities based on the dictionary vocabulary to achieve the first-stage pruning while incorporating the paraphrases in the dictionary into the subgraph to construct the HKG.Then, DHLK encodes and fuses the QA context and HKG using LM, and dynamically removes irrelevant KG entities based on the attention weights of LM for the second-stage pruning.Finally, DHLK introduces KRL to optimize the knowledge representation and perform answer reasoning on the HKG by RMSA.We evaluate DHLK at CommonsenseQA and OpenBookQA, and show its improvement on existing LM and LM+KG methods.

Legal judgment elements extraction (LJEE) aims to identify the different judgment features from the fact description in legal documents automatically, which helps to improve the accuracy and interpretability of the judgment results. In real court rulings, judges usually need to scan both the fact descriptions and the law articles repeatedly to find out the relevant information, and it is hard to acquire the key judgment features quickly, so legal judgment elements extraction is a crucial and challenging task for legal judgment prediction. However, most existing methods follow the text classification framework, which fails to model the attentive relations of the law articles and the legal judgment elements. To address this issue, we simulate the working process of human judges, and propose a legal judgment elements extraction method with a law article-aware mechanism, which captures the complex semantic correlations of the law article and the legal judgment elements. Experimental results show that our proposed method achieves significant improvements than other state-of-the-art baselines on the element recognition task dataset. Compared with the BERT-CNN model, the proposed “All labels Law Articles Embedding Model (ALEM)” improves the accuracy, recall, and F1 value by 0.5, 1.4 and 1.0, respectively.

Xuefeng Su, Ru Li, Xiaoli Li, Jeff Z. Pan, Hu Zhang, Qinghua Chai, Xiaoqi Han. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Recently graph-based methods have been adopted for Abstractive Text Summarization. However, existing graph-based methods only consider either word relations or structure information, which neglect the correlation between them. To simultaneously capture the word relations and structure information from sentences, we propose a novel Dual Graph network for Abstractive Sentence Summarization. Specifically, we first construct semantic scenario graph and semantic word relation graph based on FrameNet, and subsequently learn their representations and design graph fusion method to enhance their correlation and obtain better semantic representation for summary generation. Experimental results show our model outperforms existing state-of-the-art methods on two popular benchmark datasets, i.e., Gigaword and DUC 2004.

Comprehending unstructured text is a challenging task for machines because it involves understanding texts and answering questions. In this paper, we study the multiple‐choice task for reading comprehension based on MC Test datasets and Chinese reading comprehension datasets, among which Chinese reading comprehension datasets which are built by ourselves. Observing the above‐mentioned training sets, we find that “sentence comprehension” is more important than “word comprehension” in multiple‐choice task, and therefore we propose sentence‐level neural network models. Our model firstly uses LSTM network and a composition model to learn compositional vector representation for sentences and then trains a sentence‐level attention model for obtaining the sentence‐level attention between the sentence embedding in documents and the optional sentences embedding by dot product. Finally, a consensus attention is gained by merging individual attention with the merging function. Experimental results show that our model outperforms various state‐of‐the‐art baselines significantly for both the multiple‐choice reading comprehension datasets.