Multimodal Model Editing (MMED) aims to correct erroneous knowledge in multimodal models. Existing evaluation methods, adapted from textual model editing, overstate success by relying on low-similarity or random inputs, obscure overfitting. We propose a comprehensive locality evaluation framework, covering three key dimensions: random-image locality, no-image locality, and consistent-image locality, operationalized through seven distinct data types, enabling a detailed and structured analysis of multimodal edits. We introduce De-VQA, a dynamic evaluation for visual question answering, uncovering a phenomenon we term transient blindness, overfitting to edit-similar text while ignoring visuals. Token analysis shows edits disproportionately affect textual tokens. We propose locality-aware adversarial losses to balance cross-modal representations. Empirical results demonstrate that our approach consistently outperforms existing baselines, reducing transient blindness and improving locality by 17% on average.

Document-level event argument extraction (DEAE) is essential for knowledge acquisition, aiming to extract participants of events from documents. In the zero-shot setting, existing methods employ LLMs to generate synthetic data to address the challenge posed by the scarcity of annotated data. However, relying solely on Event-type-only prompts makes it difficult for the generated content to accurately capture the contextual and structural relationships of unseen events. Moreover, ensuring the reliability and usability of synthetic data remains a significant challenge due to the absence of quality evaluation mechanisms. To this end, we introduce a multi-agent collaboration framework for zero-shot document-level event argument extraction (ZS-DEAE), which simulates the human collaborative cognitive process of “Propose–Evaluate–Revise.” Specifically, the framework comprises a generation agent and an evaluation agent. The generation agent synthesizes data for unseen events by leveraging knowledge from seen events, while the evaluation agent extracts arguments from the synthetic data and assesses their semantic consistency with the context. The evaluation results are subsequently converted into reward signals, with event structure constraints incorporated into the reward design to enable iterative optimization of both agents via reinforcement learning. In three zero-shot scenarios constructed from the RAMS and WikiEvents datasets, our method achieves improvements both in data generation quality and argument extraction performance, while the generated data also effectively enhances the zero-shot performance of other DEAE models.

Ya Su, Hu Zhang, Dan Qiao, YuJie Wang, Yunxiao Zhao, Yue Fan, Shike Li, Ru Li, Hongye Tan. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Rationalization, a data-centric framework, aims to build self-explanatory models to explain the prediction outcome by generating a subset of human-intelligible pieces of the input data. It involves a cooperative game model where a generator generates the most human-intelligible parts of the input (i.e., rationales), followed by a predictor that makes predictions based on these generated rationales. Conventional rationalization methods typically impose constraints via regularization terms to calibrate or penalize undesired generation. However, these methods are suffering from a problem called mode collapse, in which the predictor produces correct predictions yet the generator consistently outputs rationales with collapsed patterns. Moreover, existing studies are typically designed separately for specific collapsed patterns, lacking a unified consideration. In this paper, we systematically revisit cooperative rationalization from a novel game-theoretic perspective and identify the fundamental cause of this problem: the generator no longer tends to explore new strategies to uncover informative rationales, ultimately leading the system to converge to a suboptimal game equilibrium (correct predictionsv.scollapsed rationales). To solve this problem, we then propose a novel approach, Game-theoreticPolicyOptimization orientedRATionalization (PoRat), which progressively introduces policy interventions to address the game equilibrium in the cooperative game process, thereby guiding the model toward a more optimal solution state. We theoretically analyse the cause of such a suboptimal equilibrium and prove the feasibility of the proposed method. Furthermore, we validate our method on nine widely used real-world datasets and two synthetic settings, wherePoRatachieves up to 8.1% performance improvements over existing state-of-the-art methods. Code and data are available atanonymous.4open.science/r/Rationalization-PORAT-ECE9.

Despite significant advancements in large visual-language models (LVLMs), hallucinations remain a major bottleneck in their practical applications. One key factor contributing to hallucinations is the over-reliance on language priors during the autoregressive text generation process. Visual Contrastive Decoding (VCD), a popular technique for mitigating hallucinations, perturbs the visual input and compares the perturbed output with the original. However, it often overlooks the gradual attenuation of visual information within the decoder, limiting the model's ability to generate text based on actual visual content. We propose a novel, training-free method—Visual-Enhanced Contrastive Decoding (VECD)—which addresses this issue by amplifying visual information within the decoder, thereby reducing hallucinations caused by excessive reliance on language priors. VECD dynamically selects later layers for visual injection, while retaining only essential visual tokens in early layers. This approach enhances the generation process by adaptively balancing visual and language priors. By comparing outputs with and without visual amplification, we derive a refined probability distribution for the next token. Moreover, we improve the beam search algorithm by introducing a visually guided token selection strategy, enabling the generation of text that aligns more closely with the image content. Our extensive experiments show that VECD significantly reduces hallucinations and improves the quality of generated text, demonstrating its effectiveness as a practical solution.

Event Causality Identification (ECI) aims to identify fine-grained causal relationships between events in an unstructured text. Existing ECI methods primarily rely on knowledge enhanced and graph-based reasoning approaches, but they often overlook the dependencies between similar events. Additionally, the connection between unstructured text and structured knowledge is relatively weak. Therefore, this paper proposes an ECI method enhanced by LLM Knowledge and Concept-Level Event Relations (LKCER). Specifically, LKCER constructs a conceptual-level heterogeneous event graph by leveraging the local contextual information of related event mentions, generating a more comprehensive global semantic representation of event concepts. At the same time, the knowledge generated by COMET is filtered and enriched using LLM, strengthening the associations between event pairs and knowledge. Finally, the joint event conceptual representation and knowledge-enhanced event representation are used to uncover potential causal relationships between events. The experimental results show that our method outperforms previous state-of-the-art methods on both benchmarks, EventStoryLine and Causal-TimeBank.

Data-driven pre-trained language models typically perform shortcut learning wherein they rely on the spurious correlations between the data and the ground truth. This reliance can undermine the robustness and generalization of the model. To address this issue, data augmentation emerges as a promising solution. By integrating anti-shortcut data to the training set, the models’ shortcut-induced biases can be mitigated. However, existing methods encounter three challenges: 1) Manual definition of shortcuts is tailored to particular datasets, restricting generalization. 2) The inherent confirmation bias during model training hampers the effectiveness of data augmentation. 3) Insufficient exploration of the relationship between the model performance and the augmented data quantity may result in excessive data consumption. To tackle these challenges, we propose a method of Smart Data Augmentation based on Large Language Models (SAug-LLM). It leverages the LLMs to autonomously identify shortcuts and generate their anti-shortcut counterparts. In addition, the dual validation is employed to mitigate the confirmation bias during the model retraining. Furthermore, the data augmentation process is optimized to effectively rectify model biases while minimizing data consumption. We validate the effectiveness and generalization of our method through extensive experiments across various natural language processing tasks, demonstrating an average performance improvement of 5.61%.

Multi-hop question answering (MHQA) aims to utilize multi-source intensive documents retrieved to derive the answer. However, it is very challenging to model the importance of knowledge retrieved. Previous approaches primarily emphasize single-step and multi-step iterative decomposition or retrieval, which are susceptible to failure in long-chain reasoning due to the progressive accumulation of erroneous information. To address this problem, we propose a novel Local-tO-Global optimized retrieval method (LOG) to discover more beneficial information, facilitating the MHQA. In particular, we design a pointwise conditional v-information based local information modeling to cover usable documents with reasoning knowledge. We also improve tuplet objective loss, advancing multi-examples-aware global optimization to model the relationship between scattered documents. Extensive experimental results demonstrate our proposed method outperforms prior state-of-the-art models, and it can significantly improve multi-hop reasoning, notably for long-chain reasoning.

Combining logical reasoning with large language models enhances their capacity to solve complex problems reliably and robustly. Nevertheless, previous studies for logical reasoning mainly focus on capturing entity-aware knowledge while needing help to capture logical relationships among facts. Besides, the limited labeled data further hinders the ability of data-hungry neural models. Therefore, we propose a novel symbolic-neural reasoning method based on factual evidence and Abstract Meaning Representation (AMR) to enhance the Reasoning ability of Large Language Models. The former enriches the semantics of the context to obtain implicit logical expressions according to factual evidence and logical rules. The latter exploit AMR to augment literally similar but logically different training instances for contrastive learning. Extensive experiments on benchmark datasets demonstrate that the proposed method significantly outperforms solid baselines and achieves new state-of-the-art performance.

Attribution is crucial in question answering (QA) with Large Language Models (LLMs). SOTA question decomposition-based approaches use long form answers to generate questions for retrieving related documents. However, the generated questions are often irrelevant and incomplete, resulting in a loss of facts in retrieval. These approaches also fail to aggregate evidence snippets from different documents and paragraphs. To tackle these problems, we propose a new fact decomposition-based framework called FIDES (faithful context enhanced fact decomposition and evidence aggregation) for attributed QA. FIDES uses a contextually enhanced two-stage faithful decomposition method to decompose long form answers into sub-facts, which are then used by a retriever to retrieve related evidence snippets. If the retrieved evidence snippets conflict with the related sub-facts, such sub-facts will be revised accordingly. Finally, the evidence snippets are aggregated according to the original sentences. Extensive evaluation has been conducted with six datasets, with an additionally proposed new metric called Attrauto–P for evaluating the evidence precision. FIDES outperforms the SOTA methods by over 14% in average with GPT-3.5-turbo, Gemini and Llama 70B series.

Event Causal Identification (ECI) aims to identify fine-grained causal relationships between events from unstructured text. Contrastive learning has shown promise in enhancing ECI by optimizing representation distances between positive and negative samples. However, existing methods often rely on rule-based or random sampling strategies, which may introduce spurious causal positives. Moreover, static negative samples often fail to approximate actual decision boundaries, thus limiting discriminative performance. Therefore, we propose an ECI method enhanced by Dynamic Energy-based Contrastive Learning with multi-stage knowledge Verification (DECLV). Specifically, we integrate multi-source knowledge validation and LLM-driven causal inference to construct a multi-stage knowledge validation mechanism, which generates high-quality contrastive samples and effectively suppresses spurious causal disturbances. Meanwhile, we introduce the Stochastic Gradient Langevin Dynamics (SGLD) method to dynamically generate adversarial negative samples, and employ an energy-based function to model the causal boundary between positive and negative samples. The experimental results show that our method outperforms previous state-of-the-art methods on both benchmarks, EventStoryLine and Causal-TimeBank.

Automatic Program Repair (APR) is a core technology in software development and maintenance, with aims to enable automated defect repair with minimal human intervention. In recent years, the substantial advancements in Large Language Models (LLMs) and the Chain-of-Thought (CoT) techniques have significantly enhanced the reasoning capabilities of these models. However, due to the complex logic and multi-step reasoning ability needed, the application of CoT techniques in the APR domain remains insufficient. This study systematically evaluates the performance of several common CoT techniques in APR tasks and proposes an innovative framework T3, which integrates the powerful reasoning capabilities of LLMs with tree search, effectively improving the precision of generating candidate repair solutions. Furthermore, T3provides valuable guidance for optimizing sample selection and repair strategies in APR tasks, establishing a robust framework for achieving efficient automated debugging.

Multimodal entity linking (MEL) aims to link ambiguous mentions within multimodal contexts to corresponding entities in a multimodal knowledge base. Most existing approaches to MEL are based on representation learning or vision-and-language pre-training mechanisms for exploring the complementary effect among multiple modalities. However, these methods suffer from two limitations. On the one hand, they overlook the possibility of considering negative samples from the same modality. On the other hand, they lack mechanisms to capture bidirectional cross-modal interaction. To address these issues, we propose a Multi-level Matching network for Multimodal Entity Linking(M3EL). Specifically, M3EL is composed of three different modules: (i) a Multimodal Feature Extraction module, which extracts modality-specific representations with a multimodal encoder and introduces an intra-modal contrastive learning sub-module to obtain better discriminative embeddings based on uni-modal differences; (ii) an Intra-modal Matching Network module, which contains two levels of matching granularity: Coarse-grained Global-to-Global and Fine-grained Global-to-Local, to achieve local and global level intra-modal interaction; (iii) a Cross-modal Matching Network module, which applies bidirectional strategies, Textual-to-Visual and Visual-to-Textual matching, to implement bidirectional cross-modal interaction. Extensive experiments conducted on WikiMEL, RichpediaMEL, and WikiDiverse datasets demonstrate the outstanding performance of M3EL when compared to the state-of-the-art baselines.

Multimodal Entity Linking (MEL) aims to link ambiguous mentions within multimodal contexts to associated entities in a multimodal knowledge base. Existing approaches to MEL introduce multimodal interaction and fusion mechanisms to bridge the modality gap and enable multi-grained semantic matching. However, they do not address two important problems: (i) mention ambiguity, i.e., the lack of semantic content caused by the brevity and omission of key information in the mention's textual context; (ii) dynamic selection of modal content, i.e., to dynamically distinguish the importance of different parts of modal information. To mitigate these issues, we propose a Multi-level Mixture of Experts (MMoE) model for MEL. MMoE has four components: (i) the description-aware mention enhancement module leverages large language models to identify the WikiData descriptions that best match a mention, considering the mention's textual context; (ii) the multimodal feature extraction module adopts multimodal feature encoders to obtain textual and visual embeddings for both mentions and entities; (iii)-(iv) the intra-level mixture of experts and inter-level mixture of experts modules apply a switch mixture of experts mechanism to dynamically and adaptively select features from relevant regions of information. Extensive experiments on WikiMEL, RichpediaMEL and WikiDiverse datasets demonstrate the outstanding performance of MMoE compared to the state-of-the-art. MMoE's code is available at: https://github.com/zhiweihu1103/MEL-MMoE.

Visual story generation, which involves generating short stories from sequential images, has become a core task at the intersection of computer vision and natural language processing. However, existing methods suffer from a bias in the concept predicates predicted, leading to a semantic gap between the generated stories and the images. This article proposes a novel visual story generation model that utilizes multi granularity image information to guide the generation process and correct the bias in concept predicates, resulting in more image-consistent stories. The proposed model consists of two stages: In the first stage, a set of concepts predicates is predicted from the image and enriched with external knowledge, and the most suitable concepts for story generation are selected. In the second stage, fine-grained image information are utilized to integrate image information into the story generation module, improving the bias in concept predicates. The image theme information and the generated results of previous moments are used as prompts to guide the story generation module. Experimental results show that the proposed model outperforms baseline models in all evaluation metrics. Specifically, the Bilingual Evaluation Understudy 1 (BLEU-1), BLEU-2, BLEU-3, and BLEU-4 metrics are improved by 4.0, 3.8, 3.02, and 1.98 percentage points, respectively, and the METEOR metric is improved by 1.4 percentage points. The generated stories are more consistent with the image content, maintain a consistent theme, and enhance coherence between contexts.

Attributed Question Answering (AQA) aims to provide both a trustworthy answer and a reliable attribution report for a given question. Retrieval is a widely adopted approach, including two general paradigms: Retrieval-Then-Read (RTR) and post-hoc retrieval. Recently, Large Language Models (LLMs) have shown remarkable proficiency, prompting growing interest in AQA among researchers. However, RTR-based AQA often suffers from irrelevant knowledge and rapidly changing information, even when LLMs are adopted, while post-hoc retrievalbased AQA struggles with comprehending long-form answers with complex logic, and precisely identifying the content needing revision and preserving the original intent. To tackle these problems, this paper proposes an Atomic fact decompositionbased Retrieval and Editing (ARE) framework, which decomposes the generated long-form answers into molecular clauses and atomic facts by the instruction-tuned LLMs. Notably, the instruction-tuned LLMs are fine-tuned using a well-constructed dataset, generated from large scale Knowledge Graphs (KGs). This process involves extracting one-hop neighbors from a given set of entities and transforming the result into coherent long-form text. Subsequently, ARE leverages a search engine to retrieve evidences related to atomic facts, inputting these evidences into an LLM-based verifier to determine whether the facts require expansion for re-retrieval or editing. Furthermore, the edited facts are backtracked into the original answer, with evidence aggregated based on the relationship between molecular clauses and atomic facts. Extensive evaluations demonstrate the superior performance of our proposed method over the state-of-the-arts on several datasets, with an additionally proposed new metricAttrpfor evaluating the precision of evidence attribution.

Although neural models have achieved remarkable performance, they still encounter doubts due to the intransparency. To this end, model prediction explanation is attracting more and more attentions. However, current methods rarely incorporate external knowledge and still suffer from three limitations: (1) Neglecting concept completeness. Merely selecting concepts may not sufficient for prediction. (2) Lacking concept fusion. Failure to merge semantically-equivalent concepts. (3) Difficult in manipulating model behavior. Lack of verification for explanation on original model. To address these issues, we propose a novel knowledge-aware neuron interpretation framework to explain model predictions for image scene classification. Specifically, for concept completeness, we present core concepts of a scene based on knowledge graph, ConceptNet, to gauge the completeness of concepts. Our method, incorporating complete concepts, effectively provides better prediction explanations compared to baselines. Furthermore, for concept fusion, we introduce a knowledge graph-based method known as Concept Filtering, which produces over 23% point gain on neuron behaviors for neuron interpretation. At last, we propose Model Manipulation, which aims to study whether the core concepts based on ConceptNet could be employed to manipulate model behavior. The results show that core concepts can effectively improve the performance of original model by over 26%.

Event Argument Extraction (EAE) aims to extract arguments for specified events from a text.Previous research has mainly focused on addressing long-distance dependencies of arguments, modeling co-occurrence relationships between roles and events, but overlooking potential inductive biases: (i) semantic differences among arguments of the same type and (ii) large margin separation between arguments of the different types.Inspired by prototype networks, we introduce a new model named HMPEAE, which takes the two inductive biases above as targets to locate prototypes and guide the model to learn argument representations based on these prototypes.Specifically, we set multiple prototypes to represent each role to capture intra-class differences.Simultaneously, we use hypersphere as the output space for prototypes, defining large margin separation between prototypes to encourage the model to learn significant differences between different types of arguments effectively.We solve the "argument-prototype" assignment as an optimal transport problem to optimize the argument representation and minimize the absolute distance between arguments and prototypes to achieve compactness within sub-clusters.Experimental results on the RAMS and WikiEvents datasets show that HMPEAE achieves state-of-the-art performances.

Conceptual structure is fundamental to human cognition and natural language understanding. It is significant to explore whether Large Language Models (LLMs) understand such knowledge. Since FrameNet serves as a well-defined conceptual structure knowledge resource, with meaningful frames, fine-grained frame elements, and rich frame relations, we construct a benchmark for coNceptual structure induction based on FrameNet, called NutFrame. It contains three sub-tasks: Frame Induction, Frame Element Induction, and Frame Relation Induction. In addition, we utilize prompts to induce conceptual structure of Framenet with LLMs. Furthermore, we conduct extensive experiments on NutFrame to evaluate various widely-used LLMs. Experimental results demonstrate that FrameNet induction remains a challenge for LLMs.

The abstract inference capability of the Language Model plays a pivotal role in boosting its generalization and reasoning prowess in Natural Language Inference (NLI).Entailment graphs are crafted precisely for this purpose, focusing on learning entailment relations among predicates.Yet, prevailing approaches overlook the polysemy and hierarchical nature of concepts during entity conceptualization.This oversight disregards how arguments might entail differently across various concept levels, thereby missing potential entailment connections.To tackle this hurdle, we introduce the concept pyramid and propose the HiCon-EG (Hierarchical Conceptual Entailment Graph) framework, which organizes arguments hierarchically, delving into entailment relations at diverse concept levels.By learning entailment relationships at different concept levels, the model is guided to better understand concepts so as to improve its abstract inference capabilities.Our method enhances scalability and efficiency in acquiring common-sense knowledge through leveraging statistical language distribution instead of manual labeling, Experimental results show that entailment relations derived from HiCon-EG significantly bolster abstract detection tasks.

Implicit discourse relation recognition aims to identify logical relations between two arguments without explicit connectives and is a challenging task in discourse analysis. Recent methods tend to leverage the label hierarchy to enhance discourse relation representations. However, they fail to fully utilize the connective information. Specifically, the methods overlook the guiding role of connectives in discourse relation classification by treating them as the last-level labels in the label hierarchy to leverage connective information, whereas it would be more appropriate to exploit connective information prior to relation classification. Moreover, these methods ignore the dependency degree of labels between different levels in the label hierarchy. In other words, they consider the label hierarchy as an unweighted undirected graph, and assume that the path weights between high-level labels and their corresponding low-level labels are the same, which leads to an insufficient construction of the label hierarchy. To overcome these issues, we propose a method for implicit discourse relation recognition (IDRR) utilizing Connective Prediction and Dependency-weighted Label Hierarchy (CP-DLH). Experimental results on PDTB 2.0 dataset show that our model achieves the state-of-the-art performance at all hierarchical levels.

Few-shot Named Entity Recognition aims to identify named entities from unstructured texts in various domains using a minimal amount of training samples and classify them into predefined categories. Many popular approaches decompose this process into two tasks: Span Detection and Entity Classification. However, they still have some issues: (1) Neglecting presentation optimization. Most of these methods emphasize classification, neglecting the optimization of span presentation during Span Detection. (2) Missing label semantics. They have not fully leveraged semantic information of entity type labels during Entity Classification. To address these issues, this paper proposes Multi-Granularity Dual-Aware Contrastive Learning (MGDAC) for few-shot NER. Specifically, we introduce multi-granularity contrastive learning for solve neglecting presentation optimization, focusing on both the overall vector and internal vector granularity to enhance features beneficial for span detection in token vector representations. Additionally, we design dual-aware contrastive learning for solve missing label semantics, effectively utilizing semantic information from both entity tokens and entity type labels during prototype construction to jointly optimize prototype representations. Finally, extensive experiments on the FewNERD dataset demonstrate that our proposed method exhibits improvements in both Span Detection and Entity Classification, outperforming other competitive baseline methods.

In low-texture scenes, Visual Odometry (VO) algorithms often encounter challenges stemming from sparse feature sets and reduced accuracy in feature matching. To overcome this, integrating plane features and vanishing point characteristics can provide additional constraints for refining camera poses. Optical flow-based tracking methods may also offer improved matching precision compared to traditional feature-based approaches. Motivated by these challenges, we present a robust Visual Odometry system tailored for low-texture environments. Our system combines a vanishing point-based approach for camera pose optimization with a Manhattan-aided algorithm for matching line segments using optical flow. By incorporating planes and vanishing points as supplementary features for pose estimation, we enhance overall accuracy without significant time overhead. We utilize detected line features to compute vanishing points, improving accuracy without compromising efficiency. In addition, our Manhattan-aided optical flow technique supplements and refines the results of line feature matching, further enhancing the accuracy of vanishing points. Evaluation on various public datasets demonstrates the superior accuracy and robustness of our system compared to state-of-the-art Simultaneous Localization And Mapping (SLAM) and VO methods. Notably, our method effectively addresses issues of failure in low-texture scenes and improves the accuracy of line feature matching compared to baseline methods. We will release our source code upon paper acceptance.

Document-level event extraction aims to identify event types and arguments from one document. However, existing methods fail to consider semantic distinctions between multiple mentions of one entity and ignore dynamic representation of entities across multiple events simultaneously. Therefore, the models cannot capture flexible and specific entity representations in different event types. In this article, we propose EADRE ( E vent-type- A ware D ynamic R epresentation of E ntities). Specifically, we use cross-attention between mentions and event-type prototypes to obtain event-type-aware mention features. Then, we propose ASGate ( A daptive S oft G ate), which adaptively selects mention features to reduce the influence of event-unrelated mentions. EADRE introduces no more than 1% new parameters compared with the base model and has good transportability. Experiments on two public datasets show that EADRE improves the performance of multi-event extraction by 2.6% and 3.1%, as well as outperforms previous state-of-the-art baselines by 0.2% and 1.6%, with lower resource consumption without the use of pre-trained models. Further experimental analysis shows that EADRE significantly improves extraction performance in O2M and M2M multi-event scenarios.

Commonsense question answering (CQA) generally means that the machine uses its mastered commonsense to answer questions without relevant background material, which is a challenging task in natural language processing. Existing methods focus on retrieving relevant subgraphs from knowledge graphs based on key entities and designing complex graph neural networks to perform reasoning over the subgraphs. However, they have the following problems: i) the nested entities in key entities lead to the introduction of irrelevant knowledge; ii) the QA context is not well integrated with the subgraphs; and iii) insufficient context knowledge hinders subgraph nodes understanding. In this paper, we present a heterogeneous-graph reasoning with context paraphrase method (HCP), which introduces the paraphrase knowledge from the dictionary into key entity recognition and subgraphs construction, and effectively fuses QA context and subgraphs during the encoding phase of the pre-trained language model (PTLM). Specifically, HCP filters the nested entities through the dictionary's vocabulary and constructs the Heterogeneous Path-Paraphrase (HPP) graph by connecting the paraphrase descriptions11The paraphrase descriptions are English explanations of words or phrases in WordNet and Wiktionary.with the key entity nodes in the subgraphs. Then, by constructing the visible matrices in the PTLM encoding phase, we fuse the QA context representation into the HPP graph. Finally, to get the answer, we perform reasoning on the HPP graph by Mask Self-Attention. Experimental results on CommonsenseQA and OpenBookQA show that fusing QA context with HPP graph in the encoding stage and enhancing the HPP graph representation by using context paraphrase can improve the machine's commonsense reasoning ability.

Recently, knowledge graphs (KGs) have won noteworthy success in commonsense question answering.Existing methods retrieve relevant subgraphs in the KGs through key entities and reason about the answer with language models (LMs) and graph neural networks.However, they ignore (i) optimizing the knowledge representation and structure of subgraphs and (ii) deeply fusing heterogeneous QA context with subgraphs.In this paper, we propose a dynamic heterogeneous-graph reasoning method with LMs and knowledge representation learning (DHLK), which constructs a heterogeneous knowledge graph (HKG) based on multiple knowledge sources and optimizes the structure and knowledge representation of the HKG using a two-stage pruning strategy and knowledge representation learning (KRL).It then performs joint reasoning by LMs and Relation Mask Self-Attention (RMSA).Specifically, DHLK filters key entities based on the dictionary vocabulary to achieve the first-stage pruning while incorporating the paraphrases in the dictionary into the subgraph to construct the HKG.Then, DHLK encodes and fuses the QA context and HKG using LM, and dynamically removes irrelevant KG entities based on the attention weights of LM for the second-stage pruning.Finally, DHLK introduces KRL to optimize the knowledge representation and perform answer reasoning on the HKG by RMSA.We evaluate DHLK at CommonsenseQA and OpenBookQA, and show its improvement on existing LM and LM+KG methods.

Hyper-relational knowledge graphs (HKGs) extend standard knowledge graphs by associating attribute-value qualifiers to triples, which effectively represent additional fine-grained information about its associated triple. Hyper-relational knowledge graph completion (HKGC) aims at inferring unknown triples while considering its qualifiers. Most existing approaches to HKGC exploit a global-level graph structure to encode hyper-relational knowledge into the graph convolution message passing process. However, the addition of multi-hop information might bring noise into the triple prediction process. To address this problem, we propose HyperFormer, a model that considers local-level sequential information, which encodes the content of the entities, relations and qualifiers of a triple. More precisely, HyperFormer is composed of three different modules: an entity neighbor aggregator module allowing to integrate the information of the neighbors of an entity to capture different perspectives of it; a relation qualifier aggregator module to integrate hyper-relational knowledge into the corresponding relation to refine the representation of relational content; a convolution-based bidirectional interaction module based on a convolutional operation, capturing pairwise bidirectional interactions of entity-relation, entity-qualifier, and relation-qualifier. Furthermore, we introduce a Mixture-of-Experts strategy into the feed-forward layers of HyperFormer to strengthen its representation capabilities while reducing the amount of model parameters and computation. Extensive experiments on three well-known datasets with four different conditions demonstrate HyperFormer's effectiveness. Datasets and code are available at https://github.com/zhiweihu1103/HKGC-HyperFormer.

Knowledge graph entity typing (KGET) aims at inferring plausible types of entities in knowledge graphs.Existing approaches to KGET focus on how to better encode the knowledge provided by the neighbors and types of an entity into its representation.However, they ignore the semantic knowledge provided by the way in which types can be clustered together.In this paper, we propose a novel method called Multi-view Contrastive Learning for knowledge graph Entity Typing (MCLET), which effectively encodes the coarse-grained knowledge provided by clusters into entity and type embeddings.MCLET is composed of three modules: i) Multi-view Generation and Encoder module, which encodes structured information from entity-type, entity-cluster and cluster-type views; ii) Cross-view Contrastive Learning module, which encourages different views to collaboratively improve view-specific representations of entities and types; iii) Entity Typing Prediction module, which integrates multi-head attention and a Mixture-of-Experts strategy to infer missing entity types.Extensive experiments show the strong performance of MCLET compared to the state-of-the-art.

Frame-semantic Parsing (FSP) is a challenging and critical task in Natural Language Processing (NLP). Most of the existing studies decompose the FSP task into frame identification (FI) and frame semantic role labeling (FSRL) subtasks, and adopt a pipeline model architecture that clearly causes error propagation problem. However, recent jointly learning models aim to address the above problem and generally treat FSP as a span-level structured prediction task, which, unfortunately, leads to cascading error propagation problem between roles and less-efficient solutions due to huge search space of roles. To address these problems, we reformulate the FSRL task into a target-aware relation classification task and propose a novel and lightweight jointly learning framework that simultaneously processes three subtasks of FSP, including frame identification, argument identification, and role classification. The novel task formulation and jointly learning with interaction mechanisms among subtasks can help improve the overall system performance and reduce the search space and time complexity, compared with existing methods. Extensive experimental results demonstrate that our proposed model significantly outperforms 10 state-of-the-art models in terms of F1 score across two benchmark datasets.

We investigate the knowledge graph entity typing task which aims at inferring plausible entity types.In this paper, we propose a novel Transformer-based Entity Typing (TET) approach, effectively encoding the content of neighbors of an entity.More precisely, TET is composed of three different mechanisms: a local transformer allowing to infer missing types of an entity by independently encoding the information provided by each of its neighbors; a global transformer aggregating the information of all neighbors of an entity into a single long sequence to reason about more complex entity types; and a context transformer integrating neighbors content based on their contribution to the type inference through information exchange between neighbor pairs.Furthermore, TET uses information about class membership of types to semantically strengthen the representation of an entity.Experiments on two real-world datasets demonstrate the superior performance of TET compared to the state-of-the-art.

Multi-hop reasoning over real-life knowledge graphs (KGs) is a highly challenging problem as traditional subgraph matching methods are not capable to deal with noise and missing information. Recently, to address this problem a promising approach based on jointly embedding logical queries and KGs into a low-dimensional space to identify answer entities has emerged. However, existing proposals ignore critical semantic knowledge inherently available in KGs, such as type information. To leverage type information, we propose a novel type-aware model, TypE-aware Message Passing (TEMP), which enhances the entity and relation representation in queries, and simultaneously improves generalization, and deductive and inductive reasoning. Remarkably, TEMP is a plug-and-play model that can be easily incorporated into existing embedding-based models to improve their performance. Extensive experiments on three real-world datasets demonstrate TEMP’s effectiveness.

Legal judgment elements extraction (LJEE) aims to identify the different judgment features from the fact description in legal documents automatically, which helps to improve the accuracy and interpretability of the judgment results. In real court rulings, judges usually need to scan both the fact descriptions and the law articles repeatedly to find out the relevant information, and it is hard to acquire the key judgment features quickly, so legal judgment elements extraction is a crucial and challenging task for legal judgment prediction. However, most existing methods follow the text classification framework, which fails to model the attentive relations of the law articles and the legal judgment elements. To address this issue, we simulate the working process of human judges, and propose a legal judgment elements extraction method with a law article-aware mechanism, which captures the complex semantic correlations of the law article and the legal judgment elements. Experimental results show that our proposed method achieves significant improvements than other state-of-the-art baselines on the element recognition task dataset. Compared with the BERT-CNN model, the proposed “All labels Law Articles Embedding Model (ALEM)” improves the accuracy, recall, and F1 value by 0.5, 1.4 and 1.0, respectively.

Xuefeng Su, Ru Li, Xiaoli Li, Jeff Z. Pan, Hu Zhang, Qinghua Chai, Xiaoqi Han. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Sentence-level extractive text summarization aims to select important sentences from a given document. However, it is very challenging to model the importance of sentences. In this paper, we propose a novel Frame Semantic-Enhanced Sentence Modeling for Extractive Summarization, which leverages Frame semantics to model sentences from both intra-sentence level and inter-sentence level, facilitating the text summarization task. In particular, intra-sentence level semantics leverage Frames and Frame Elements to model internal semantic structure within a sentence, while inter-sentence level semantics leverage Frame-to-Frame relations to model relationships among sentences. Extensive experiments on two benchmark corpus CNN/DM and NYT demonstrate that our model outperforms six state-of-the-art methods significantly.

Recently graph-based methods have been adopted for Abstractive Text Summarization. However, existing graph-based methods only consider either word relations or structure information, which neglect the correlation between them. To simultaneously capture the word relations and structure information from sentences, we propose a novel Dual Graph network for Abstractive Sentence Summarization. Specifically, we first construct semantic scenario graph and semantic word relation graph based on FrameNet, and subsequently learn their representations and design graph fusion method to enhance their correlation and obtain better semantic representation for summary generation. Experimental results show our model outperforms existing state-of-the-art methods on two popular benchmark datasets, i.e., Gigaword and DUC 2004.

Sentence representation (SR) is the most crucial and challenging task in Machine Reading Comprehension (MRC).MRC systems typically only utilize the information contained in the sentence itself, while human beings can leverage their semantic knowledge.To bridge the gap, we proposed a novel Frame-based Sentence Representation (FSR) method, which employs frame semantic knowledge to facilitate sentence modelling.Specifically, different from existing methods that only model lexical units (LUs), Frame Representation Models, which utilize both LUs in frame and Frame-to-Frame (F-to-F) relations, are designed to model frames and sentences with attention schema.Our proposed FSR method is able to integrate multiple-frame semantic information to get much better sentence representations.Our extensive experimental results show that it performs better than state-of-the-art technologies on machine reading comprehension task.

Machine reading comprehension (MRC) is one of the most critical yet challenging tasks in natural language understanding(NLU), where both syntax and semantics information of text are essential components for text understanding.It is surprising that jointly considering syntax and semantics in neural networks was never formally reported in literature.This paper makes the first attempt by proposing a novel Syntax and Frame Semantics model for Machine Reading Comprehension (SS-MRC), which takes full advantage of syntax and frame semantics to get richer text representation.Our extensive experimental results demonstrate that SS-MRC performs better than ten state-of-the-art technologies on machine reading comprehension task.

Comprehending unstructured text is a challenging task for machines because it involves understanding texts and answering questions. In this paper, we study the multiple‐choice task for reading comprehension based on MC Test datasets and Chinese reading comprehension datasets, among which Chinese reading comprehension datasets which are built by ourselves. Observing the above‐mentioned training sets, we find that “sentence comprehension” is more important than “word comprehension” in multiple‐choice task, and therefore we propose sentence‐level neural network models. Our model firstly uses LSTM network and a composition model to learn compositional vector representation for sentences and then trains a sentence‐level attention model for obtaining the sentence‐level attention between the sentence embedding in documents and the optional sentences embedding by dot product. Finally, a consensus attention is gained by merging individual attention with the merging function. Experimental results show that our model outperforms various state‐of‐the‐art baselines significantly for both the multiple‐choice reading comprehension datasets.

Cold-start link prediction is a term for information starved link prediction where little or no topological information is present to guide the determination of whether links to a node will form. Due to the lack of topological information, traditional topology-based link prediction methods cannot be applied to solve the cold-start link prediction problem. Therefore, an effective approach is presented through establishing connections between non-topological and topological information. In the approach, topological information is first extracted by a latent-feature representation model, then a logistic model is proposed to establish the connections between topological and non-topological information, and finally the linking possibility between cold-start users and existing users is calculated. Experiments with three types of real-world social networks Weibo, Facebook, and Twitter show that the proposed approach is more effective in solving the cold-start link prediction problem and establishing connections between topological and non-topological information.

Ru Li, Juan Wu, Zhiqiang Wang, Qinghua Chai. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2015.