Jianqi Gao, Hang Yu, Jian Cao, Ranran Bu, Jinghua Tang, Nengjun Zhu, Yonggang Zhang. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Knowledge base question answering (KBQA) aims to answer natural language questions using large-scale knowledge bases (KBs). Among various KBQA approaches, semantic parsing-based (SP-based) methods have demonstrated strong effectiveness by generating concise logical forms (LFs) that capture complex subgraph structures and semantic information. Recent research suggests that integrating large language models (LLMs) with SP can achieve significant improvements in the performance and efficiency of KBQA by facilitating the direct generation of LFs with minimal retrieval. However, generating complete LFs with LLMs continues to pose a challenge due to the complexity of the required graph structures and constraints, leading to the significant issue of non-executability. To address these challenges, we propose GCA-KBQA, a step-wise fine-tuned LLM-based framework that employs hop-wise generation, knowledge-assisted calibration, and path-level assembly to construct complete LFs for KBQA. Specifically, we decompose the complex SP process into manageable steps: first, we iteratively generate LFs for each topic entity one hop at a time using a fine-tuned LLM, leveraging KB knowledge to calibrate intermediate outputs and mitigate error propagation. Subsequently, we guide the LLM in assembling path-level LFs from different topic entities, resulting in optimized final LF. We evaluate the proposed method on four KBQA benchmarks spanning two distinct KBs, demonstrating its superior performance compared to state-of-the-art baselines. The code is available at https://github.com/pvfeldt/GCA-KBQA.

As users' preferences evolve over time, personalized online services increasingly rely on sequential recommender systems to predict future interactions by modeling patterns in historical user behavior. However, existing methods for sequential recommendation (SR) face two key challenges: they struggle to simultaneously leverage collaborative, semantic, and rating information, and the use of hard labels during training provides limited supervision. In this paper, we introduce SEAR, an LLM-powered Sequential recommEndation framework via fusion of collAborative, semantic, and Rating information. The proposed deep model comprises an embedding layer and a sequence encoder. The embedding layer transforms user-item interactions into three types of embeddings: collaborative, semantic, and rating. The sequence encoder then integrates these embeddings and identifies sequential patterns to model user representations. To enhance the utilization of item semantics, we integrate a large language model (LLM) to extract LLM embeddings. These embeddings are then employed to initialize the semantic embedding layer, collaborative embedding layer, and item embeddings. To capture more nuanced user behavior patterns, we generate preference-weighted soft labels based on the next k interactions. Extensive experiments validate the effectiveness of SEAR, and ablation studies further highlight the distinct contributions of the collaborative, semantic, and rating information.

Knowledge base question answering (KBQA) refers to the task of answering natural language questions using factual information from large-scale knowledge bases (KBs). To obtain accurate answers, recent research optimizes semantic parsing methods, a major KBQA approach, with large language models (LLMs), where concise logical forms (LFs) are generated by LLMs and executed in KBs. Although these methods demonstrate superior performance, they still encounter the problem that some generated LFs fail to yield answers when executed, significantly limiting their effectiveness. To mitigate this issue, we propose KARV, a Knowledge-Assisted reasoning path Reconstruction and hierarchical Voting approach for non-executable LFs. This method extracts semantic knowledge from KBs as guidance to correct and reconstruct reasoning paths, deriving answers through a voting-based strategy. The insight is that non-executable LFs generated by LLMs still contain rich semantic information, and the knowledge retrieved from KBs can effectively correct them. Specifically, we fine-tune LLMs to generate high-quality LFs, and the nonexecutable LFs are decomposed into multiple path branches based on mentioned entities. Semantic knowledge from KBs is then leveraged to correct the entities and relations within these branches, effectively reconstructing the reasoning paths. To obtain precise final answers, we apply a hierarchical voting strategy both within and across the non-executable LFs. Our proposed method achieves state-of-the-art performance on benchmarks including WebQuestionSP (WebQSP), ComplexWebQuestions (CWQ), and FreebaseQA.

Knowledge Base Question Answering (KBQA) aims to answer natural language questions by reasoning over large-scale structured Knowledge Bases (KBs). Among existing approaches, semantic parsing-based methods have emerged as a mainstream solution, where Large Language Models (LLMs) are employed to translate questions into structured graph queries such as Logical Forms (LFs). However, this paradigm faces two critical challenges: (1) The complex semantic mapping and graph retrieval operations render direct one-shot LF generation difficult; (2) LLMs suffer from inherent hallucination issues, generating semantically plausible-seeming but factually incorrect or invalid LFs, which are non-executable. To address these challenges, this article proposes HP-Corr , a novel framework that integrates H op-wise P lanning with iterative explainable self- Corr ection for faithful knowledge reasoning. Specifically, the system utilizes a fine-tuned open source LLM for query planning and explainable self-correction. The query planner generates reasoning paths hop-by-hop, while an explainable self-correction provides hop-wise feedback, enabling interpretable path editing based on existing reasoning paths and retrieved KB knowledge. By introducing the dual-module cooperative architecture, our system performs iterative plan-then-correct to refine query paths progressively, ensuring answer reliability and LFs executability. Experimental results demonstrate significant improvements, with our approach achieving higher accuracy while substantially reducing the search space, particularly in complex multi-hop KBQA scenarios.

Detecting anomalies in business processes is imperative for achieving operational success, especially as multiple participants increasingly engage in collaborative efforts to complete processes, i.e., collaborative business processes, amidst rapid economic development. However, existing business process anomaly detection approaches are typically designed for centralized training, making them impractical for collaborative business processes that require stringent privacy measures. In this paper, we introduce a feDerated Unsupervised AnomaLy detection framework for collaborative business processes, named DUAL. DUAL enables participants to collaboratively detect anomalies via a third-party coordinator, which constructs a global view from the hidden representations of participants' local sub-traces without exposing the sub-traces themselves. To further strengthen privacy protection, DUAL injects differential privacy noise into the transmitted hidden representations. To mitigate the global-local discrepancy, we introduce novel event execution errors which indicate whether the client who is supposed to execute the reconstructed event is consistent with the observed one. Extensive experiments demonstrate that DUAL effectively detects anomalies in collaborative processes while preserving privacy, achieving performance comparable to a centralized setting that requires access to participants' raw sub-traces.

Knowledge base question answering (KBQA) refers to the system that produces answers to user queries by reasoning with a large-scale structured knowledge base. Advanced works have achieved great success either by generating logical forms (LF) or directly generating answers. Although the former typically yields better performance, these generated LF could be inaccurate, e.g., non-executable. In this regard, large language models (LLMs) have shown exciting potential for accurate generation. However, it is challenging to fine-tune LLMs to generate LF. This is because the context retrieved for prediction typically leads to an excessive number of reasoning paths. In this context, LLMs can generate numerous LF corresponding to these reasoning paths, but a few LF can result in correct answers. Thus, fine-tuning LLMs to generate answer-relevant LF would conflict with the prior knowledge of the LLMs. In this work, we propose a novel learning framework, FM-KBQA, to fine-tune LLMs using multi-task learning for KBQA. Specifically, we propose to fine-tune LLMs using an additional objective: generating the index of reasoning paths that lead to correct answers. This will direct LLMs to pay attention to answer-relevant paths among numerous reasoning paths by completing a simple task where the selected reasoning paths can be supplementary for non-executable LF. Directly generating answers can make LLMs pay attention to the answer-relevant reasoning paths, but it is much more challenging than generating the index of reasoning paths. To verify FM-KBQA's effectiveness, we conduct experiments on mainstream benchmarks, such as WebQuestionsSP (WQSP) and ComplexWebQuestions (CWQ). Extensive evaluations across two public benchmark datasets underscore the superiority of FM-KBQA over current state-of-the-art methods.

Detecting anomalies in business processes is crucial for ensuring operational success. While many existing methods rely on statistical frequency to detect anomalies, it's important to note that infrequent behavior doesn't necessarily imply undesirability. To address this challenge, detecting anomalies from a semantic viewpoint proves to be a more effective approach. However, current semantic anomaly detection methods treat a trace (i.e., process instance) as multiple event pairs, disrupting long-distance dependencies. In this paper, we introduce DABL, a novel approach for detecting semantic anomalies in business processes using large language models (LLMs). We collect 143,137 real-world process models from various domains. By generating normal traces through the playout of these process models and simulating both ordering and exclusion anomalies, we fine-tune Llama 2 using the resulting log. Through extensive experiments, we demonstrate that DABL surpasses existing state-of-the-art semantic anomaly detection methods in terms of both generalization ability and learning of given processes. Users can directly apply DABL to detect semantic anomalies in their own datasets without the need for additional training. Furthermore, DABL offers the ability to interpret anomalies' causes in natural language, providing valuable insights into the detected anomalies.

Graph-based fraud detection has emerged as a pivotal tool in risk management, leveraging the power of graph neural networks to enhance node representations by aggregating information from neighboring nodes. However, this aggregation process can sometimes introduce noise by incorporating neighbors from different categories, potentially diluting the central node’s representation. To tackle this issue, we introduce an innovative Adversarial Confidence Reweighting (ACR) technique designed to allocate discriminative weights to samples automatically. This approach effectively minimizes the impact of noisy neighbors on the representation of nodes. By introducing controlled adversarial perturbations to the nodes being classified, we can assess the extent of representation dilution. Furthermore, we employ an adaptive node re-weighted learning objective, which dynamically adjusts node weights based on a confidence measure derived from prediction accuracy. Our experimental evaluations across three public datasets demonstrate that our adversarial strategy significantly surpasses the baseline model in terms of detection performance.

In recent years, leveraging pre-trained language models (PLMs) to generate embeddings for downstream tasks has achieved remarkable success. To further enhance the adaptability of PLMs to downstream tasks, the prevailing strategy involves incorporating auxiliary tasks as regularization terms, (e.g., contrastive learning), for fine-tuning the pre-trained models. However, this approach encounters challenges due to task conflicts between auxiliary tasks and specific downstream tasks. To overcome these issues, we introduce a novel strategy termed Consistency Adversarial Training (CAT). CAT first dynamically identifies the most inconsistent cases between specific and auxiliary tasks by introducing perturbations and then eliminates the inconsistency in an adversarial learning manner. Performance evaluations on the GLUE benchmark demonstrate that CAT minimizes task conflicts during the fine-tuning process of PLMs, leading to a notable improvement in the overall fine-tuning performance of PLMs.

The large-scale knowledge base question-answering (KBQA) has become increasingly vital across various fields. In the era of large language models (LLMs), leveraging knowledge base retrieval combined with large models for knowledge reasoning has become the mainstream approach for KBQA. However, this method faces two primary challenges: (1) the high computational cost and low accuracy of similarity-based path retrieval, and (2) the relatively low accuracy of directly obtaining answers from large models. In this paper, we introduce a novel method of retrieval enhancement-stepwise reasoning (RESR), which transforms path retrieval into text semantic understanding to minimize unnecessary interference from path information in the reasoning process, guiding the LLM to generate interpretable reasoning paths rather than directly producing answers. Specifically, RESR fine-tunes the generative model through text semantic understanding to swiftly and accurately filter path information relevant to the query from a large-scale knowledge graph (KG). Additionally, we employ the Chain-of-Thought (CoT) method to guide LLMs in step-by-step reasoning, verifying the logical coherence of reasoning paths rather than directly deriving answers. Our proposed method achieves state-of-the-art (SOTA) performance on the WebQuestionsSP (WQSP) and ComplexWebQuestions (CWQ) benchmarks.

In graph-based fraud detection, accurately distinguishing between normal and anomalous account nodes is crucial. Recent graph-based approaches have achieved remarkable progress, primarily by aggregating neighborhood information to learn node representations and enhance model discriminability. However, most existing methods employ a unified aggregation function to neighbors with different class labels, which weakens feature differences and allows anomalous node features to be obscured by normal ones, leading to representation contamination. Moreover, existing methods assign uniform confidence weights to all labeled nodes during training, without accounting for label quality differences caused by varying distances to the decision boundary, which weakens the robustness of model learning. To overcome these limitations, we propose a novel method named Confidence-Aware Neighborhood Aggregation (CANA). CANA introduces a category-aware aggregation mechanism that performs separate aggregation operations for normal, anomalous, and unlabeled neighbors, thereby mitigating the issue of representation contamination. In addition, CANA incorporates a confidence-aware node-reweighted learning objective function, which dynamically adjusts the confidence weights of the labeled nodes based on a confidence metric computed through controlled perturbations in neighbors and prediction accuracy, thus improving the model's ability to leverage high quality labeled nodes and improving its learning capability and classification performance. Specifically, the function includes two optional perturbation strategies, namely Homo-Hetero Neighbor Perturbation (HHNP) and Robust Neighbor Perturbation (RNP). Extensive evaluations of four public benchmark datasets underscore the superiority of CANA over current state-of-the-art methods.

Fine-tuning pre-trained language models (PLMs) for downstream tasks has achieved remarkable success in various natural language processing (NLP) applications. To further enhance the overall performance of PLMs across different NLP tasks, we propose a novel adversarial contrastive training (ACT) method that incorporates adversarial perturbations in the parameter space. Specifically, ACT introduces adversarial perturbations to the model’s parameters, deliberately degrading its performance on downstream tasks. Subsequently, we apply contrastive learning to align the representations of specific tasks with those of the perturbed model, thereby improving the model’s robustness and generalization ability. Experimental results demonstrate that ACT significantly enhances the generalization performance of PLMs and achieves state-of-the-art results on several text classification benchmarks.

There are a large number of text processing tasks in web applications, such as sentiment classification, summary extraction, and question answering. Recently, fine-tuning pre-trained language models (PLMs) to adapt to downstream text-processing tasks has attracted much attention. However, due to the differences in data, model, and tasks between the pre-training and fine-tuning processes, the fine-tuning process may suffer from catastrophic forgetting of pre-training knowledge, which may implicitly limit the model's performance and generalization ability. To address these challenges, we propose a novel dual-model framework, termed as consistency alignment (CoAi). The insight of CoAi lies in building an auxiliary model that simulates the distribution of pre-training knowledge in real-time according to the current task, and co-training the task-specific model and the auxiliary model to balance the pre-training knowledge and task-specific knowledge during fine-tuning. Specifically, the auxiliary model is constructed on-the-fly to maintain the pre-training knowledge. Subsequently, CoAi simulates the pre-training process by performing distributional exploration in the parameter space, which is built upon our novel insight into the transformation between data and model parameter space. However, the objectives leveraged to construct the auxiliary model lead to the misalignment between the pre-training and task-specific knowledge. To alleviate the inconsistency, we employ an auxiliary variable to align the prediction distribution of the task-specific and the auxiliary models, inspired by constrastive clustering. We validate the effectiveness of CoAi on nine classic classification tasks and three generation tasks, showing consistent and significant improvements compared with state-of-the-art methods.

Graph-based fraud detection has gained significant attention due to its effectiveness in risk management. Recent approaches utilizing graph neural networks highlight the benefits of representing nodes through aggregating homogeneous neighbors. However, in real-world situations, fraudsters disguise their activities by forming connections with benign entities, leading to limitations in the homogeneous aggregation method. Because combining representations from neighborhoods of different categories can dilute the effectiveness of the aggregation. On the other hand, some methods neglect quality of labels and introduce uniform weights for all training nodes, producing challenges for model robustness. To tackle these challenges, we propose a robust model for graph-based fraud detection that leverages adversarial reweighting to improve neighborhood aggregation. In graph-based fraud detection, account nodes can be classified into two types: normal and abnormal. Our model enables dual encoders to assimilate information from both normal and abnormal neighbors, respectively, ensuring resilience against different classes of neighbors. For neighbors without labels, we employ pseudolabels to ascertain the weights for aggregated features across different category spaces. This process still introduces information contamination due to the quality of labeled nodes. To mitigate this issue, we introduce perturbation strategies based on edge structures and node features. These strategies dynamically adjust node weights during training based on the accuracy of model predictions after the introduction of perturbation. This adjustment enhances the contribution of reliable nodes in the loss function, ultimately improving the model’s detection capabilities. Our comprehensive evaluation across four public benchmark datasets-Amazon, Yelp, T-Finance, and T-Social-demonstrates that our proposed method outperforms current state-of-the-art techniques.

Abstract Causal event extraction (CEE) aims to identify and extract cause‐effect event pairs from texts, which is a fundamental task in natural language processing. Recent research treat CEE as a sequence labeling problem. However, the linguistic complexity and ambiguity of textual description results in the low accuracy of extractors. To address the above issues, considering the prior knowledge like the causal network constructed based on the causal indicators, which can represent information transition between cause and effect, may helpful for CEE. In this article, we propose causality‐associated graph neural network to incorporate in‐domain knowledge by taking important causal words into account. External causal knowledge is modeled as causal associated graph (CAG). Then we use graph neural networks (GNN) to capture the complex relationship of intraevent mentions and interevent causality in a sentence based on the relationship obtained from CAG. Finally, sentence sequence and prior causal knowledge of GNN embedding are fed into multiscaled convolution and bidirectional long short‐term memory networks. Experimental results on two datasets show that our method outperforms the state‐of‐the‐art baseline.

Abstract Event causality extraction is a challenging task in natural language processing (NLP), which plays an important role in event prediction, scene generation, question answering and textual entailment. Most existing methods focus on extracting single‐scale (such as phrase) event causality, while fails to extract multi‐scale (such as word, phrase, sentence) event causality. To fill the gap, we propose multi‐scale event causality extraction via simultaneous knowledge‐attention and convolutional neural network (KA‐CNN). First, knowledge‐attention takes N‐gram embedding as input and takes semantic features, fused with prior knowledge through causal associative link network (CALN), as output. Second, multi‐scale CNN is designed with word embedding as input and semantic feature of corpus as output. Third, bidirectional long short‐term memory with conditional random field (BiLSTM + CRF) is conducted after concatenation of features from knowledge‐attention and multi‐scale CNN. Finally, we compare our results with other baselines. The experimental results show that our proposed method shows promising result in extracting multi‐scale event causality.

Causal Event Extraction (CEE) is a joint extraction task of events and causality, which can help text understanding, event prediction and so on. Recent research has achieved state-of-the-art performance in various Natural Language Processing (NLP) tasks by combining pre-trained models with neural networks. However, ambiguity of event description and long-distance dependence of event causality result in the low accuracy of extractors. In this paper, we propose a model to incorporate in-domain knowledge by taking frequent expression of event causality into account, and use iterated dilated convolutions to expand the perception field of event causality. External causal knowledge is modeled as frequent n-grams with different length, which is used as convolution filters during kernel initialization, enhancing the ability of model to capture the boundary of event description. To obtain long-distance dependence of event causality, we use iterated dilated convolutions to aggregate context from the entire sentence. Experimental results show that our method significantly outperform the baselines with faster convergence speed.

Summary Event evolutionary graph (EEG) reflects sequential and causal relations between events, which is of great value for event prediction. However, lacking event context in the EEG raises the problems of direction uncertainty and low accuracy when making predictions. In this article, we propose a conditional event evolutionary graph (CEEG) to deal with these problems. CEEG extends EEG with an additional four types of event context, including state, cause, sub‐type, and object. We first extract event context by matching the input with self‐adaptive semantic templates and generalize the context for each event. To identify the evolution direction, we treat it as a binary classification problem and calculate the event transition probability for each direction given the generalized context. Experimental results show that CEEG has a strong ability to generate better event evolutionary paths compared with NAR, EEM, and other non‐context‐based methods.