We introduce FinMMDocR, a novel bilingual multimodal benchmark for evaluating multimodal large language models (MLLMs) on real-world financial numerical reasoning. Compared to existing benchmarks, our work delivers three major advancements. (1) Scenario Awareness: 57.9% of 1,200 expert-annotated problems incorporate 12 types of implicit financial scenarios (e.g., Portfolio Management), challenging models to perform expert-level reasoning based on assumptions; (2) Document Understanding: 837 Chinese/English documents spanning 9 types (e.g., Company Research) average 50.8 pages with rich visual elements, significantly surpassing existing benchmarks in both breadth and depth of financial documents; (3) Multi-Step Computation: Problems demand 11-step reasoning on average (5.3 extraction + 5.7 calculation steps), with 65.0% requiring cross-page evidence (2.4 pages average). The best-performing MLLM achieves only 58.0% accuracy, and different retrieval-augmented generation (RAG) methods show significant performance variations on this task. We expect FinMMDocR to drive improvements in MLLMs and reasoning-enhanced methods on complex multimodal reasoning tasks in real-world scenarios.

Junpeng Ding, Zichen Tang, Haihong E, Mengyuan Ji, Yang Liu, Haolin Tian, Haiyang Sun, Pengqi Sun, Yang Xu, Yichen Liu, Haocheng Gao, Zijie Xi, Ruomeng Jiang, Peizhi Zhao, Rongjin Li, Yuanze Li, Jiacheng Liu, Zhongjun Yang, Jintong Chen, Siying Lin. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Bo Zhang, Tzu-Yen Ma, Zichen Tang, Junpeng Ding, Zirui Wang, Yizhuo Zhao, Peilin Gao, Zijie Xi, Zixin Ding, Haiyang Sun, Haocheng Gao, Yuan Liu, Liangjia Wang, Yiling Huang, Yujie Wang, Yuyue Zhang, Ronghui Xi, Yuanze Li, Jiacheng Liu, Zhongjun Yang, Haihong E. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Although precise recall is a core objective in Retrieval-Augmented Generation (RAG), a critical oversight persists in the field: improvements in retrieval performance do not consistently translate to commensurate gains in downstream reasoning. To diagnose this gap, we propose the Recall Conversion Rate (RCR), a novel evaluation metric to quantify the contribution of retrieval to reasoning accuracy. Our quantitative analysis of mainstream RAG methods reveals that as Recall@5 improves, the RCR exhibits a near-linear decay. We identify the neglect of retrieval quality in these methods as the underlying cause. In contrast, approaches that focus solely on quality optimization often suffer from inferior recall performance. Both categories lack a comprehensive understanding of retrieval quality optimization, resulting in a trade-off dilemma. To address these challenges, we propose comprehensive retrieval quality optimization criteria and introduce the NeocorRAG framework. This framework achieves holistic retrieval quality optimization by systematically mining and utilizing Evidence Chains. Specifically, NeocorRAG first employs an innovative activated search algorithm to obtain a refined candidate space. Then it ensures precise evidence chain generation through constrained decoding. Finally, the retrieved set of evidence chains guides the retrieval optimization process. Evaluated on benchmarks including HotpotQA, 2WikiMultiHopQA, MuSiQue, and NQ, NeocorRAG achieves SOTA performance on both 3B and 70B parameter models, while consuming less than 20% of tokens used by comparable methods. This study presents an efficient, training-free paradigm for RAG enhancement that effectively optimizes retrieval quality while maintaining high recall. Our code is released at https://github.com/BUPT-Reasoning-Lab/NeocorRAG.

Zichen Tang, Haihong E, Ziyan Ma, Haoyang He, Jiacheng Liu, Zhongjun Yang, Zihua Rong, Rongjin Li, Kun Ji, Qing Huang, Xinyang Hu, Yang Liu, Qianhe Zheng. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Multi-hop complex reasoning over incomplete knowledge graphs (KGs) has been extensively studied, but research on numerical knowledge graphs (NKGs) remains relatively limited.Recent approaches focus on separately encoding entities and numerical values, using neural networks to process query encodings for reasoning.However, in complex multi-hop reasoning tasks, numerical values are not merely symbols, and they carry specific semantics and logical relationships that must be accurately represented.In this work, we propose a Complex Numerical Reasoning with Numerical Semantic Pre-training Framework (CNR-NST).The CNR-NST framework can perform binary operations on numerical attributes in NKGs, enabling it to infer new numerical attributes from existing knowledge.Our approach effectively handles up to 102 types of complex numerical reasoning queries.On three public datasets, CNR-NST demonstrates SOTA performance in complex numerical queries, achieving an average improvement of over 40% compared to existing methods.Notably, this work expands the query types for complex multi-hop numerical reasoning and introduces a new evaluation metric for numerical answers, which has been validated through comprehensive experiments.

Temporal Knowledge Graph(TKG) serves as an efficacious way to store dynamic facts in real-world. Extrapolation reasoning on TKGs, which aims at predicting possible future events, has attracted consistent research interest. Recently, some rule-based methods have been proposed, which are considered more interpretable compared with embedding-based methods. Existing rule-based methods apply rules through path matching or subgraph extraction, which falls short in inference ability and suffers from missing facts in TKGs. Besides, during rule application period, these methods consider the standing of facts as a binary 0 or 1 problem and ignores the validity as well as frequency of historical facts under temporal settings. In this paper, by designing a novel paradigm for rule application, we propose INFER, a neural-symbolic model for TKG extrapolation. With the introduction of Temporal Validity Function, INFER firstly considers the frequency and validity of historical facts and extends the truth value of facts into continuous real number to better adapt for temporal settings. INFER builds Temporal Weight Matrices with a pre-trained static KG embedding model to enhance its inference ability. Moreover, to facilitates potential integration with existing embedding-based methods, INFER adopts a rule projection module which enables it apply rules through conducting matrices operation on GPU. This feature also improves the efficiency of rule application. Experimental results show that INFER achieves state-of-the-art performance on various TKG datasets and significantly outperforms existing rule-based models on our modified, more sparse TKG datasets, which demonstrates the superiority of our model in inference ability.

Knowledge Base Question Answering (KBQA) aims to answer natural language questions with a large-scale structured knowledge base (KB). Despite advancements with large language models (LLMs), KBQA still faces challenges in weak KB awareness, imbalance between effectiveness and efficiency, and high reliance on annotated data. To address these challenges, we propose KBQA-o1, a novel agentic KBQA method with Monte Carlo Tree Search (MCTS). It introduces a ReAct-based agent process for stepwise logical form generation with KB environment exploration. Moreover, it employs MCTS, a heuristic search method driven by policy and reward models, to balance agentic exploration’s performance and search space. With heuristic exploration, KBQA-o1 generates high-quality annotations for further improvement by incremental fine-tuning. Experimental results show that KBQA-o1 outperforms previous low-resource KBQA methods with limited annotated data, boosting Llama-3.1-8B model’s GrailQA F1 performance to 78.5% compared to 48.5% of the previous sota method with GPT-3.5-turbo. Our code is publicly available.

Standard Retrieval-Augmented Generation (RAG) relies on chunk-based retrieval, whereas GraphRAG advances this approach by graph-based knowledge representation. However, existing graph-based RAG approaches are constrained by binary relations, as each edge in an ordinary graph connects only two entities, limiting their ability to represent the n-ary relations (n >= 2) in real-world knowledge. In this work, we propose HyperGraphRAG, the first hypergraph-based RAG method that represents n-ary relational facts via hyperedges. HyperGraphRAG consists of a comprehensive pipeline, including knowledge hypergraph construction, retrieval, and generation. Experiments across medicine, agriculture, computer science, and law demonstrate that HyperGraphRAG outperforms both standard RAG and previous graph-based RAG methods in answer accuracy, retrieval efficiency, and generation quality.

Intradialytic hypotension (IDH) is a common complication among hemodialysis patients, adversely affecting quality of life and elevating mortality risk. IDH prediction enables physicians to take proactive measures, effectively reducing its occurrence. However, most prediction works rely on machine learning models, with a focus on real-time or session-level IDH. Hemodialysis patient data is multi-type and temporal, necessitating research on patient condition representation and temporal information utilization. Knowledge graphs (KGs) offer flexible data modeling and encompass rich structured information. This study represents patients using KGs and reason on graph structures to predict IDH. To study monthly IDH and utilize temporal information, a temporal patient KG is constructed. Patient KGs are first built at the monthly granularity based on data of 532 patients between January 2017 and August 2022. Six sequential monthly KGs are then combined into an observation window, resulting in a temporal KG dataset of 15,807 independent windows from 458 patients. The aim of this study is to utilize information from multiple months within a window to predict frequent IDH in the last month. However, the characteristics of IDH scenario and generalizability requirement pose challenges for the application of general KG reasoning models. Therefore, we adopt neighborhood-based KG reasoning and devise a visible feature guided patient-centric graph convolution to obtain patients' generalizable representations. Finally, patient representations in a window are fused using a sequential model, and processed by a prediction MLP to obtain the prediction results. Compared to 7 classic machine learning models, our model demonstrates superior performance in comprehensive metrics such as accuracy and F1 score.

Beyond traditional binary relational facts, n-ary relational knowledge graphs (NKGs) are comprised of n-ary relational facts containing more than two entities, which are closer to real-world facts with broader applications. However, the construction of NKGs remains at a coarse-grained level, which is always in a single schema, ignoring the order and variable arity of entities. To address these restrictions, we propose Text2NKG, a novel fine-grained n-ary relation extraction framework for n-ary relational knowledge graph construction. We introduce a span-tuple classification approach with hetero-ordered merging and output merging to accomplish fine-grained n-ary relation extraction in different arity. Furthermore, Text2NKG supports four typical NKG schemas: hyper-relational schema, event-based schema, role-based schema, and hypergraph-based schema, with high flexibility and practicality. The experimental results demonstrate that Text2NKG achieves state-of-the-art performance in F1 scores on the fine-grained n-ary relation extraction benchmark. Our code and datasets are publicly available.

Landmark-based 3-hop cover labeling is a category of approaches for shortest distance/path queries on large-scale complex networks. It pre-computes an index offline to accelerate the online distance/path query. Most real-world graphs undergo rapid changes in topology, which makes index maintenance on dynamic graphs necessary. So far, the majority of index maintenance methods can handle only one edge update (either an addition or deletion) each time. To keep up with frequently changing graphs, we research the ful ly b atch m aintenance problem for the 3-hop cover labeling, and proposed the method called FulBM . FulBM is composed of two algorithms: InsBM and DelBM, which are designed to handle batch edge insertions and deletions, respectively. This separation is motivated by the insight that batch maintenance for edge insertions are much more time-efficient and the fact that most edge updates in the real world are incremental. Both InsBM and DelBM are equipped with well-designed pruning strategies to minimize the number of vertex accesses. We have conducted comprehensive experiments on both synthetic and real-world graphs to verify the efficiency of FulBM and its variants for weighted graphs. The results show that our methods achieve 5.5× to 228× speedup compared with the state-of-the-art method.

In the field of representation learning on knowledge graphs (KGs), a hyper-relational fact consists of a main triple and several auxiliary attribute-value descriptions, which is considered more comprehensive and specific than a triple-based fact. However, currently available hyper-relational KG embedding methods in a single view are limited in application because they weaken the hierarchical structure that represents the affiliation between entities. To overcome this limitation, we propose a dual-view hyper-relational KG structure (DH-KG) that contains a hyper-relational instance view for entities and a hyper-relational ontology view for concepts that are abstracted hierarchically from the entities. This paper defines link prediction and entity typing tasks on DH-KG for the first time and constructs two DH-KG datasets, JW44K-6K, extracted from Wikidata, and HTDM based on medical data. Furthermore, we propose DHGE, a DH-KG embedding model based on GRAN encoders, HGNNs, and joint learning. DHGE outperforms baseline models on DH-KG, according to experimental results. Finally, we provide an example of how this technology can be used to treat hypertension. Our model and new datasets are publicly available.

Complex query answering (CQA) is an essential task for multi-hop and logical reasoning on knowledge graphs (KGs). Currently, most approaches are limited to queries among binary relational facts and pay less attention to n-ary facts (n≥2) containing more than two entities, which are more prevalent in the real world. Moreover, previous CQA methods can only make predictions for a few given types of queries and cannot be flexibly extended to more complex logical queries, which significantly limits their applications. To overcome these challenges, in this work, we propose a novel N-ary Query Embedding (NQE) model for CQA over hyper-relational knowledge graphs (HKGs), which include massive n-ary facts. The NQE utilizes a dual-heterogeneous Transformer encoder and fuzzy logic theory to satisfy all n-ary FOL queries, including existential quantifiers (∃), conjunction (∧), disjunction (∨), and negation (¬). We also propose a parallel processing algorithm that can train or predict arbitrary n-ary FOL queries in a single batch, regardless of the kind of each query, with good flexibility and extensibility. In addition, we generate a new CQA dataset WD50K-NFOL, including diverse n-ary FOL queries over WD50K. Experimental results on WD50K-NFOL and other standard CQA datasets show that NQE is the state-of-the-art CQA method over HKGs with good generalization capability. Our code and dataset are publicly available.

Haoran Luo, Haihong E, Yuhao Yang, Yikai Guo, Mingzhi Sun, Tianyu Yao, Zichen Tang, Kaiyang Wan, Meina Song, Wei Lin. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

Multi-hop logical reasoning over knowledge graph plays a fundamental role in many artificial intelligence tasks. Recent complex query embedding methods for reasoning focus on static KGs, while temporal knowledge graphs have not been fully explored. Reasoning over TKGs has two challenges: 1. The query should answer entities or timestamps; 2. The operators should consider both set logic on entity set and temporal logic on timestamp set. To bridge this gap, we introduce the multi-hop logical reasoning problem on TKGs and then propose the first temporal complex query embedding named Temporal Feature-Logic Embedding framework (TFLEX) to answer the temporal complex queries. Specifically, we utilize fuzzy logic to compute the logic part of the Temporal Feature-Logic embedding, thus naturally modeling all first-order logic operations on the entity set. In addition, we further extend fuzzy logic on timestamp set to cope with three extra temporal operators (**After**, **Before** and **Between**). Experiments on numerous query patterns demonstrate the effectiveness of our method.

Youri Xu, Haihong E, Meina Song, Wenyu Song, Xiaodong Lv, Wang Haotian, Yang Jinrui. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

Abstract Natural language inference (NLI) is the basic task of many applications such as question answering and paraphrase recognition. Existing methods have solved the key issue of how the NLI model can benefit from external knowledge. Inspired by this, we attempt to further explore the following two problems: (1) how to make better use of external knowledge when the total amount of such knowledge is constant and (2) how to bring external knowledge to the NLI model more conveniently in the application scenario. In this paper, we propose a novel joint training framework that consists of a modified graph attention network, called the knowledge graph attention network, and an NLI model. We demonstrate that the proposed method outperforms the existing method which introduces external knowledge, and we improve the performance of multiple NLI models without additional external knowledge.

A spoken language understanding (SLU) system includes two main tasks, slot filling (SF) and intent detection (ID).The joint model for the two tasks is becoming a tendency in SLU.But the bi-directional interrelated connections between the intent and slots are not established in the existing joint models.In this paper, we propose a novel bi-directional interrelated model for joint intent detection and slot filling.We introduce an SF-ID network to establish direct connections for the two tasks to help them promote each other mutually.Besides, we design an entirely new iteration mechanism inside the SF-ID network to enhance the bi-directional interrelated connections.The experimental results show that the relative improvement in the sentence-level semantic frame accuracy of our model is 3.79% and 5.42% on ATIS and Snips datasets, respectively, compared to the state-of-the-art model.

A mainstream task-oriented dialogue system is stuck in the independence of modules since it follows pipeline design, and it also suffers from sequence dependence and time dependence of recurrent neural network (RNN). Thus, these systems are complicated and usually trained slowly. In this paper, we propose FPSeq, a novel, fully parallel framework to simplify and accelerate task-oriented dialogue systems. Specifically, FPSeq turns pipeline design into a single sequence-to-sequence (seq2seq) model to achieve integration of each module for end-to-end training. In addition, multi-layer convolutional neural networks (CNNs) and attention mechanisms are applied in seq2seq learning to achieve parallel computations for speed improvement. Compared to the existing best model, the training speed of FPSeq is 3-10 times faster while only one-third of the number of parameters. Experimental results on CamRest676 and KVRET datasets indicate that FPSeq achieves the state-of-the-art performance in both task completion and quality of language generation.

In a recommender system, items can be rated across multiple fields by users with varying degrees of familiarity. Hence, the ratings in a recommender system should have different recommended weights. Ratings in fields where in the user has high or low familiarity should be given high or low recommended weights, respectively. However, current recommendation algorithms ignore this problem and use the ratings indiscriminately, thus affecting the accuracy of the recommendation system. In this paper, we provide a focused study of user-familiarity degree-aware recommendation and develop a user-familiarity degree-aware latent factor model for recommendations that considers both user familiarity and item features reflected by the tagging information. We also design a user-familiarity degree-aware probability matrix factorization model, which computes the degree of familiarity of a user with the items he/she has rated. By using the user-familiarity degree, different recommended weights are given to every rating to obtain precise recommendations. The experiment results on real-world datasets show that our algorithm significantly outperforms state-of-the-art latent factor models and effectively improves the accuracy of the recommendation results.

Cloud computing can help data center consolidate batch and gratis tasks with over-provisioned production applications, and fulfill their diverse resource demands and performance objectives with high scalability and flexibility. One challenge in this hybrid data center is that the dramatic fluctuation of batch and gratis workload may impact performance of production applications, cause task failure, decrease efficiency, and waste computing resources. One way to tackle the challenge is to reduce resource allocation to prevent host overload by delay scheduling tasks if resources are predicted in short. In this paper, we propose hierarchical prediction method for hybrid workload. We use last-state based ARMA model to predict stationary process of production workload, and use feedback based online AR model to predict the vibrated workload of batch and gratis tasks. Evaluation shows that the hierarchical prediction based task scheduling can reduce host overload by more than 85 percent, reduce tasks evicted and killed by more than 60 percent, and reduce 40 percent of average task scheduling delay.

The traditional database technology can't meet the rapidly growing information and the demands of extreme scalability, high availability and reliability for the mass data. In this paper, we have designed a distributed storage system based MPP(massively parallel processing) architecture on RDBMS to solve these problems. First, we present the main framework and the function architecture of the design. Then we do some experiments and performance tests. At last, we conclude that with the good scalability and massively parallel processing advantage, the system can solve the mass data storage problem. The idea of distributed storage system based MPP architecture in relational databases speeds up reading and writing, improving the shortcomings of traditional database.

At present the service science and engineering research are mainly focus on service discovery, service composition, service reputation and other key technical of the service computing. However the basic theory of service science, in particular, the basic principle of service and the evolution mechanism of service network only have some preliminary of research and exploration. The atomic services as the network nodes and the relationships of services combination as the network edges constitute the service network based services provision environment. This paper researched the basic characteristics of services and service networks, and proposed a new research method to explore the service network's "small world", "scale-free" characteristics and service network topology, based on the theory of complex network and existing networked software research works. It presents a new research perspective and methods for the service science and engineering research.