Retrieval-augmented generation (RAG) provides an efficient solution for expanding the knowledge boundaries of large language models (LLMs), where the indexing serves as a compass to guide LLMs in locating query-relevant external knowledge. Nevertheless, current indexing methods commonly encounter a critical challenge: native indexing is convenient to construct, but it usually disrupts contextual associations and constrains the expressive capacity of rich knowledge. Conversely, knowledge indexing can structure contextual knowledge, but it is often based on preset schemas that limit its generalizability. To address it, we propose a universal and flexible knowledge indexing called pseudo-graph (PG) indexing. During the indexing construction phase, we use the advanced LLMs to transform the knowledge of each raw text into a concise and structured mind map, organizing intra-document knowledge. Subsequently, independent mind maps are linked by associating highly relevant topics or consistent facts across documents, thereby establishing inter-document knowledge connections. Eventually, using the resulting knowledge network PG as the knowledge indexing can circumvent the challenges associated with schema design reliant on preset knowledge and relationship types. During the knowledge retrieval phase, we develop a PG knowledge retriever to mimic human note-reviewing, adaptively navigating and recalling query-relevant knowledge from PG. Experimental results demonstrate that retrieving relevant pseudo-subgraphs from the PG via PG indexing and retriever significantly improves performance in fact-based Q&A, hallucination correction, and two multi-document Q&A tasks, achieving$F1_{QE}$improvements of 15.85%, 8.12%, 3.34%, and 5.73%, respectively, and outperforming the state-of-the-art baseline KGP-LLaMA. Our code is available at:https://github.com/IAAR-Shanghai/PGRAG.

The significance of Temporal Knowledge Graphs (TKGs) in Artificial Intelligence (AI) lies in their capacity to incorporate time-dimensional information, support complex reasoning and prediction, optimize decision-making processes, enhance the accuracy of recommendation systems, promote multimodal data integration, and strengthen knowledge management and updates. This provides a robust foundation for various AI applications. To effectively learn and apply both static and dynamic temporal patterns for reasoning, a range of embedding methods and large language models (LLMs) have been proposed in the literature. However, these methods often rely on a single underlying embedding space, whose geometric properties severely limit their ability to model intricate temporal patterns, such as hierarchical and ring structures. To address this limitation, this paper proposes embedding TKGs into projective geometric space and leverages LLMs technology to extract crucial temporal node information, thereby constructing the 5EL model. By embedding TKGs into projective geometric space and utilizing Möbius Group transformations, we effectively model various temporal patterns. Subsequently, LLMs technology is employed to process the trained TKGs. We adopt a parameter-efficient fine-tuning strategy to align LLMs with specific task requirements, thereby enhancing the model's ability to recognize structural information of key nodes in historical chains and enriching the representation of central entities. Experimental results on five advanced TKG datasets demonstrate that our proposed 5EL model significantly outperforms existing models.

Xun Liang, Simin Niu, Zhiyu Li, Sensen Zhang, Hanyu Wang, Feiyu Xiong, Zhaoxin Fan, Bo Tang, Jihao Zhao, Jiawei Yang, Shichao Song, Mengwei Wang. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Retrieval-augmented citation generation (RACG) helps users trust the large language model output by retrieving evidence from reliable sources. However, most current RACG research focuses on single-language tasks, particularly in English, and overlooks the need for cross-lingual evidence retrieval and utilization in real-world applications. To address this issue, we introduce a plug-and-play Retrieval-Augmented Multilingual Citation Generation method (RAMCG) which uses a multilingual retriever to identify relevant evidence from a multilingual knowledge base. The evidence is then combined with the query and processed by a multilingual citation generator. The result is citations that are both accurate and comprehensive. Experiments show that RAMCG outperforms baseline methods in multilingual citation generation and is well-suited for practical use.

The rapid advancement of Large Language ModelS (LLMs) has inaugurated a transformative era in natural language processing, fostering unprecedented capabilities in text generation, understanding, and contextual analysis. However, effectively handling extensive contexts, which are crucial for many applications, remains a significant challenge due to the intrinsic limitations of the context window sizes of the models and the computational burdens associated with their operations. This study proposes an innovative framework that uses unsupervised natural language summarization to provide more efficient context handling. Our methodology is dubbed LONGPC. We demonstrate that our framework significantly reduces computational overhead and enhances LLMs’ performance across various datasets while maintaining or even enhancing the quality of the generated content.

Large Language Models, even when specifically trained to process long input contexts, struggle to capture relevant information located in the middle of their input. This phenomenon is known as the "lost-in-the-middle" problem. In this study, We propose a new method Found In The Distribution (FITD) with Latent Dirichlet Allocation (LDA) to solve the problem of "lost-in-the-middle". Specifically, we first apply the LDA to capture the critical information within the text. Based on the captured results, we generate corresponding probabilities and use a masking method to compress the prompt, enabling LLMs to correctly focus on these important pieces of information, even if they are situated in the middle. Finally, we demonstrate that our approach performs better in identifying pertinent information within long contexts.

Graph Meta-learning methods have improved the performance of few-shot node classification by means of applying meta-learning to the data in non-Euclidean domains. However, most works focus on adopting a single domain, ignoring the fact that tasks in various domains may be distinct, which can cause overfitting problems and thus limit generalizability. To tackle this challenge, we propose a novel Graph Meta-learning framework called Feature-Enhanced Cross-domain Graph Meta-learning that consists of two crucial modules: 1) A feature information extraction module that aims to capture discriminative node importance and simulate various node feature distributions under distinct domains; 2) A heterogeneous graph encoder module that leverages the enhanced node features and topological information to generate task-specific node embeddings with simple fine-tuning. Moreover, we meta-learn the parameters involved to ensure the generalizability in the unseen domains. Results show that our method markedly outperforms the existing state-of-the-art methods.

Currently, deep neural networks (DNNs) are susceptible to adversarial attacks, particularly when the network's structure and parameters are known, while most of the existing attacks do not perform satisfactorily in the presence of black-box settings. In this context, model augmentation is considered to be effective to improve the success rates of black-box attacks on adversarial examples. However, the existing model augmentation methods tend to rely on a single transformation, which limits the diversity of augmented model collections and thus affects the transferability of adversarial examples. In this paper, we first propose the random diversity ensemble method (RDE-MI-FGSM) to effectively enhance the diversity of the augmented model collection, thereby improving the transferability of the generated adversarial examples. Afterwards, we put forward the random diversity variance ensemble method (RDE-VRA-MI-FGSM), which adopts variance reduction augmentation (VRA) to improve the gradient variance of the enhanced model set and avoid falling into a poor local optimum, so as to further improve the transferability of adversarial examples. Furthermore, experimental results demonstrate that our approaches are compatible with many existing transfer-based attacks and can effectively improve the transferability of gradient-based adversarial attacks on the ImageNet dataset. Also, our proposals have achieved higher attack success rates even if the target model adopts advanced defenses. Specifically, we have achieved an average attack success rate of 91.4% on the defense model, which is higher than other baseline approaches.

Cross-domain Graph Meta-learning (CGML) has shown its promise, where meta-knowledge is extracted from few-shot graph data in multiple relevant but distinct domains. However, several recent efforts assume target data available, which commonly does not established in practice. In this paper, we devise a novel Cross-domain Data Augmentation for Graph Meta-Learning (CDA-GML), which incorporates the superiorities of CGML and Data Augmentation， has addressed intractable shortcomings of label sparsity, domain shift, and the absence of target data simultaneously. Specifically, our method simulates instance-level and task-level domain shift to alleviate the cross-domain generalization issue in conventional graph meta-learning. Experiments show that our method outperforms the existing state-of-the-art methods.

Detecting anomalies on attributed graphs is a challenging task since labelled anomalies are highly labour-intensive by taking specialized domain knowledge to make anomalous samples not as available as normal ones. Moreover, graphs contain complex structure information as well as attribute information, leading to anomalies that can be typically hidden in the structure space, attribute space, and the mix of both. In this paper, we propose a novel model for graph anomaly detection named ProGAD. Specifically, ProGAD takes advance of label propagation to infer high-quality pseudo labels by considering the structure and attribute inconsistencies between normal and abnormal samples. Meanwhile, ProGAD introduces the prior knowledge of class distribution to correct and refine pseudo labels with a prototype-aware strategy. Experiments demonstrate that ProGAD achieves strong performance compared with the current state-of-the-art methods.

Researchers have applied knowledge graph embedding (KGE) techniques with advanced neural network techniques, such as capsule networks, for predicting drug-drug interactions (DDIs) and achieved remarkable results. However, most ignore molecular structure and position features between drug pairs. They cannot model the biomedical field's significant relational mapping properties (RMPs,1-N, N-1, N-N) relation. To solve these problems, we innovatively propose CDHse that consists of two crucial modules: 1) Entity embedding module, we obtain position feature obtained by PubMedBERT and Convolutional Neural Network (CNN), obtain molecular structure feature with Graphic Nuaral Network (GNN), obtain entity embedding feature of drug pairs, and then incorporate these features into one synthetic feature. 2) Knowledge graph embedding module, the synthetic feature is Householder projections and then embedded in the complex vector space for training. In this paper, we have selected several advanced models for the DDIs task and performed experiments on three standard BioKG to validate the effectiveness of CDHse.

The rapid development of Knowledge Graph (KG) technology has led to the emergence of Temporal Knowledge Graphs (TKGs), which hold significant research importance and value. Temporal Knowledge Graph Embedding (TKGE) techniques complement TKGs and predict links within them. The efficacy of TKGE hinges upon its capability to effectively model intrinsic temporal relation patterns. However, existing methodologies often need to capture temporal relation patterns adequately or establish intrinsic connections between evolving relations. We propose a highly robust TKGE framework called HouRP to address this limitation and incorporate a more comprehensive range of relational information. This framework introduces two types of Householder transformations into TKGE. Specifically, the Householder projections enable the generation of temporal-specific representations for each entity. At the same time, the relational Householder rotations facilitate high-dimensional rotations between projected entities, thereby capturing relation-specific properties. Our proposed model’s effectiveness is demonstrated through extensive experimentation on four widely used TKGs datasets.

Session-based recommendation (SBR) aims to predict the user’s action at the next timestamp according to an anonymous yet short interaction sequence (i.e., session). Almost all the existing SBR solutions for user preference are only based on the current session without exploiting the high-order relations among other sessions, which may restrict the SBR representation ability and even deteriorate the performance. To this end, we propose a Hyper-relation alignment hyperGraph Convolutional Network, called Hyra-GCN, for better inferring the user preference of the current session. Specifically, we first model session-based data as a hyper-graph capable of representing high-order relationships to exploit item transitions over sessions in a more subtle manner. Subsequently, we explore self-supervised learning on item-session hypergraphs, so as to alleviate the problem of data sparsity. Experimental results on real-world datasets demonstrate the effectiveness of our proposed Hyra-GCN against state-of-the-art baselines.

Temporal Knowledge Graph Embedding (TKGE) is a crucial technique for performing Temporal Knowledge Graph Completion (TKGC). The effectiveness of TKGE largely depends on the ability to model intrinsic relation patterns. However, as we know, most existing TKGE models usually embed KGs into a single geometric space such as Euclidean, hyperbolic or hyperspherical space to maintain their specific geometric structures (e.g., chain, hierarchy, and ring structures). None of the existing methods can simultaneously model relation patterns of chain, hierarchy, ring structures, and relation mapping properties. This paper constructs a hybrid interaction TKGE model HyIE, which learns spatial structures interactively between the Euclidean, hyperbolic and hyperspherical spaces. HyIE performs two Householder transformations of head and tail entities parameterized by relations in a high-dimensional mixed vector space. The curvature of hyperbolic and hyperspherical spaces depends on the product of both relation and temporal. The core of HyIE lies in implementing transformations and interactions of vectors in Euclidean, hyperbolic and hyperspherical spaces, and Household transformation of head and tail entities. Theoretically, HyIE can model crucial relation patterns and mapping properties simultaneously. Experimental results on five temporal knowledge graph benchmarks show that our HyIE achieves state-of-the-art performance.

As the scope of receptive field and the depth of Graph Neural Networks (GNNs) are two completely orthogonal aspects for graph learning, existing GNNs often have shallow layers with truncated-receptive field and far from achieving satisfactory performance. In this article, we follow the idea of decoupling graph convolution into propagation and transformation processes, which generates representations over a sequence of increasingly larger neighborhoods. Though this manner can enlarge the receptive field, it has two critical problems unsolved: how to find the suitable receptive field to avoid under-smoothing or over-smoothing? and how to balance different diffusion operators for better capturing the local and global dependencies? We tackle these challenges and propose a S calable, A daptive G raph C onvolutional N etworks ( SAGCN ) with Transformer architecture. Concretely, we propose a novel non-heuristic metric method that quickly finds the suitable number of diffusing iterations and produces smoothed local embeddings that enable the truncated receptive field to become scalable and independent of prior experience. Furthermore, we devise smooth2seq and diffusion-based position schemes introduced into Transformer architecture for better capturing local and global information among embeddings. Experimental results show that SAGCN enjoys high accuracy, scalability and efficiency on various open benchmarks and is competitive with other state-of-the-art competitors.

Recently, with the development of temporal knowledge graph technology, more and more Temporal Knowledge Graph Embedded (TKGE) models have been developed. The effectiveness of TKGE largely depends on the ability to model intrinsic relation patterns and capture specific information about entities and relations. However, existing approaches can capture only some of them with insufficient modeling capacity, and none has a “deep” architecture for modeling the entries in a quadruple at the same dimension. In this article, we propose a more powerful KGE framework named DuCape , which combines a dual quaternion and capsule network in modeling for the first time to make up for the defects of existing TKGE models. In dual quaternion vector space, the head entity learns a k -dimensional rigid transformation parametrized by relation and time, falling near its corresponding tail entity. Further, we employ the embeddings of entities, relations, and time trained from dual quaternion vector space as the input to capsule networks. Experimental results on several basic datasets show that the DuCape model constructed in this article is superior to existing state-of-the-art models.

The human recognition system has presented the remarkable ability to effortlessly learn novel knowledge from only a few trigger events based on prior knowledge, which is called insight learning. Mimicking such behavior on Knowledge Graph Reasoning (KGR) is an interesting and challenging research problem with many practical applications. Simultaneously, existing works, such as knowledge embedding and few-shot learning models, have been limited to conducting KGR in either “seen-to-seen” or “unseen-to-unseen” scenarios. To this end, we propose a neural insight learning framework named Eureka to bridge the “seen” to “unseen” gap. Eureka is empowered to learn the seen relations with sufficient training triples while providing the flexibility of learning unseen relations given only one trigger without sacrificing its performance on seen relations. Eureka meets our expectation of the model to acquire seen and unseen relations at no extra cost, and eliminate the need to retrain when encountering emerging unseen relations. Experimental results on two real-world datasets demonstrate that the proposed framework also outperforms various state-of-the-art baselines on datasets of both seen and unseen relations.