Session-based recommendation (SBR) aims to predict anonymous users' next interaction based on their interaction sessions. In practical recommendation scenario, low-exposure items constitute the majority of interactions, creating a long-tail distribution that severely compromises recommendation diversity. Existing approaches attempt to address this issue by promoting tail items but incur accuracy degradation, exhibiting a "see-saw" effect between long-tail and accuracy performance. We attribute such conflict to session-irrelevant noise within the tail item set, which existing long-tail approaches fail to identify and constrain effectively. To resolve our fundamental conflict, we propose HID (Hybrid Intent-based Dual Constraint Framework), a plug-and-play framework that transforms the conventional "see-saw" into a "win-win" relationship through introducing the hybrid intent-based dual constraints. Two key innovations are incorporated in this framework: (i) Hybrid Intent Learning, where we reformulate the intent extraction strategies by employing attribute-aware spectral clustering to reconstruct the item-to-intent mapping. Furthermore, discrimination of session-irrelevant noise is achieved through the assignment of both target and noise intents to each sessions. (ii) Intent Constraint Loss, where we propose two novel constraint paradigms regarding the diversity and accuracy to regulate the representation learning process, and unify the two optimization objectives into a unique loss. Extensive experiments across multiple SBR models and datasets demonstrate that HID can enhance both long-tail performance and recommendation accuracy, establishing new state-of-the-art performance in long-tail recommender systems.

Large language models (LLMs) show promising performance on small-scale graph reasoning tasks but fail when handling real-world graphs with complex queries. This phenomenon arises from LLMs' working memory constraints, which result in their inability to retain long-range graph topology over extended contexts while sustaining coherent multi-step reasoning. However, real-world graphs are often structurally complex, such as Web, Transportation, Social, and Citation networks. To address these limitations, we propose GraphCogent, a collaborative agent framework inspired by human Working Memory Model that decomposes graph reasoning into specialized cognitive processes: sense, buffer, and execute. The framework consists of three modules: Sensory Module standardizes diverse graph text representations via subgraph sampling, Buffer Module integrates and indexes graph data across multiple formats, and Execution Module combines tool calling and tool creation for efficient reasoning. We also introduce Graph4real, a comprehensive benchmark that contains four domains of real-world graphs (Web, Transportation, Social, and Citation) to evaluate LLMs' graph reasoning capabilities. Our Graph4real covers 21 different graph reasoning tasks, categorized into three types (Structural Querying, Algorithmic Reasoning, and Predictive Modeling tasks), with graph scales up to 10 times larger than existing benchmarks. Experiments show that Llama3.1-8B based GraphCogent achieves a 50% improvement over massive-scale LLMs like DeepSeek-R1 (671B). Compared to state-of-the-art code-based baseline, our framework outperforms by 20% in accuracy while reducing token usage by 80% for in-toolset tasks and 30% for out-toolset tasks.

Dataset distillation (DD) has demonstrated the promise of synthesizing smaller datasets that enable competitive performance. While most of DD methods operate in pixel space, they often suffer from poor scalability on high-resolution datasets. Recent works have thus shifted towards parameterizing synthetic data using deep generative priors. However, these approaches apply latent updates to all generator layers, leading to substantial computational overhead. To address this limitation, we propose Generative Lightweight Distillation (GLiD), a unified framework that jointly compresses the generator and accelerates latent optimization for efficient distillation. GLiD introduces two key components: (1) We introduce a Classification-Diversity Sensitivity Pruning mechanism that quantifies both discriminative utility and semantic diversity of each output channel to guide structural pruning and selective layer-wise optimization. (2) We also present a Layer-Adaptive Scheduling strategy that dynamically allocates latent update steps across generator stages based on convergence behavior. Extensive experiments on CIFAR-10 and ImageNet-1K and its subsets demonstrate that our GLiD achieves up to 10× acceleration across different datasets, while maintaining performance competitive with state-of-the-art methods.

Session-based recommendation (SBR) predicts the next interaction of anonymous users based on their session data. In practical scenarios, low-exposure tail items constitute the majority of interactions, leading to the biased recommendation whereby a small set of high-exposure head items is preferentially recommended. Existing solutions focus on promoting those tail items but with a sacrifice in recommendation accuracy. We attribute such a tradeoff to session-irrelevant noise within the tail items, which existing long-tail approaches fail to identify and constrain effectively. To resolve this fundamental conflict, we propose Hybrid Intent-based Dual Constraint framework (HID) , a plug-and-play framework that converts the traditional tradeoff between long-tail performance and accuracy into a synergistic relationship by introducing hybrid intent-based dual constraints. HID incorporates two key innovations: (i) Hybrid Intent Learning , which reformulates intent extraction through attribute-aware spectral clustering and discriminates session-irrelevant noise by assigning target and noise intents to each session and (ii) Intent Constraint Loss , which introduces two constraints—diversity and accuracy—to regulate item and session representation learning. These two objectives are unified into a single training loss through rigorous theoretical derivation. To handle ambiguous user intent in the real-world, we also propose a slack version of HID. Extensive experiments show that HID enhances both long-tail performance and accuracy, establishing new state-of-the-art performance in long-tail recommender systems. The implementation code is available at: https://github.com/jarviswww/Code4TradeSynergy .

Protein language models often struggle to capture biological functions due to their lack of factual knowledge (e.g., gene descriptions). Existing solutions leverage protein knowledge graphs (PKGs) as auxiliary pre-training objectives, but lack explicit integration of task-oriented knowledge, making them suffer from limited knowledge exploitation and catastrophic forgetting. The root cause is that they fail to align PKGs with task-specific data, forcing their knowledge modeling to adapt to the knowledge-isolated nature of downstream tasks. In this paper, we propose Knowledge-aware retrieval augmented protein language model (Kara), achieving the first task-oriented and explicit integration of PKGs and protein language models. With a knowledge retriever learning to predict linkages between PKG and task proteins, Kara unifies the knowledge integration of the pre-training and fine-tuning stages with a structure-based regularization, mitigating catastrophic forgetting. To ensure task-oriented integration, Kara uses contextualized virtual tokens to extract graph context as task-specific knowledge for new proteins. Experiments show that Kara outperforms existing knowledge-enhanced models in 6 representative tasks, achieving on average 5.1% improvements.

Large language models (LLMs) have been demonstrated to possess the capabilities to understand fundamental graph properties and address various graph reasoning tasks. Existing methods fine-tune LLMs to understand and execute graph reasoning tasks by specially designed task instructions. However, these Text-Instruction methods generally exhibit poor performance. Inspired by tool learning, researchers propose Tool-Instruction methods to solve various graph problems by special tool calling (e.g., function, API and model), achieving significant improvements in graph reasoning tasks. Nevertheless, current Tool-Instruction approaches focus on the tool information and ignore the graph structure information, which leads to significantly inferior performance on small-scale LLMs (less than 8B). To tackle this issue, we propose GraphTool-Instruction, an innovative Instruction-tuning approach that decomposes the graph reasoning task into three distinct subtasks (i.e., graph extraction, tool name identification and tool parameter extraction), and design specialized instructions for each subtask. Our GraphTool-Instruction can be used as a plug-and-play prompt for different LLMs without fine-tuning. Moreover, building on GraphTool-Instruction, we develop GTools, a dataset that includes twenty graph reasoning tasks, and create a graph reasoning LLM called GraphForge based on Llama3-8B. We conduct extensive experiments on twenty graph reasoning tasks with different graph types (e.g., graph size or graph direction), and we find that GraphTool-Instruction achieves SOTA compared to Text-Instruction and Tool-Instruction methods. Fine-tuned on GTools, GraphForge gets further improvement of over 30% compared to the Tool-Instruction enhanced GPT-3.5-turbo, and it performs comparably to the high-cost GPT-4o. Our codes and data are available at https://github.com/RongzhengWang/GraphTool-Instruction.

Recently, prompt tuning has garnered considerable attention due to its success across various Vision-Language (VL) tasks. However, unimodal prompts, coupled prompts, and joint prompts in these models often lead to suboptimal performance due to differences in information density and complexity between modalities. Particularly, in scenarios with missing modalities, these prompt-based approaches tend to exacerbate 'Channel Bias'-a phonomenon where models overly rely on specific feature (such as unmissing-modal feature) channels from the base tasks, thereby undermining the model's ability to capture crucial shared knowledge applicable to new tasks and affecting its generalizability. To address this challenge, we propose RobustPT, a dynamic disentanglement prompt tuning model designed to enhance the robustness of VL models under modality missing conditions. RobustPT utilizes a multi-channel prompting mechanism to dynamically disentangle and align prompts. Specifically, RobustPT is divided into single-channel tuning and alignment-channel tuning, where prompts for each modality run independently in sequence to delve deeply into their intrinsic characteristics, followed by an integration through a non-strong coupling strategy to effectively balance information contributions and enhance overall performance. Extensive experiments demonstrate that our RobustPT achieve significant improvements over the current state-of-the-art across all benchmark datasets. Our codes are available at https://github.com/Trae1ounG/RobustPT.

The growing demand for efficient deep learning has positioned dataset distillation as a pivotal technique for compressing training dataset while preserving model performance. However, existing inner-loop optimization methods for dataset distillation typically rely on random truncation strategies, which lack flexibility and often yield suboptimal results. In this work, we observe that neural networks exhibit distinct learning dynamics across different training stages—early, middle, and late—making random truncation ineffective. To address this limitation, we propose Automatic Truncated Backpropagation Through Time (AT-BPTT), a novel framework that dynamically adapts both truncation positions and window sizes according to intrinsic gradient behavior. AT-BPTT introduces three key components: (1) a probabilistic mechanism for stage-aware timestep selection, (2) an adaptive window sizing strategy based on gradient variation, and (3) a low-rank Hessian approximation to reduce computational overhead. Extensive experiments on CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet-1K show that AT-BPTT achieves state-of-the-art performance, improving accuracy by an average of 6.16\% over baseline methods. Moreover, our approach accelerates inner-loop optimization by 3.9 × while saving 63\% memory cost.

Retrieval-augmented generation (RAG) greatly enhances large language models (LLMs) performance in knowledge-intensive tasks. However, naive RAG methods struggle with multi-hop question answering due to their limited capacity to capture complex dependencies across documents. Recent studies employ graph-based RAG to capture document connections. However, these approaches often result in a loss of semantic coherence and introduce irrelevant noise during node matching and subgraph construction. To address these limitations, we propose NeuroPath, an LLM-driven semantic path tracking RAG framework inspired by the path navigational planning of place cells in neurobiology. It consists of two steps: Dynamic Path Tracking and Post-retrieval Completion. Dynamic Path Tracking performs goal-directed semantic path tracking and pruning over the constructed knowledge graph (KG), improving noise reduction and semantic coherence. Post-retrieval Completion further reinforces these benefits by conducting second-stage retrieval using intermediate reasoning and the original query to refine the query goal and complete missing information in the reasoning path. NeuroPath surpasses current state-of-the-art baselines on three multi-hop QA datasets, achieving average improvements of 16.3\% on recall@2 and 13.5\% on recall@5 over advanced graph-based RAG methods. Moreover, compared to existing iter-based RAG methods, NeuroPath achieves higher accuracy and reduces token consumption by 22.8\%. Finally, we demonstrate the robustness of NeuroPath across four smaller LLMs (Llama3.1, GLM4, Mistral0.3, and Gemma3), and further validate its scalability across tasks of varying complexity. Code is available at [https://github.com/KennyCaty/NeuroPath](https://github.com/KennyCaty/NeuroPath).

While large language models (LLMs) show considerable promise across various fields, they have notable limitations in handling multi-document question answering (Multi-doc QA) tasks. The first challenge is long-range dependency modeling, where LLMs struggle to focus on key information in long texts, which weakens important semantic connections. Second, most LLMs suffer from the ''lost-in-the-middle'' issue, where they have difficulty processing information in the middle of long inputs. Current solutions either truncate global dependencies or demand costly finetuning, ultimately lacking a universal and simple solution for these challenges. To resolve these limitations, we propose Dual-Stage Adaptive Sharpening (DSAS) containing two modules. (i) The Contextual Gate Weighting (CGW) module alleviates ''lost-in-the-middle'' by assessing paragraph relevance through layer-wise attention tracking and position-aware weighting. (ii) The Reciprocal Attention Suppression (RAS) module enhances focus on critical paragraphs by suppressing information exchange between key and irrelevant texts, thus mitigating the limitations in long-range dependency modeling. Extensive experiments on four benchmarks demonstrate DSAS's efficacy across mainstream LLMs (Llama, Qwen, Mistral, and Deepseek), with an average F1-score improvement of 4.2% in Multi-doc QA tasks on Llama-3.1-8B-Instruct and Qwen2.5-14B-Instruct. Ablation studies confirm the essential contributions of both the CGW and RAS modules. In addition, detailed discussions in the Appendix further validate the robustness and scalability of DSAS.

Abstract Session-based recommendation (SBR) focuses on predicting the next potential item for anonymous users based on short-click sessions. However, these interaction sessions often contain noise items, which arise from misclicks or shifts in user interests. Existing denoising methods typically presume a strong exclusionary relationship between noise items and the recommendation target, assuming that reducing noise can enhance recommendation accuracy. In contrast, our observations reveal a nuanced phenomenon: as the length of the interaction session shortens, the effect of noise removal on recommendation performance gradually transitions from positive to negative. This finding suggests that in short sessions with insufficient contextual information, relying solely on the exclusion of noise items within the session may fail to improve and could even hinder-the recommendation performance. Such complexities have been largely overlooked in prior research. To bridge this gap, we propose two solutions: (i) expanding the view of denoising from a single session to multiple sessions (i.e., from local to global), and (ii) introducing relevant contextual information into each session by employing enhancement strategies. Therefore, we design the Hybrid Prototype-based In-and-Out Flow Network (HyPro), which employs both denoising and enhancing processes for each session based on our proposed hybrid prototypes. Specifically, for each item, HyPro first learns the hybrid prototype by aggregating information from the item’s semantic and topological neighbors across all sessions. Then, based on the hybrid prototypes, HyPro employs an in-and-out flow network comprising two components: (i) the out-flow channel, which targets the removal of irrelevant information at both the data and feature levels, and (ii) the in-flow channel, which integrates global information for each session at the item and session levels. Extensive experiments conducted on three real-world datasets demonstrate that HyPro outperforms the state-of-the-art baselines. The implementation code is available at: https://github.com/jarviswww/Code4HyPro .

Recent advancements in language models have greatly improved dialogue systems, but they still face challenges in generating personalized responses that are consistent with the user’s persona and dialogue context. Existing approaches typically model dialogue context and persona information together in a unified manner, but they lack fine-grained differentiation between the two, leading to inaccurate user modeling. This misalignment hinders the ability of dialogue systems to produce personalized responses. In this work, we propose cross-graph knowledge exchange (CKE), a novel algorithm designed to enhance personalized response generation in dialogue systems. CKE constructs separate dialogue user graphs for each party in the dialogue, representing both their dialogue context and persona information. These graphs are then utilized to perform cross-graph structured knowledge aggregation, where the aggregation is under supervised by both its own and the other party’s persona and dialogue context, providing richer, more accurate representations. Furthermore, CKE introduces a hybrid prompt template that combines both discrete and continuous elements, improving the language model’s ability to leverage graph-structured information. The experimental results demonstrate that CKE significantly outperforms existing baseline methods in generating more coherent, contextually appropriate, and personalized responses in dialogue systems.

Predicting missing facts for temporal knowledge graphs (TKGs) is a fundamental task, called temporal knowledge graph completion (TKGC). One key challenge in this task is the imbalance in data distribution, where facts are unevenly spread across entities and timestamps. This imbalance can lead to poor completion performance for long-tail entities and timestamps, and unstable training due to the introduction of false negative samples. Unfortunately, few previous studies have investigated how to mitigate these effects. Moreover, for the first time, we found that existing methods suffer from model preferences, revealing that entities with specific properties (e.g., recently active) are favored by different models. Such preferences will lead to error accumulation and further exacerbate the effects of imbalanced data distribution. To alleviate the impacts of imbalanced data and model preferences, we introduce Booster , the first data augmentation strategy for TKGs. The unique requirements here lie in generating new samples that fit the complex semantic and temporal patterns within TKGs, and identifying hard-learning samples specific to models. Therefore, we propose a hierarchical scoring algorithm based on triadic closures within TKGs. By incorporating both global semantic patterns and local time-aware structures, the algorithm enables pattern-aware validation for new samples. Meanwhile, we propose a two-stage training approach to identify samples that deviate from the model's preferred patterns. With a frequency-based filtering strategy, this approach also helps to avoid the misleading of false negatives. Experiments justify that Booster can seamlessly adapt to existing TKGC models and achieve on average 4.5% performance improvement.

Multimodal knowledge graphs (MMKGs) have gained widespread adoption across various domains. However, existing transformer-based methods for MMKG representation learning primarily focus on enhancing representation performance, while overlooking time and memory costs, which reduces model efficiency. To tackle these limitations, we introduce a multimodal lightweight transformer (MLFormer) model, which not only ensures robust representation capabilities but also considerably improves computational efficiency. We find that the self-attention mechanism in transformers leads to substantial performance overheads. As a result, we optimize the traditional MMKGE model in two aspects: modality processing and modality fusion, by incorporating a filter gate and Fourier transform. Our experimental results on real-world multimodal knowledge graph completion datasets demonstrate that MLFormer achieves significant improvements in computational efficiency while maintaining competitive performance.

Offline meta-reinforcement learning (meta-RL) methods, which adapt to unseen target tasks with prior experience, are essential in robot control tasks. Current methods typically utilize task contexts and skills as prior experience, where task contexts are related to the information within each task and skills represent a set of temporally extended actions for solving subtasks. However, these methods still suffer from limited performance when adapting to unseen target tasks, mainly because the learned prior experience lacks generalization, i.e., they are unable to extract effective prior experience from meta-training tasks by exploration and learning of continuous latent spaces. We propose a framework called decoupled meta-reinforcement learning (DCMRL), which (1) contrastively restricts the learning of task contexts through pulling in similar task contexts within the same task and pushing away different task contexts of different tasks, and (2) utilizes a Gaussian quantization variational autoencoder (GQ-VAE) for clustering the Gaussian distributions of the task contexts and skills respectively, and decoupling the exploration and learning processes of their spaces. These cluster centers which serve as representative and discrete distributions of task context and skill are stored in task context codebook and skill codebook, respectively. DCMRL can acquire generalizable prior experience and achieve effective adaptation to unseen target tasks during the meta-testing phase. Experiments in the navigation and robot manipulation continuous control tasks show that DCMRL is more effective than previous meta-RL methods with more generalizable prior experience.

Commonsense question answering has demonstrated considerable potential across various applications like assistants and social robots.Although fully fine-tuned Pre-trained Language Model(PLM) has achieved remarkable performance in commonsense reasoning, their tendency to excessively prioritize textual information hampers the precise transfer of structural knowledge and undermines interpretability.Some studies have explored combining Language Models (LM) with Knowledge Graphs (KGs) by coarsely fusing the two modalities to perform Graph Neural Network (GNN)-based reasoning that lacks a profound interaction between heterogeneous modalities.In this paper, we propose a novel Graph-based Structure-Aware Prompt Learning Model for commonsense reasoning, named G-SAP, aiming to maintain a balance between heterogeneous knowledge and enhance the cross-modal interaction within the LM+GNNs model.In particular, an evidence graph is constructed by integrating multiple knowledge sources, i.e.ConceptNet, Wikipedia, and Cambridge Dictionary to boost the performance.Afterward, a structure-aware frozen PLM is employed to fully incorporate the structured and textual information from the evidence graph, where the generation of prompts is driven

Dynamic text-attributed graphs (DyTAGs) are prevalent in various real-world scenarios, where each node and edge are associated with text descriptions, and both the graph structure and text descriptions evolve over time. Despite their broad applicability, there is a notable scarcity of benchmark datasets tailored to DyTAGs, which hinders the potential advancement in many research fields. To address this gap, we introduce Dynamic Text-attributed Graph Benchmark (DTGB), a collection of large-scale, time-evolving graphs from diverse domains, with nodes and edges enriched by dynamically changing text attributes and categories. To facilitate the use of DTGB, we design standardized evaluation procedures based on four real-world use cases: future link prediction, destination node retrieval, edge classification, and textual relation generation. These tasks require models to understand both dynamic graph structures and natural language, highlighting the unique challenges posed by DyTAGs. Moreover, we conduct extensive benchmark experiments on DTGB, evaluating 7 popular dynamic graph learning algorithms and their variants of adapting to text attributes with LLM embeddings, along with 6 powerful large language models (LLMs). Our results show the limitations of existing models in handling DyTAGs. Our analysis also demonstrates the utility of DTGB in investigating the incorporation of structural and textual dynamics. The proposed DTGB fosters research on DyTAGs and their broad applications. It offers a comprehensive benchmark for evaluating and advancing models to handle the interplay between dynamic graph structures and natural language. The dataset and source code are available at https://github.com/zjs123/DTGB.

The representation of semantic information pertaining to the real world has been active research for some time now. Among the available methods, knowledge graphs have emerged as a widely accepted approach. Meanwhile, graph neural networks (GNNs) have demonstrated excellent performance in embedding graph-based information. Given the natural graph structure of knowledge graphs, employing GNNs to embed them is expected to yield a more interpretable and trustworthy representation of the learned knowledge. In this paper, we propose three customized GNNs for different scenarios of knowledge graph representation, including traditional, multimodal, and uncertain knowledge graphs. In the traditional knowledge graph scenario, we present a graph self-supervised learning method, named deep relation graph infomax (DRGI), which incorporates both the complete graph structure information and semantic information. In the multimodal knowledge graph scenario, we introduce a novel network, named hyper-node relational graph attention network (HRGAT), which combines different modal information with graph structure information for a more precise representation of multimodal knowledge graphs. In the uncertain knowledge graph scenario, we define a novel message-passing paradigm with box embedding, named box graph neural network (BGNN). BGNN leverages both the graph structure information of uncertain knowledge graphs and the probabilistic semantics of box embedding. To validate the effectiveness of our proposed methods, we conduct a series of experiments and report the results. We also discuss possible future work in GNN-based knowledge graph embedding.

Knowledge graph (KG) enhanced recommendation, which aims to solve the cold start and explainability in recommender systems, has attracted considerable research interest recently. Existing recommender systems usually focus on implicit feedback such as purchase history without negative feedback. Most of them apply the negative sampling strategy to deal with the implicit feedback data, which may ignore the latent positive user-item interaction. Some other works adopt the non-sampling strategy that treats all non-observed interactions as negative samples and assigns a weight for each negative sample to represent the probability that this sample is a positive sample. However, they use a simple and intuitive weight assignment strategy and cannot catch the latent relationship from all interaction data. To address these problems, we consider graph structure information of both user-item interaction and knowledge graph, and propose a Graph-based Non-Sampling strategy to achieve efficient performance in Knowledge graph enhanced Recommendation (GNSKR). GNSKR utilizes node centrality to significantly improve recommendation performance with low computation cost. Meanwhile, we combine knowledge graph embedding and recommendation task with a local aggregation block, which efficiently catches the high-order connection information in KG enhanced recommendation. Experiments on three datasets show that GNSKR embraces the state-of-the-art with competitive efficiency.

Recently, many knowledge graph embedding models for knowledge graph completion have been proposed, ranging from the initial translation-based models such as TransE to recent convolutional neural network (CNN) models such as ConvE. However, these models only focus on semantic information of knowledge graph and neglect the natural graph structure information. Although graph convolutional network (GCN)-based models for knowledge graph embedding have been introduced to address this issue, they still suffer from fact incompleteness, resulting in the unconnectedness of knowledge graph. To solve this problem, we propose a novel model called deep relational graph infomax (DRGI) with mutual information (MI) maximization which takes the benefit of complete structure information and semantic information together. Specifically, the proposed DRGI consists of two encoders which are two identical adaptive relational graph attention networks (ARGATs), corresponding to catching semantic information and complete structure information respectively. Our method establishes new state-of-the-art on the standard datasets for knowledge graph completion. In addition, by exploring the complete structure information, DRGI embraces the merits of faster convergence speed over existing methods and better predictive performance for entities with small indegree.

Knowledge graphs often suffer from incompleteness, and knowledge graph completion (KGC) aims at inferring the missing triplets through knowledge graph embedding from known factual triplets. However, most existing knowledge graph embedding methods only use the relational information of knowledge graph and treat the entities and relations as IDs with simple embedding layer, ignoring the multi-modal information among triplets, such as text descriptions, images, etc. In this work, we propose a novel network to incorporate different modal information with graph structure information for more precise representation of multi-modal knowledge graph, termed as hyper-node relational graph attention (HRGAT) network. In HRGAT, we use low-rank multi-modal fusion to model the intra-modality and inter-modality dynamics, which transforms the original knowledge graph to a hyper-node graph. Then, relational graph attention (RGAT) network is used, which contains relation-specific attention and entity-relation fusion operation to capture the graph structure information. Finally, we aggregate the updated multi-modal information and graph structure information to generate the final embeddings of knowledge graph to achieve KGC. By exploring multi-modal information and graph structure information, HRGAT embraces faster convergence speed and achieves the state-of-the-art for KGC on the standard datasets. Implementation code is available at https://github.com/broliang/HRGAT.

Recently, many knowledge graph embedding models for knowledge graph completion have been proposed, ranging from the initial translation-based models such as TransE to recent convolutional neural network (CNN) models such as ConvE. However, these models only focus on semantic information of knowledge graph and neglect the natural graph structure information. Although graph convolutional network (GCN)-based models for knowledge graph embedding have been introduced to address this issue, they still suffer from fact incompleteness, resulting in the unconnectedness of knowledge graph. To solve this problem, we propose a novel model called deep relational graph infomax (DRGI) with mutual information (MI) maximization which takes the benefit of complete structure information and semantic information together. Specifically, the proposed DRGI consists of two encoders which are two identical adaptive relational graph attention networks (ARGATs), corresponding to catching semantic information and complete structure information respectively. Our method establishes new state-of-the-art on the standard datasets for knowledge graph completion.