Reinforcement learning-based knowledge graph reasoning requires complex logical reasoning based on given query relations. Existing methods rely exclusively on delayed reward signals to train the model to perceive the logical reasonableness of the whole reasoning path. This paradigm cannot recognize the logical reasonableness of intermediate reasoning actions, thereby limiting its performance. The logical reasonableness of the same reasoning action changes under different reasoning histories, which makes it difficult for the model to perceive the logical reasonableness of reasoning actions. This paper proposes a Dynamic Logic Alignment-based knowledge graph reasoning method (DLA). DLA dynamically combines the reasoning history and actions, and aligns its logical meaning with the query relation to assess the reasonableness of actions. Firstly, considering that actions have different logical meanings under different history paths, we design a reasoning history-aware dynamic action enhancement mechanism. The mechanism enhances the representation of the current action by injecting the logical composition of history and action at each time step. Secondly, considering that logical composition can provide the reasoning basis for the reinforcement learning agent, we design a query enhancement mechanism for logical composition alignment. The mechanism selects the action with high reasonableness by aligning the logical composition and query relation. Our method has been evaluated on five datasets of different scales, and the experimental results reveal that our method outperforms existing methods.

Online reviews allow consumers to provide detailed feedback on various aspects of items. Existing methods utilize these aspects to model users' fine-grained preferences for specific item features through graph neural networks. We argue that the performance of items on different aspects is important for making precise recommendations, which has not been taken into account by existing approaches, due to lack of data. In this paper, we propose an aspect performance-aware hypergraph neural network (APH) for the review-based recommendation, which learns the performance of items from the conflicting sentiment polarity of user reviews. Specifically, APH comprehensively models the relationships among users, items, aspects, and sentiment polarity by systematically constructing an aspect hypergraph based on user reviews. In addition, APH aggregates aspects representing users and items by employing an aspect performance-aware hypergraph aggregation method. It aggregates the sentiment polarities from multiple users by jointly considering user preferences and the semantics of their sentiments, determining the weights of sentiment polarities to infer the performance of items on various aspects. Such performances are then used as weights to aggregate neighboring aspects. Experiments on six real-world datasets demonstrate that APH improves MSE, Precision@5, and Recall@5 by an average of 2.30%, 4.89%, and 1.60% over the best baseline. The source code and data are available at https://github.com/dianziliu/APH.

Abstract Traditional image retrieval methods often face challenges in adapting to varying user preferences and dynamic datasets. To address these limitations, this research introduces a novel image retrieval framework utilizing deep deterministic policy gradients (DDPG) augmented with a self-adaptive reward mechanism (SARM). The DDPG-SARM framework dynamically adjusts rewards based on user feedback and retrieval context, enhancing the learning efficiency and retrieval accuracy of the agent. Key innovations include dynamic reward adjustment based on user feedback, context-aware reward structuring that considers the specific characteristics of each retrieval task, and an adaptive learning rate strategy to ensure robust and efficient model convergence. Extensive experimentation with the three distinct datasets demonstrates that the proposed framework significantly outperforms traditional methods, achieving the highest retrieval accuracy having 3.38%, 5.26%, and 0.21% improvement overall as compared to the mainstream models over DermaMNIST, PneumoniaMNIST, and OrganMNIST datasets, respectively. The findings contribute to the advancement of reinforcement learning applications in image retrieval, providing a user-centric solution adaptable to various dynamic environments. The proposed method also offers a promising direction for future developments in intelligent image retrieval systems.

Log-based anomaly detection is crucial for software reliability assurance. System logs are semi-structured data containing constant and variable contents, both of which can provide valuable features for anomaly detection. Due to variables being heterogeneous and discrete, there is a lack of effective approaches that can comprehensively incorporate features of variables into log-based anomaly detection. In this paper, we propose VCRLog, an anomaly detection method that mines the relationships among the heterogeneous and discrete variables and extracts important features contributing to anomaly detection. Firstly, considering parsing methods cannot accurately extract variables from logs, we propose a variable extraction method based on domain knowledge. Secondly, to capture and extract the relationship feature among heterogeneous and discrete variables, we design a conceptual model based on system operation to construct variable attributed graph, which can mine important feature vectors by structural embeddings. Finally, considering constants directly express the meaning of logs, we combine relationship vectors with semantic vectors of constants to achieve transformer-based anomaly detection. Experimental results show that our proposed method can accurately detect anomalies and maintain high accuracy as the training data size decreases, outperforming existing methods. Our source code and experimental data are publicly available at https://github.com/Fridaywjy/VCRLog.

Knowledge graphs (KGs) model entities or concepts and their relations in a structural manner. The incompleteness has turned out to be the main challenge that hinders the application of KG. Recently, reinforcement learning (RL) has been recognized as an effective method to deal with such a challenge, which models research tasks into a sequence decision problem without labels. Although an increasing number of studies investigate and analyze knowledge graphs using reinforcement learning, there lacks a systematic literature review that comprehensively and quantitatively analyzes the landscape of RL-based KG research (RL-KG for short). As a result, researchers may have encountered difficulties in appropriately adopting RL techniques in KG research, even reinventing the wheels. In this paper, we follow the Systematic Literature Review (SLR) methodology to survey, screen, and investigate papers of RL-KG. Specifically, we identify 109 highly related papers from 1542, and systematically investigate them with regard to the following five aspects: (1) to what extent RL-KG have been investigated; (2) what application domains have been covered; (3) what RL techniques have been mainly considered; (4) whether there is a connection between the influence and reproducibility of these papers; (5) what specialized datasets, evaluation metrics, and publication venues have been applied. Through an in-depth analysis of the review results, we systematically and comprehensively identify some significant phenomena and analyze the reasons and difficulties of these phenomena. Based on such analysis, we tentatively propose promising future research topics to promote the RL-KG.

Entity alignment (EA) aims to find equivalent entities in knowledge graphs (KGs) from multiple data sources and is a crucial step in integrating KGs. Recent studies learn the similarity of entity embeddings by aggregating neighboring entities. However, these methods solely compare neighboring entities and do not incorporate the connected relation between an entity and its neighbors. In this paper, we propose a novel Entity and Relation joint Interaction Learning (ERIL) approach, which effectively captures the interaction between entities and relations, enhancing the precision of alignment across different KGs. Specifically, the ERIL model jointly learns the neighborhood features of entities and the spatial structure of relations to train a shared permutation matrix, capturing comprehensive associative relations within KGs. Moreover, a semi-supervised iterative framework is designed to leverage the positive interactions between entities and relations to identify more aligned entities. Extensive experiments are conducted on five benchmark datasets to demonstrate the effectiveness of ERIL compared with existing state-of-the-art EA methods. On DBP15K, our model ERIL outperforms currently available EA methods by 1.9% on Hits@10.